diff --git a/content/blog/ffxiv-may2025/astra.webp b/content/blog/ffxiv-may2025/astra.webp new file mode 100644 index 0000000..7ce597a Binary files /dev/null and b/content/blog/ffxiv-may2025/astra.webp differ diff --git a/content/blog/ffxiv-may2025/delete-files.webp b/content/blog/ffxiv-may2025/delete-files.webp new file mode 100644 index 0000000..7073ccf Binary files /dev/null and b/content/blog/ffxiv-may2025/delete-files.webp differ diff --git a/content/blog/ffxiv-may2025/edit-password.webp b/content/blog/ffxiv-may2025/edit-password.webp new file mode 100644 index 0000000..b8cbbc4 Binary files /dev/null and b/content/blog/ffxiv-may2025/edit-password.webp differ diff --git a/content/blog/ffxiv-may2025/excel-languages.webp b/content/blog/ffxiv-may2025/excel-languages.webp new file mode 100644 index 0000000..a1be866 Binary files /dev/null and b/content/blog/ffxiv-may2025/excel-languages.webp differ diff --git a/content/blog/ffxiv-may2025/existing-install.webp b/content/blog/ffxiv-may2025/existing-install.webp new file mode 100644 index 0000000..4e019f7 Binary files /dev/null and b/content/blog/ffxiv-may2025/existing-install.webp differ diff --git a/content/blog/ffxiv-may2025/index.md b/content/blog/ffxiv-may2025/index.md index 283e101..268ad96 100644 --- a/content/blog/ffxiv-may2025/index.md +++ b/content/blog/ffxiv-may2025/index.md @@ -1,7 +1,7 @@ --- title: "Astra & Novus Status Update: May 2025" date: 2025-05-06 -draft: true +draft: false tags: - Open Source - FFXIV @@ -13,63 +13,88 @@ series: Sooo... it's been a while since I wrote one of these. We have a years worth of progress to catch up on, so we'd better begin! -If this is your first time hearing any of this, [Astra]() is my unofficial FFXIV launcher that supports Linux[^1]. It supports launching Dalamud, can run as a Steam Compatibility Tool, and supports custom servers like [Kawari](): +If this is your first time hearing any of any of these projects, [Astra](https://github.com/redstrate/Astra) is my unofficial FFXIV launcher that supports Linux[^1]. It launches Dalamud, can run as a Steam Compatibility Tool, and supports alternative servers like [Kawari](https://github.com/redstrate/Kawari): -Novus is my [unofficial FFXIV tool suite]() which includes programs to view model files, edit gear and a lot more. I got frustrated years ago that almost none of the pre-existing FFXIV modding tooling worked on Linux, so I made my own: +![Screenshot of Astra on the login screen.](astra.webp) -Updates to my Kawari project are happening in another [series of blog posts], because the scope of that project is large enough on it's own! +Novus is my [unofficial FFXIV tool suite](https://github.com/redstrate/Novus) which includes programs to view model files, edit gear and a lot more. I got frustrated years ago that (almost) none of the pre-existing FFXIV modding tooling ran on Linux, so I made my own: + +![Older (but great) screenshot of the various Novus applications.](novus.webp) + +Updates to my Kawari alternative server project are happening in another [series of blog posts](/blog/series/kawari-progress-report), because the scope of that project is large enough on it's own! These status updates also serves as a place to discuss my other, smaller FFXIV projects that aren't interesting enough to live in their own blog post. # Astra ## Initial Setup -I have redone the initial setup, making the entire process clearer: +I have redone the initial setup again, and the entire process should be cleaner and easier now. There are three major "flows" you can take: Install the game, install the benchmark, or import from another launcher: -Astra now supports importing an existing game install from another launcher: +![Screenshot of the new initial setup screen in Astra.](initial-setup.webp) + +Astra now supports importing an existing game install from another launcher, such as from XivLauncher.Core: + +![Screenshot of the "Select Existing Installation" screen in Astra.](existing-install.webp) + +There is a limited amount of launchers it supports, [patches are welcome](https://github.com/redstrate/Miscel/blob/main/src/existing_dirs.rs)! This same logic can be re-used for other launchers too, and I should probably integrate it into Novus sometime. ## Developer Settings -Advanced Dalamud settings are now hidden behind the Developer Settings toggle, because they should rarely (if ever) be touched. +Advanced Dalamud settings are now hidden behind the "Developer Settings" toggle, because they should rarely (if ever) be touched. Here is how it looks when they're disabled: -A new button has been added to the Developer Settings to launch the game immediately, which been more useful to me than you'd think: +![The simpler Dalamud settings page.](simple-dalamud-settings.webp) -If you develop Dalamud, there's now an easier way to use your local dev build in Astra. Plop it into `~/.local/share/astra/dalamud/local` and select the "Local" option for "Dalamud Channel" in your profile settings. This also skips all update and version checks for Dalamud. +A new button has been added to the Developer Settings to launch the game immediately, which has been more useful to me on more than one occasion! -Most custom server settings are now moved to be per-account, so you can have an account for an alternative server and still play on your Square Enix account easily. +If you develop Dalamud, there's now an easier way to use a local build in Astra. Plop it into `~/.local/share/astra/dalamud/local` (or the Flatpak equivalent) and select the "Local" option for "Dalamud Channel" in your profile settings. This also skips all update and version checks for Dalamud. -You can now specify a custom lobby and game master server under developer settings, useful for connecting to any of the various alternative server projects. +You can now specify custom servers under your account's developer settings, and Astra can now function without being connected to any of SqEx's servers. I made these per-account so I can have my test Kawari account and still play with my real Square Enix account. + +![The server settings page under Account settings.](server-settings.webp) + +You can now specify the platform, boot and game update channels inside of your profile's developer settings: + +![The update settings page under Profile settings.](update-channel.webp) ## Misc UX The password field is now writeable again, and you can also click a button to view your password: -It's now possible to update the game even if the worlds are closed for maintenance. +![Thanks to Kirigami, which has a very nice PasswordField.](edit-password.webp) -Before Astra starts patching, it checks if you have enough disk space - not only to download the update but also to install it. This should prevent some nasty scenarios when updating the game. +Before Astra starts patching, it checks if you have enough disk space - not only to download the update but also to install it. This should prevent some nasty scenarios while patching: -I have added back the "Tools" menu that was lost in the QML transition, so you can access the Official Launcher again if needed. +![Error dialog if there's not enough space to download the patches.](update-space.webp) -When removing a profile, we now provide an option to delete the game files too. +On a related note, Astra will now warn you before it starts updating. This is to help prevent "gotchas" like your installation being considered invalid, profiles you weren't supposed to update or you're on a connection with limited data: -I added an option to disable updates. Useful if you're like me, and need to keep the game at a specific version. +![Confirmation dialog if you still want to update.](update-confirm.webp) + +I have added back the "Tools" menu that was lost during the QML transition, so you can access the Official Launcher and other tools if needed: + +![The tools menu, located in the top-right of the main page.](tools.webp) + +When deleting a profile, there's now an option to delete the game files as well: + +![Confirmation dialog when deleting a profile.](delete-files.webp) ## Bugfixes -Patches are no longer kept in memory, which was proven to be problematic on systems like the Steam Deck. This greatly improves patch stability for everyone, however! +Here's a rundown list of some of interesting bugfixes: -If Dalamud reports that it's incompatible with your game version, it should disable itself automatically instead of crashing. +* It's now possible to update the game even if the worlds are closed for maintenance. +* Patches are no longer kept in memory, which was problematic for systems like the Steam Deck. +* When Dalamud reports that it's incompatible with your game version, it disables itself now instead of crashing on launch. +* In the rare case where Astra does fail to apply a patch - like if the download was corrupted - an error is shown to the user and the errant patch is deleted: +* The method Astra uses to track if the game is open (e.g. for keeping your computer from sleeping) is now more reliable on Linux when Dalamud is enabled. +* The Steam Compatibility Tool should be much more reliable now in general! -In the rare case where Astra does fail to apply a patch - like if the download was corrupted, (or a genuine bug in Physis) - an error is shown to the user and the errant patch is deleted: - -The method Astra uses to track if the game is open (e.g. for keeping your computer from sleeping) is now way more reliable on Linux when running with Dalamud. - -## Removals +## Feature Removals I have done several feature removals to claw back the feature creep: -* Sapphire support is removed. I only used this feature _once_ (if ever?) and I kept breaking it. -* Feral Gamemode support is removed. I don't use it anymore, and I have an alternative solution up my sleeve if I want to add it back. -* Gamescope support is removed. I originally added this to workaround various Wine bugs when it runs under XWayland. But now that they have a native Wayland backend and even the XWayland bugs are gone, there's very little reason to keep it. -* The news is always shown, and the option to disable it has been removed. +* Sapphire support is removed. I only used this feature _once_ (if ever?) and I kept breaking it. There are several other launchers that support Sapphire's account system you can use instead. +* Feral Gamemode support is removed. I just didn't use it anymore. +* Gamescope support is removed. I originally added this to workaround bugs when the game was running under XWayland. But now that we have a native Wine on Wayland (and even the XWayland bugs are gone) I didn't see a reason to keep it. +* The news on the main login page is now always shown, and the option to hide it was removed. ## Steam Service Accounts @@ -80,86 +105,134 @@ One of the last remaining features to implement was Steam service account suppor I'm happy to report I solved all of those problems, and Astra can now login to Steam service accounts just fine. There is one caveat: it's only supported when running as a Steam compatibility tool for now. +Delving into the technical tidbits, [Miscel has a Blowfish implementation you can use for problem #2](https://github.com/redstrate/Miscel/blob/main/src/blowfish/mod.rs) (the Steam Ticket is further "encrypted" with the process [done here](https://github.com/goatcorp/FFXIVQuickLauncher/blob/master/src/XIVLauncher.Common/Encryption/Ticket.cs). Astra simply has a C++ version of that.) Problem #3 is done using [Steamwrap](https://code.ryne.moe/redstrate/steamwrap). I'm unsure how OK it is to redistribute the Steam API headers freely, so I decided to compile a separate application that I build myself and pack into the Flatpak. Astra communicates with this daemon through HTTP, and it might be useful to someone else too. + ## Nightly Flatpaks -Are you interested in trying out Astra after all of these changes? If so, you can grab a nightly Flatpak from GitHub Actions. I will eventually update these to my server if you don't have a GitHub account. +Are you interested in trying out Astra? If you have a GitHub account, you can grab a nightly Flatpak [from GitHub Actions](https://github.com/redstrate/Astra/actions). I'm also preparing to release this properly on Flathub, so any testing of the Flatpak is appreciated. (I personally have switched over to only using the Flatpak version now.) # Novus -## Game Launcher - -Added a new application for launching the game, it's very simple right now though. - ## Gear Editor -Added various Penumbra actions to the Armour menubar. +Added various Penumbra actions to the Armour menubar, like redrawing the models and opening the window: -## Game Renderer +![The new Penumbra menu in the Gear Editor.](penumbra-actions.webp) -Debug markers have been added to the game renderer to aid debugging graphical issues. - -I'm still working on Dawntrail support for the game renderer, which isn't complete yet but looks less broken now. +Redrawing is also called automatically when importing models! ## Excel Editor -You can switch between any available language in an Excel sheet. I'm stupid and CJK characters are not shown correctly yet. +You can switch between any available language in an Excel sheet. I'm stupid and CJK characters are not shown correctly yet, but other languages work fine: + +![The language selector in the Excel Editor.](excel-languages.webp) + +(Apart from SeString support, that's also something I need to work on.) ## Data Explorer -I added a status bar to the bottom of Data Explorer, listing various useful information about the currently selected file or folder. +As you may have seen in my Kawari progress reports, I have been delving into how the game's event system works. To aid in this effort, I added a Lua decompiler to the Data Explorer which makes exploring *way* easier: -As you may have seen in my Kawari updates, I have been busy delving into how the game's event system works. To aid in this effort, I added a Lua decompiler to the Data Explorer which makes exploring *way* easier. +![Showcasing the (rough but useful) decompiled output of the Gridania Inn script.](lua-decompiler.webp) + +I also have improved the import process from [ResLogger2](https://rl2.perchbird.dev/), it should import *a lot* faster now. + +# Auracite + +[Auracite](https://auracite.xiv.zone) is my character backup tool that runs in your web browser, and can optionally connect to a Dalamud plugin. If you're wondering how it's useful, check out my [FFXIV Shrine](/shrines/ffxiv). + +The following data is now fetched from the Lodestone: +* Free company name +* Currently equipped item names +* Grand company name and rank + +And the following data is now fetched when connected to the Dalamud plugin: +* Voice + +The following data now has numerical values alongside English text, to ease the difficulty in building tools on top of Auracite data: +* City-state +* World +* Nameday +* Guardian +* Race +* Gender +* Tribe + +Additionally, a character appearance save data file is included (if connected to the Dalamud plugin) that can be read by the retail client. # Physis -This is the library that powers my FFXIV-related projects. It has received some significant changes, most of them related to fixing problems introduced by changes in Dawntrail. +This is the library that powers my FFXIV-related projects. It has received some significant changes, most of them related to fixing problems introduced by changes in Dawntrail. Unfortunately due to the nature of it being a library, the update is text-only and extremely dry (hence why it's at the end!) -## Dawntrail Updates +## Dawntrail Support -I'm happy to report that everything *mostly* works now, with the biggest exception being model I/O. You can read Dawntrail models, but writing them results in an invalid model that crashes the retail client. I will be redoing how model I/O works at some point, which will fix inherent design problems with how it's structured today, but I'm unsure about how soon that will happen. +I'm happy to report that everything *mostly* works now, with the biggest exception being model I/O still. You can read Dawntrail models, but writing them results in an invalid model that crashes the retail client. I will be redoing how model I/O works at some point, which will fix inherent design problems with how it's structured today, but I'm unsure about how soon that will happen. -## Patching Patches +## Save Data -ZiPatch support has been improving, I not only fixed more patching bugs but also working towards write support. Physis-created patches can add new files, but that's about the extent of what I wrote so far. +As seen in the Auracite section above, I needed write support for character creator save data and that's now added. I also added unit tests for all of the versions, and the API is functionally complete! -I written a more comprehensive patch list parser for Kawari. I then moved it to Physis, so it can be shared with Astra to replace it's C++ implementation. A cool thing about this implementation is being able to write completely new patchlists, something that will be essential for a future feature in Kawari. +```rust +let char_dat = physis::chardat::CharacterData { + version: 7, + customize: physis::chardat::CustomizeData { + race: Race::Hyur, + tribe: Tribe::Midlander, + gender: Gender::Female, + ... everything else ... + }, + timestamp, + comment: "Generated by your tool!".to_string(), +}; +``` -## WebAssembly - -Physis can now be compiled to WebAssembly, which was used in the web version of Auracite. I also hacked away a lot of dependencies in Physis that either wasn't needed or I could safely take out, and the result is a _much_ lighter library to build! - -## Gearsets - -A contributor has kindly written support for the GEARSETS.dat save data, and I hope we can expand into more save data in the future! Currently we can read Character Creation data, Gearsets data, Log data and Plain-text config files. +Loius Brunner has [kindly written support for the `GEARSET.DAT` save data](https://github.com/redstrate/Physis/pull/7), and I hope we can expand into reading even more in the future! We can read the following save data now: +* Character creator presets +* Gearsets +* Chat logs +* Plain-text configs ## Retail Testing Platform -I have a lot of tests in Physis (which I'm really proud of) but one thing that's really lacking is testing against many retail game versions. For example, I end up breaking support for Benchmarks _way_ more often than you'd think. To solve this, I have begun writing a testing platform for running Physis against a bunch of different game versions. I have yet to write a web frontend for it, and it's still a script I have to manually run. +I have a lot of tests in Physis (which I'm really proud of!) but one thing that's really lacking is being able to test against many game versions. I end up breaking support for Benchmark versions accidentally, for example. To solve this, I have begun writing a testing platform for running Physis against a bunch of different game versions. I need to write a web frontend for it, and it's still a simple script I have to manually run. ## Miscel -In an effort to reduce the scope of the library API, I have moved a lot of functionality really only used by Astra to Miscel. This includes frontier URL lookup, installer helpers, and finding existing game directories. - -(libphysis - the C/C++ bindings - now depend on both libraries for simplicity.) +In an effort to reduce the scope of Physis, I have moved a lot of functionality really only used by Astra to a new library called [Miscel](https://github.com/redstrate/Miscel). This includes frontier URL lookup, installer helpers, and locating existing game installations. [libphysis](https://github.com/redstrate/libphysis) exposes both libraries to make things simple for now. ## Excel Improvements -One of the first things I wrote was the Excel parser, and boooy does it have a ton of mistakes. A good example is how I handled row IDs - or to be more specific how I _didn't_ handle them. I have begun shifting around a ton of the API, not only to make it more functionally correct but also to prepare for writing support. (Writing custom Excel sheets is for a future unannounced project!) +One of the first things I wrote was the Excel parser, and naturally it has a _ton_ of mistakes. A good example is how I handled row IDs - or to be more specific how I _didn't_ handle them and so looking up rows never really worked properly. To fix things, I have begun breaking a lot of the Excel API - not only to make it more functionally correct but also to prepare for writing support[^2]. ## Terrain and Layer Groups -Another focus of mine has been terrain and layer groups, the two formats that make up the Zones in the game - from background objects, to Event NPCs and even deciding where you teleport. I have tidied up the terrain parsing some, and begun implementing write support for both formats. Layer groups are going to be really tricky though, don't expect that for a while. +Another focus of mine has been terrain (TERA) and layer group (LGB) files, the two formats that make up most of the zone data in-game: terrain, background objects, NPCs and even deciding where you teleport to are read from these two formats. I have tidied up the terrain parsing some, and begun implementing write support for both formats. Writing LGBs are going to be really tricky, don't expect that for a long while. -## Character Save Data +I have begun exposing LGB objects in the public Physis API, which is used in Kawari to get pop ranges: -A feature needed for Auracite was writing character creation saved data. If you don't know, Auracite is a tool (and website!) where you can "save" your FFXIV character into a ZIP file. It contains a bunch of cool things like your Adventurer Plate (as showcased in my FFXIV shrine), information pulled from the Lodestone and more! +```rust +for group in &self.planmap.as_ref().unwrap().chunks[0].layers { + for object in &group.objects { + if let LayerEntryData::PopRange(pop_range) = &object.data { + if object.instance_id == instance_id { + return Some((object, pop_range)); + } + } + } +} +``` -## SQDB +## Everything Else -A little known format called SQDB (Square Enix Database?) is used to detail what is inside of data files. This isn't shipped in retail anymore, but is such a simple format it just makes sense to add support for it. +* The little-known format called SQDB (Square Enix Database?) is used to detail what is inside of data files. This isn't shipped in the retail game anymore, but is such a simple format it was easy to add support for it. +* Physis can now be compiled to WebAssembly, which was used in the web version of Auracite. I also hacked away a lot of dependencies in Physis that either wasn't needed or I could safely take out, and the result is a _much_ lighter library to build! +* ZiPatch support has been improving, I not only fixed more patching bugs but also working towards write support. Physis-created patches can add new files, but that's about the extent of what I wrote so far. # Documentation -I begun the process of moving from docs.xiv.zone to wiki.xiv.zone. You may alread have seen I linked to it in newer projects. The wiki format has proven to be a lower barrier for me to edit it, so I hope more interesting stuff pops up on there. Note that the wiki is _not_ publicly editable, and I have no immediate plans to allow sign ups. +I begun the process of moving from [docs.xiv.zone](https://docs.xiv.zone/) to [wiki.xiv.zone](https://wiki.xiv.zone). (You may have already seen links to it in some of my newer projects.) The wiki has proven to be a lower barrier for me to edit it, so I hope more interesting stuff pops up on there. Note that the wiki is _not_ publicly editable, and I have no immediate plans to allow sign ups. + +I will be adding redirects once the transition is complete. [^1]: It technically supports Windows, but there's very little reason to use it instead of XIVQuickLauncher. +[^2]: For a future, unannounced project. Oooo! diff --git a/content/blog/ffxiv-may2025/initial-setup.webp b/content/blog/ffxiv-may2025/initial-setup.webp new file mode 100644 index 0000000..aedda47 Binary files /dev/null and b/content/blog/ffxiv-may2025/initial-setup.webp differ diff --git a/content/blog/ffxiv-may2025/lua-decompiler.webp b/content/blog/ffxiv-may2025/lua-decompiler.webp new file mode 100644 index 0000000..9d34d5d Binary files /dev/null and b/content/blog/ffxiv-may2025/lua-decompiler.webp differ diff --git a/content/blog/ffxiv-may2025/novus.webp b/content/blog/ffxiv-may2025/novus.webp new file mode 100644 index 0000000..c99b833 Binary files /dev/null and b/content/blog/ffxiv-may2025/novus.webp differ diff --git a/content/blog/ffxiv-may2025/penumbra-actions.webp b/content/blog/ffxiv-may2025/penumbra-actions.webp new file mode 100644 index 0000000..a9ac057 Binary files /dev/null and b/content/blog/ffxiv-may2025/penumbra-actions.webp differ diff --git a/content/blog/ffxiv-may2025/server-settings.webp b/content/blog/ffxiv-may2025/server-settings.webp new file mode 100644 index 0000000..7e325e0 Binary files /dev/null and b/content/blog/ffxiv-may2025/server-settings.webp differ diff --git a/content/blog/ffxiv-may2025/simple-dalamud-settings.webp b/content/blog/ffxiv-may2025/simple-dalamud-settings.webp new file mode 100644 index 0000000..1f4832e Binary files /dev/null and b/content/blog/ffxiv-may2025/simple-dalamud-settings.webp differ diff --git a/content/blog/ffxiv-may2025/tools.webp b/content/blog/ffxiv-may2025/tools.webp new file mode 100644 index 0000000..5b5475c Binary files /dev/null and b/content/blog/ffxiv-may2025/tools.webp differ diff --git a/content/blog/ffxiv-may2025/update-channel.webp b/content/blog/ffxiv-may2025/update-channel.webp new file mode 100644 index 0000000..d395b05 Binary files /dev/null and b/content/blog/ffxiv-may2025/update-channel.webp differ diff --git a/content/blog/ffxiv-may2025/update-confirm.webp b/content/blog/ffxiv-may2025/update-confirm.webp new file mode 100644 index 0000000..ed1d084 Binary files /dev/null and b/content/blog/ffxiv-may2025/update-confirm.webp differ diff --git a/content/blog/ffxiv-may2025/update-space.webp b/content/blog/ffxiv-may2025/update-space.webp new file mode 100644 index 0000000..d355665 Binary files /dev/null and b/content/blog/ffxiv-may2025/update-space.webp differ