Update lots of documentation

This commit is contained in:
Joshua Goins 2023-12-22 15:52:04 -05:00
parent 467d11a385
commit a0682b1367
11 changed files with 381 additions and 83 deletions

View file

@ -13,4 +13,44 @@ Alternative programs for the [official launcher]({{< ref "/executable/ffxivlaunc
| Name | Description | Website |
| --- | --- | --- |
| XIVQuickLauncher | A faster launcher for our favorite critically acclaimed MMO, with various available addons and enhancements to the game! | https://goatcorp.github.io/ |
| XIVLauncher.Core | Cross-platform version of XIVLauncher, optimized for Steam Deck | https://github.com/goatcorp/XIVLauncher.Core |
| XIV on Mac | Wine Wrapper, Setup tool and launcher for FFXIV on mac | http://www.xivmac.com/ |
| microlaunch | A native Linux launcher for FINAL FANTASY XIV Online written in Rust | https://github.com/eorzeatools/microlaunch |
| Astra | A FFXIV launcher that supports profiles, multiple accounts and Dalamud plugins. | https://xiv.zone/astra/ |
# Clients
Alternative programs for the [official client]({{< ref "/executable/ffxiv" >}}).
| Name | Description | Website |
| --- | --- | --- |
| SaBOTender | FFXIV Clientless | https://github.com/shalzuth/SaBOTender |
# Servers
Alternative programs for the official servers such as [Frontier]({{< ref "frontier" >}}) or [Lobby]({{< ref "lobby" >}}).
| Name | Description | Website |
| --- | --- | --- |
| Sapphire | A Final Fantasy XIV 4.0+ Server Emulator written in C++ | https://github.com/SapphireServer/Sapphire/ |
| Kawari | Substitute for some official FFXIV servers | https://git.sr.ht/~redstrate/kawari |
# Libraries
Libraries to interact with game data, read formats and more.
| Name | Description | Website |
| --- | --- | --- |
| xivModdingFramework | The library that powers TexTools. Can read/write models, textures, materials etc. | https://github.com/TexTools/xivModdingFramework |
| Lumina | A simple, performant and extensible framework for interacting with FFXIV game data | https://github.com/NotAdam/Lumina |
| Xande | A C# library for interacting with FINAL FANTASY XIV models | https://github.com/xivdev/Xande |
| Physis | Data framework for FFXIV | https://git.sr.ht/~redstrate/physis |
| libxiv | A (deprecated) modding framework for FFXIV written in C++ | https://git.sr.ht/~redstrate/libxiv |
# Plugin Frameworks
Ways to extend the game at runtime with new capabilities.
| Name | Description | Website |
| --- | --- | --- |
| Dalamud | FFXIV plugin framework and API | https://github.com/goatcorp/Dalamud |

View file

@ -8,25 +8,29 @@ This is the game client executable.
# Arguments
* DEV.DataPathType
{{< info "Unlike other executables, passing arguments encrypted with SqexArg is optional for the client. It's recommended that you do so if you're planning to use actual SIDs though." >}}
There a few known arguments that work on the normal game client:
* `DEV.DataPathType`
* **Guessing** that this controls the asset data type used by the client, this is will always be `1`.
* DEV.UseSqPack
* `DEV.UseSqPack`
* **Guessing** that this tells the client to try to load data from SqPack files instead of from regular, uncompressed files. Seems to always be `1`.
* DEV.MaxEntitledExpansionID
* `DEV.MaxEntitledExpansionID`
* This is the max entitled expansion ID that the currently logging in user has access to, you will want to set this from your login response.
* DEV.TestSID
* `DEV.TestSID`
* This is the SID you get from your login response.
* SYS.Region
* `SYS.Region`
* Your login region.
* language
* `language`
* Your login language.
* ver
* `ver`
* The (base) game version string.
* DEV.GMServerHost
* `DEV.GMServerHost`
* This is the address of the frontier server to connect to. If empty, this will default to the official Square Enix frontier server.
* DEV.LobbyHost0X
* `DEV.LobbyHost0X`
* This is the address of the Nth lobby available to the client, these are numbered 0-9. If empty, this will default to the official Square Enix lobbies.
* DEV.LobbyPort0X
* `DEV.LobbyPort0X`
* This is the port number of the Nth lobby available to the client, these are numbered 0-9. If empty, this will default to the official Square Enix lobbies.
{{< info "For benchmark versions of the game, there is a whole host of graphical options available as game arguments - but this seems to be missing in the retail version." >}}

View file

@ -28,7 +28,7 @@ For details on how logging into the Square Enix servers work, see the [relevant
{{< note "Like the other executables, it requires you to pass these using the [SqexArg](concept/sqexarg) format." >}}
- ExecuteArg (**Required**)
- `ExecuteArg` (**Required**)
- This is a strange argument. This appears to be a random gibberish of numbers:
`/T =1000000 /ExecuteArg =14431503 /UserPath =C:/users/yourname/Documents/My Games/FINAL FANTASY XIV - A Realm Reborn`
@ -42,8 +42,8 @@ For details on how logging into the Square Enix servers work, see the [relevant
- 3rd arg - current hour
- 4th arg - current minute
- UserPath
- Your usual path to your FFXIV data folder in Documents.
- `UserPath`
- Your usual path to your FFXIV data folder in `My Documents`.
# Alternative Implementations

View file

@ -2,15 +2,13 @@
title: "Configuration File (.cfg)"
---
{{< note "This documentation is incomplete." >}}
A plaintext configuration file.
# Structure
The file is plaintext, which is easy to parse. The file is split into categories, which are surrounded by `<` and `>`. Within those categories are key-value pairs which are separated by tab characters (`\t`).
```
```cfg
<FINAL FANTASY XIV Boot Config File>
<Version>

View file

@ -4,49 +4,108 @@ title: "Character Creation Data (.dat)"
{{< note "This documentation is incomplete." >}}
Saved data from the character creation screen.
This contains the data you can save and load from the character creator, used not only in the retail client but the benchmark versions too.
# Structure
# Reading
There seems to be a few different versions of this format. Below is version 4:
The structure is very simple, it's only a series of fields.
## Version 4
| Offset | Type | Purpose |
| ------ | ----- | ------ |
| 0x00 | 32-bit integer | Magic, should be 0x2013FF14. |
| 0x04 | 32-bit integer | The version of the file, in this case "4". |
| 0x08 | 32-bit integer | Checksum of the data. |
| 0x0C | 32-bit integer | Unknown. |
| 0x0D | 8-bit unsigned integer | The character's race. |
| 0x0E | 8-bit unsigned integer | The character's gender. |
| 0x0F | 8-bit unsigned integer | The character's age. |
| 0x10 | 8-bit unsigned integer | The character's age. 1 is "Normal", 3 is "Old" and 4 is "Young". |
| 0x11 | 8-bit unsigned integer | The character's height from 0-255. |
| 0x12 | 8-bit unsigned integer | The character's subrace. |
| 0x13 | 8-bit unsigned integer | The character's head. |
| 0x14 | 8-bit unsigned integer | The character's hair. |
| 0x15 | 8-bit unsigned integer | If the character has hair highlights. |
| 0x16 | 8-bit unsigned integer | The character's skin tone. |
| 0x17 | 8-bit unsigned integer | The character's right eye color. |
| 0x18 | 8-bit unsigned integer | The character's hair color. |
| 0x19 | 8-bit unsigned integer | The character's hair highlights color. |
| 0x1A | 8-bit unsigned integer | The character's facial features. |
| 0x1B | 8-bit unsigned integer | If the character has limbal eyes. |
| 0x1C | 8-bit unsigned integer | The character's eyebrows. |
| 0x1D | 8-bit unsigned integer | The character's left eye color. |
| 0x1E | 8-bit unsigned integer | The character's eyes. |
| 0x1F | 8-bit unsigned integer | The character's nose. |
| 0x20 | 8-bit unsigned integer | The character's jaw. |
| 0x21 | 8-bit unsigned integer | The character's mouth. |
| 0x22 | 8-bit unsigned integer | The character's lip/tone/fur/pattern. |
| 0x23 | 8-bit unsigned integer | The character's tail. |
| 0x24 | 8-bit unsigned integer | The character's face paint. |
| 0x25 | 8-bit unsigned integer | The character's bust size from 0-255. |
| 0x26 | 8-bit unsigned integer | The character's face paint color. |
| 0x27 | 8-bit unsigned integer | The character's voice. |
| 0x28 | 8-bit unsigned integer | Unknown. |
| 0x29 | 32-bit unsigned integer | Timestamp? |
```c++
struct CharacterData {
// The version of the character data, the only supported version right now is 4.
uint32_t version: u32,
// The checksum of the data fields.
uint32_t checksum: u32,
uint32_t _unknown1;
// The race of the character.
uint8_t race: Race,
// The gender of the character.
uint8_t gender: Gender,
// The age of the character. Normal = 1, Old = 3, Young = 4.
uint8_t age: u8,
// The height of the character.
uint8_t height: u8,
// The character's subrace.
uint8_t subrace: Subrace,
// The character's selected head.
uint8_t head: u8,
// The character's selected hair.
uint8_t hair: u8,
// If hair highlights are enabled for this character.
uint8_t enable_highlights,
// The character's skin tone.
uint8_t skin_tone: u8,
// The character's right eye color.
uint8_t right_eye_color: u8,
// The character's hair color.
uint8_t hair_tone: u8,
// The color of the hair highlights.
uint8_t highlights: u8,
// The selected facial features.
uint8_t facial_features: u8,
// If the character has limbal eyes.
uint8_t limbal_eyes: u8,
// The character's selected eyebrows.
uint8_t eyebrows: u8,
// The character's left eye color.
uint8_t left_eye_color: u8,
// The character's selected eyes.
uint8_t eyes: u8,
// The character's selected nose.
uint8_t nose: u8,
// The character's selected jaw.
uint8_t jaw: u8,
// The character's selected mouth.
uint8_t mouth: u8,
// The character's selected pattern.
uint8_t lips_tone_fur_pattern: u8,
// The character's selected tail.
uint8_t tail: u8,
// The character's choice of face paint.
uint8_t face_paint: u8,
// The size of the character's bust.
uint8_t bust: u8,
// The color of the face paint.
uint8_t face_paint_color: u8,
// The character's chosen voice.
uint8_t voice: u8,
uint8_t _garbage;
// The timestamp when the preset was created.
uint8_t timestamp[4];
};
```
# Alternative Implementations

View file

@ -8,7 +8,7 @@ I recommend reading the ["Excel List" section](https://xiv.dev/game-data/file-fo
Describes Excel sheets that exist in the game data. A good example is is `exd/rool.exl` which contains many references to the game data. It's a text file containing a list of comma-separated strings, where each line is separated by a newline:
```
```txt
EXLT,2
Achievement,209
Action,4

View file

@ -4,7 +4,7 @@ title: "File Info (.fiin)"
I don't know the actual purpose of this file, it seems to contain SHA1 of certain files. The filename is usually `fileinfo.fiin` (like in `/boot`) and start with a header like this:
```
```c++
struct FileInfoHeader {
char magic[9];
uint8_t dummy1[16];
@ -16,11 +16,11 @@ struct FileInfoHeader {
};
```
`magic` is always `FileInfo`.
The content of `magic` is always `FileInfo`.
To get the number of entries, calculate it like so:
```
```c++
int overflow = info.header.unknown2;
int extra = overflow * 256;
int first = info.header.unknown1 / 96;
@ -31,7 +31,7 @@ _(Note: I have no idea if this is actually necessary, and it could just be a uns
Then each entry is located right after the header, and is structured as so:
```
```c++
struct FileInfoEntry {
uint8_t dummy[8]; // length of file name in some format
char str[64]; // simple \0 encoded string

View file

@ -4,7 +4,69 @@ title: "Chat Log (.log)"
{{< note "This documentation is incomplete." >}}
A binary store of recent chat logs.
A binary store of recent chat logs. These are usually found in the character folder, e.g. "FFXIV_CHR<number>/log/00000000.log"
## Reading
At the beginning of the file is a header with this content:
```c++
struct ChatLogHeader {
uint32_t contentSize;
uint32_t fileSize;
uint32_t offsets[fileSize - contentSize];
};
```
The content of the log begins at `(8 + fileSize * 4)`. The number of elements in `offsets` is also the number of messages in this log.
To read the messages, start reading at the offset described above. After seeking, read this short header:
```c++
enum EventFilter : uint8_t {
SystemMessages = 3,
Unknown = 20,
ProgressionMessage = 64,
NPCBattle = 41,
Unknown2 = 57,
Unknown7 = 29,
Unknown3 = 59,
EnemyBattle = 170,
}
enum EventChannel : uint8_t {
System = 0,
ServerAnnouncement = 3,
Unknown1 = 50,
Unknown7 = 29,
Others = 32,
Unknown5 = 41,
NPCEnemy = 51,
NPCFriendly = 59,
Unknown4 = 64,
Unknown6 = 170,
}
struct ChatLogEntry {
uint32_t timestamp;
EventFilter filter;
EventChannel channel;
uint32_t _garbage;
};
```
Then the string is ASCII(?) from the end of header to the element in `offsets`. The loop might look something like this:
```c++
uint32_t contentOffset = (8 + fileSize * 4);
seek(contentOffset);
for (uint32_t offset : offsets) {
readChatLogEntry();
message = read(cursorPos + offset);
seek(contentOffset + offset); // ensure we always end up at the right place.
}
```
# Alternative Implementations

View file

@ -4,7 +4,71 @@ title: "Model (.mdl)"
{{< note "This documentation is incomplete." >}}
A visual model.
Represents a visual model in the game, from furniture to the world, and characters.
# Reading
First read the model header, which contains some information like the data offsets:
```c++
struct ModelFileHeader {
uint32_t version;
uint32_t stackSize;
uint32_t runtimeSize;
uint16_t vertexDeclarationCount;
uint16_t materialCount;
uint32_t vertexOffsets[3];
uint32_t indexOffsets[3];
uint32_t vertexBufferSize[3];
uint32_t indexBufferSize[3];
uint8_t lodCount;
uint8_t indexBufferStreamingEnabled;
uint8_t hasEdgeGeometry;
uint8_t _unknown1;
};
```
Immediately after this header begins the vertex declarations which describe how vertex data is sequenced and eventually read. These structures define the basis of the vertex declarations:
```c++
enum VertexType : uint8_t {
Invalid = 0,
Single3 = 2,
Single4 = 3,
UInt = 5,
ByteFloat4 = 8,
Half2 = 13,
Half4 = 14,
};
enum VertexUsage : uint8_t {
Position = 0,
BlendWeights = 1,
BlendIndices = 2,
Normal = 3,
UV = 4,
Tangent = 5,
BiTangent = 6,
Color = 7,
};
struct VertexElement {
uint8_t stream;
uint8_t offset;
VertexType vertexType;
VertexUsage vertexUsage;
uint8_t usageIndex;
uint8_t unknown[3];
};
```
For every vertex declaration (the count in `vertexDeclarationCount`) first read a single `VertexElement`. Then keep reading a `VertexElement` (and keeping these in a buffer) until `VertexElement::stream` is `0xFF` (or 255 in decimal.) This implies an "end of stream" and that's the end of that declaration. Before reading the next declaration you need to advance your cursor `17 + 8 - (elements + 1) * 8` where `elements` is the amount of valid `VertexElement`s you just read.
# Alternative Implementations

View file

@ -2,34 +2,74 @@
title: "Shader Package (.shpk)"
---
{{< note "This documentation is incomplete, resource and scalar parameters are not documented yet." >}}
{{< note "This documentation is incomplete, nodes and keys are not documented yet." >}}
A collection of vertex and pixel shaders. A good example is _"character.shpk"_ which contains - you guessed it - character shaders.
A collection of vertex and pixel shaders. For example, _"character.shpk"_ which contains shaders used for gear.
# Header Structure
# Reading
All values are in **little-endian**.
At the beginning of the binary file is a simple header which describes what DirectX version the shader package is for, and the sizes of various structures.
| Offset | Type | Purpose |
| ------ | ----- | ------ |
| 0x00 | 4 character ASCII string | Magic, should read "ShPk". |
| 0x04 | Unknown 4 byte. | Unknown. |
| 0x08 | 4 character ASCII string | The format of the shader, e.g. "DX11". |
| 0x0C | 32-bit integer | File length in bytes. |
| 0x10 | 32-bit integer | Offset in bytes to the shader bytecode. |
| 0x14 | 32-bit integer | Offset in bytes to the parameter list. |
| 0x18 | 32-bit integer | Number of vertex shaders. |
| 0x1C | 32-bit integer | Number of pixel shaders. |
| 0x20 | 32-bit integer | Number of scalar parameters. |
| 0x24 | 32-bit integer | Number of resource parameters. |
## Header
# Reading shader bytecode
```c++
struct ShaderPackageHeader {
// Should be "ShPk" (capitalization intentional)
uint8_t magic[4];
Getting the shader bytecode from a shader package is easy, it begins at the shader bytecode offset in the header. Then, move your cursor up 12 bytes (to skip the initial `DXBC` ASCII, and some garbage data).
// Reads "DX9\0" (\0 being a null byte) for DX9 shaders, and "DX11" for DX11 shaders
uint8_t format[4];
Then, collect a WORD (4 bytes) at a time and putting them into a buffer. Keep doing this until you hit the next `DXBC` ASCII and you found a complete shader bytecode. Rinse and repeat and you should have `N` shaders where `N = number of vertex shaders + number of pixel shaders`.
uint32_t fileLength;
uint32_t shaderDataOffset;
uint32_t stringsOffset;
uint32_t vertexShaderCount;
uint32_t pixelShaderCount;
uint32_t materialParametersSize;
uint32_t materialParameterCount;
uint32_t scalarParameterCount;
uint32_t resourceParameterCount;
uint32_t uavParameterCount;
uint32_t systemKeyCount;
uint32_t sceneKeyCount;
uint32_t materialKeyCount;
uint32_t nodeCount;
uint32_t nodeAliasCount;
};
```
The format of this shader bytecode is nothing special, it's DXBC (the bytecode format used by DirectX before DXIL.) If you want to reverse this into something usable, like SPIR-V, HLSL or GLSL then you need to find a decompiler.
Right after the header is the beginning of shader data. The process is identical between vertex and pixel shaders, and they're read in that order (the vertex shaders are all read first, and then the pixel shaders.)
```c++
struct Shader {
uint32_t dataOffset;
uint32_t dataSize;
uint16_t scalarParameterCount;
uint16_t resourceParameterCount;
uint16_t uavParameterCount;
uint16_t _unknown;
ResourceParameter scalarParameters[scalarParameterCount];
ResourceParameter resourceParameters[resourceParameterCount];
ResourceParameter uavParameters[uavParameterCount];
};
```
Resource parameters are defined like so:
```c++
struct ResourceParameter {
uint32_t id;
uint32_t localStringOffset;
uint32_t stringLength;
uint16_t slot;
uint16_t size;
};
```
The shaders are stored as DXBC bytecode (the bytecode used by DirectX before DXIL.) targetted to the shader level of their respective API as described in `format` above. The data blob starts at `ShaderPackageHeader::shaderDataOffset + Shader::dataOffset` to a length of `dataSize` bytes. To access the ResourceParameter name, they start at `ShaderPackageHeader::stringsOffset + ResourceParameter::localStringOffset` to a length of `stringLength` bytes.
# Alternative Implementations

View file

@ -2,9 +2,40 @@
title: "Skeleton (.sklb)"
---
{{< note "This documentation is incomplete." >}}
A Havok skeleton, used for all of the animated characters and objects in the game. FFXIV uses the Havok SDK to do most of the heavy lifting for animation, so there's not much here.
A Havok skeleton.
# Reading
The first few bytes are the magic, which should be a `int32_t` that equals `0x736B6C62`. Then, there is a `uint32_t` that describes which version the skeleton is.
This is used if `version` is `0x3132_3030`:
```c++
struct SklbV1 {
uint16_t unk_offset;
uint16_t havok_offset;
uint32_t body_id;
uint32_t mapper_body_id1;
uint32_t mapper_body_id2;
uint32_t mapper_body_id3;
};
```
This is used if the `version` is `0x3133_3030` or `0x3133_3031`:
```c++
struct SklbV2 {
uint32_t unk_offset;
uint32_t havok_offset;
uint32_t _unknown;
uint32_t body_id;
uint32_t mapper_body_id1;
uint32_t mapper_body_id2;
uint32_t mapper_body_id3;
};
```
The rest of the file is the Havok binary tagfile format which you will probably need the Havok SDK or another method to read. The data starts at `SklbV1::havok_offset` or `SklbV2::havok_offset` and is contiguous until the end of the file.
# Alternative Implementations