mirrors/EXDSchema
1
Fork 0
mirror of https://github.com/xivdev/EXDSchema.git synced 2025-06-06 16:17:46 +00:00
Code Activity

Initial commit

Browse source
This commit is contained in:
liam 2023-09-16 16:13:06 -04:00
commit c3e126cabb
9 changed files with 356 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

5
.gitignore vendored Normal file
View file

@ -0,0 +1,5 @@
bin/
obj/
/packages/
riderModule.iml
/_ReSharper.Caches/

22
EXDSchema.sln Normal file
View file

@ -0,0 +1,22 @@

Microsoft Visual Studio Solution File, Format Version 12.00
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "SchemaConverter", "SchemaConverter\SchemaConverter.csproj", "{9AC0C7BB-939B-496A-A882-44327DAF7C3F}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "SchemaValidator", "SchemaValidator\SchemaValidator.csproj", "{22B92DA2-D46B-4219-A0E7-CC55F9213BAD}"
EndProject
Global
GlobalSection(SolutionConfigurationPlatforms) = preSolution
Debug|Any CPU = Debug|Any CPU
Release|Any CPU = Release|Any CPU
EndGlobalSection
GlobalSection(ProjectConfigurationPlatforms) = postSolution
{9AC0C7BB-939B-496A-A882-44327DAF7C3F}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{9AC0C7BB-939B-496A-A882-44327DAF7C3F}.Debug|Any CPU.Build.0 = Debug|Any CPU
{9AC0C7BB-939B-496A-A882-44327DAF7C3F}.Release|Any CPU.ActiveCfg = Release|Any CPU
{9AC0C7BB-939B-496A-A882-44327DAF7C3F}.Release|Any CPU.Build.0 = Release|Any CPU
{22B92DA2-D46B-4219-A0E7-CC55F9213BAD}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{22B92DA2-D46B-4219-A0E7-CC55F9213BAD}.Debug|Any CPU.Build.0 = Debug|Any CPU
{22B92DA2-D46B-4219-A0E7-CC55F9213BAD}.Release|Any CPU.ActiveCfg = Release|Any CPU
{22B92DA2-D46B-4219-A0E7-CC55F9213BAD}.Release|Any CPU.Build.0 = Release|Any CPU
EndGlobalSection
EndGlobal

41
README.md Normal file
View file

@ -0,0 +1,41 @@
# EXDSchema
## Introduction
This is the schema repository for SqPack [Excel data](https://xiv.dev/game-data/file-formats/excel).
## Sheets
Inside SqPack, category 0A consists of Excel sheets serialized into a proprietary binary format read by the game.
The development cycle generates header files for each sheet, which are then compiled into the game, thus, all structure information
is lost on the client side when the game is compiled. This repository is an attempt to consolidate efforts into a
language agnostic schema, easily parsed into any language that wishes to consume it, that accurately describes the structure
of the EXH files as they are provided to the client.
## Schema
Schemas are written in YML to define the fields of the structure in an EXH file and the links between different fields.
The schema provides a number of features, all of which is enforced by the provided JSON schema for the schema. When applied
against an EXD schema file, it will provide IDE completion and error-checking to improve the manual editing experience.
## Features
Since EXH files define the data types for each column, the schema does not care or define any data types in the standard sense.
Instead, it focuses on declaratively defining the structure of the compiled EXH data structures and the relationships between fields.
The schema includes the following:
- Full declaration of fields is required, nothing can be omitted
- Support for a few common types across sheets, such as `modelId`, `color`, and `icon`
- While these do not affect the overall parsing, they are
useful for research as they provide an important hint for the purpose of the data
- Field names
- Arrays
- Links between fields
- Multi-targeting another sheet
- Complex linking between fields based on a `switch` conditional
- Comment support on any schema object
- Maps out-of-the-box to a very simple object mapping
- JSON schema for the schema itself, providing IDE completion and error-checking
- Example code
- A C# project that perform the conversion from the
common JSON schemas ("SC Schema") to the EXDSchema format
- A C# project that will validate EXDSchemas against a given game installation
- A tool for editing schemas and viewing the results of parsing on-the-fly
## Usage
See Usage.md.

3
SchemaConverter/Program.cs Normal file
View file

@ -0,0 +1,3 @@
// See https://aka.ms/new-console-template for more information
Console.WriteLine("Hello, World!");

10
SchemaConverter/SchemaConverter.csproj Normal file
View file

@ -0,0 +1,10 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>net7.0</TargetFramework>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
</PropertyGroup>
</Project>

3
SchemaValidator/Program.cs Normal file
View file

@ -0,0 +1,3 @@
// See https://aka.ms/new-console-template for more information
Console.WriteLine("Hello, World!");

10
SchemaValidator/SchemaValidator.csproj Normal file
View file

@ -0,0 +1,10 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>net7.0</TargetFramework>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
</PropertyGroup>
</Project>

0
Schemas/.gitkeep Normal file
View file

262
Usage.md Normal file
View file

@ -0,0 +1,262 @@
# Usage
## Initial Creation
To define a schema, you should create a file with the same name as the sheet it is defining.
The name field must contain the name of the sheet as well. If we were to write a schema for `AozActionTransient`, we would do the following:
```yml
name: AozActionTransient
fields:
- name: Field1
- name: Field2
- name: Field3
# etc ...
```
#### DisplayField
The `displayField` key is provided for consumers that wish to resolve a sheet reference within a single cell. It provides a hint
of what a user will *most likely* want to see when the current sheet is targeted by a link. For example, when linking to `BNpcName`,
the most likely column to reference would be `Name`. For `Item`, the most likely column might be `Name` or `Singular`.
## Defining Fields
All sheets must have a number of field entries that corresponds to the number of columns in that sheet.
If not, parsing should fail.
We can define fields like this:
```yml
type: sheet
fields:
- name: Stats
- name: Description
- name: Icon
- name: RequiredForQuest
- name: PreviousQuest
- name: Location
- name: Number
- name: LocationKey
- name: CauseStun
- name: CauseBlind
- name: CauseInterrupt
- name: CauseParalysis
- name: TargetsSelfOrAlly
- name: CauseSlow
- name: TargetsEnemy
- name: CausePetrify
- name: CauseHeavy
- name: CauseSleepy
- name: CauseBind
- name: CauseDeath
```
This schema is valid because it is accurate - not in name, but in structure. It defines a field for each column in the EXH file as of 6.48.
### Types
Valid types for fields in a schema are `scalar`, `array`, `icon`, `modelId`, and `color`.
#### scalar
The default type. If the `type` is omitted from a field, it will be assumed to be a scalar. Effectively does nothing except tell consumers that
"this field is not an array".
#### icon : uint32
In the above AozActionTransient example,
```yml
- name: Icon
```
can become
```yml
- name: Icon
type: icon
```
While this may seem redundant, there are many fields in column that refer to an icon within the `06`, or the `ui/` category,
but the field itself is just a uint32. This is a hint for any consumer that attempts to display this field that the data in this column
can be used to format an icon path, like generating `ui/icon/132000/132122_hr1.tex` when the field contains `132122`, without the consumer having
to manually determine which columns contain icons.
#### modelId : uint32, uint64
Model IDs in the game are packed into either a uint32 or a uint64.
uint32 packing is like so:
```
uint16 modelId
uint8 variantId
uint8 stain
```
uint64 packing is like so:
```
uint16 skeletonId
uint16 modelId
uint16 variantId
uint16 stainId
```
To anyone *viewing* the data for research, the packed values are useless, so consumers that provide a view into sheet data can opt
to unpack these values and display them as their unpacked counterparts. Many tools utilize these values individually rather than packed,
so it's important to have the ability to define a field this way.
#### color : uint32
Some fields contain an RGB value for color in the ARGB format with no alpha. This is simply a hint if a consumer opts to display these
columns' fields as actual colors rather than the raw value.
#### array
Array fields provide the ability to group and repeat nested structures. For example, the notorious SpecialShop sheet:
```yml
name: SpecialShop
fields:
- name: Name
- name: Item
type: array
count: 60
fields:
- name: ReceiveCount
type: array
count: 2
- name: CurrencyCost
type: array
count: 3
- name: Item
type: array
count: 2
- name: Category
type: array
count: 2
- name: ItemCost
type: array
count: 3
- name: Quest
type: array
count: 2
- name: Unknown
- name: AchievementUnlock
- name: CollectabilityCost
type: array
count: 3
- name: PatchNumber
- name: HqCost
type: array
count: 3
- type: array
count: 3
- name: ReceiveHq
type: array
count: 3
- name: Quest
- type: scalar
- type: scalar
- name: CompleteText
- name: NotCompleteText
- type: scalar
- name: UseCurrencyType
- type: scalar
- type: scalar
```
As you can see, we have nested arrays in this structure. This means that the in-memory structure follows like so:
```C
struct SpecialShop
{
struct
{
example_type ReceiveCount[2];
example_type CurrencyCost[3];
example_type Item[2];
example_type Category[2];
example_type ItemCost[3];
example_type Quest[2];
example_type Unknown;
example_type AchievementUnlock;
example_type CollectabilityCost[3];
example_type PatchNumber;
example_type HqCost[3];
example_type Unknown2[2];
example_type ReceiveHq[3];
} Items[60];
example_type Quest;
example_type Unknown;
example_type Unknown2;
example_type CompleteText;
example_type NotCompleteText;
example_type Unknown3;
example_type UseCurrencyType;
example_type Unknown4;
example_type Unknown5;
};
```
As you can see, the overall schema is similar to defining structures in YML but omitting the actual data type.
This nested capability allows you to define complex structures. However, to cut down on overall parsing complexity,
based on existing knowledge of the EXH data, **you may only nest twice.**
### Linking
The sheets that power the game are relational in nature, so the schema supports a few different kinds of linking.
#### Single Link
To define a link, simply add a link object:
```yml
- name: Quest
link:
target: [Quest]
```
Note that the link target is an array of strings. They must be sheet names, and there must be at least one sheet. To link to one sheet, leave a single sheet in the array.
#### Multi Link
A sheet's single column can link to multiple columns:
```yml
- name: Requirement
link:
target: [Quest, GrandCompany]
```
In this case, disparate sheet key ranges will provide the ability to determine which sheet a link should resolve to.
For example, if a row's `Requirement` is `2`, it will resolve to `GrandCompany`, because row `2` exists in `GrandCompany` and not in `Quest.`
The same thing happens in the other direction: if `Requirement` is `69208`, it will link to `Quest` and not `GrandCompany` for the same reason.
#### Conditional Link
A sheet's single column can link to multiple columns depending on another field in the sheet:
```yml
- name: Location
comment: PlaceName when LocationKey is 1, ContentFinderCondition when LocationKey is 4
link:
target: [PlaceName, ContentFinderCondition]
condition:
switch: LocationKey
cases:
1: [0]
4: [1]
```
The targets array must contain all possible sheets that this field can link to.
When defining the link, add a `condition` object with a `switch` key that defines the field to switch on the value of.
The `cases` dictionary contains arrays of *the indexes of* sheets to resolve to when the case matches.
Yes, the `case` dictionary may contain an *array*. This means that each case can be a [multi link](#multi-link) as well. Take `Item` for example:
```yml
- name: AdditionalData
link:
target:
- Stain
- TreasureHuntRank
- GardeningSeed
- AetherialWheel
- CompanyAction
- TripleTriadCard
- AirshipExplorationPart
- Orchestrion
- SubmarinePart
- HousingExterior
- HousingInterior
- HousingYardObject
- HousingFurniture
- HousingPreset
- HousingUnitedExterior
condition:
switch: FilterGroup
cases:
14: [9, 10, 11, 12, 13, 14]
15: [0]
18: [1]
20: [2]
25: [3]
26: [4]
27: [5]
28: [6]
32: [7]
36: [8]
```
The `AdditionalData` column does a lot of heavy lifting. We can assume during game execution that the use of the field is heavily based on context,
but for research and data exploration, having the ability to define the exact sheet is very useful. Here, we can see that when `FilterGroup` is `14`,
we can link to any of `HousingExterior`, `HousingInterior`, `HousingYardObject`, `HousingFurniture`, `HousingPreset`, or finally `HousingUnitedExterior`.
This works because the value for `AdditionalData` are distinct ranges, even when `FilterGroup` is 14, thus allowing the definition here to behave like a multi link.