util: add basic support for CEs to encounter_tools

[jacob-keller]

Add some basic support for detecting and splitting encounters by CE, based on the 0x21 network log line. The IDs for this are manually determined, but we now pull the encounter names from xivapi, and generate appropriate mappings.

This works ok in my tests, and so far doesn't appear to break any other encounter logic. However, it likely needs more testing and fine tuning.

I also now added logic to pull the CE info from xivapi, with a new generator script. I did pull a bit more data than we currently use (since the current code doesn't use anything but the english names). I don't yet pull the Chinese or Korean data like how zone_ids.ts does yet.

There's probably some further cleanup we can do around encounter support, but I wanted to try and get the very basic splitting functionality done.

• util: add support for sheets with subrow_id to xivapi
• util: log eslint errors when failing in writeFile
• util: generate CE info from xivapi
• encounter_tools: detect and report CEs as separate encounters

Signed-off-by: Jacob Keller [email protected]

[cactbotbot]

@jacob-keller Thanks for your contribution! 🌵🚀

[jacob-keller]

@valarnin do you think something like this could be merged? I tried to figure out a way to make the encounter listing work without breaking existing support.

[jacob-keller]

I guess my question is whether this approach is good, or if we need to do some other refactoring on the encounter logic as a whole to properly support CEs.

[jacob-keller]

I have some more thoughts on this, and I will try to clean it up before making it a non-draft

[valarnin]

I'll hold off on reviewing actual code changes in detail until you switch to non-draft.

---

As far as I've been able to determine, the mapping of DirectorUpdate param to CE ID is done in the game's lua, which means that unfortunately we can't pull that automatically. What we can do is pull DynamicEventSet -> DynamicEvent mappings from XIVAPI and expose those as a data file, and at least have automatic translations and such for the data.

DynamicEventSet's data in the base CSV is e.g. 3.5,37, where 3 is for OC, 5 is the 5th CE in OC, and 37 is the row ID in DynamicEvent corresponding to Calamity Bound.

Unfortunately, XIVAPI has a bug with this data sheet where the Unkown0 field is invalid. Once the XIVAPI bug is fixed, I envision something like:

// normal typedef for XIVAPI exported data

const data: DynamicEventSetType = {
  '3.5': {
    // ... other fields
    'Name': {
      'en': 'Calamity Bound',
      // other langs
    },
  },
} as const;

export default data;

export type CEMap<CEList> = {
  [directorCEID: string]: keyof CEList;
};

// `<foo>CEs` auto-generated based on XIVAPI sheet
// `<foo>CEs` keys auto mapped by stripping non-alphanumeric characters
// and lowercasing first character maybe?
// `<foo>CEMap` is a manual mapping in the util script, like in `zone_overrides.ts`

export const bozjaCEs = {
  // entries from `data` starting with `1.`
} as const;

export const bozjaCEMap: CEMap<typeof bozjaCEs> = {};

export const zadnorCEs = {
  // entries from `data` starting with `2.`
} as const;

export const zadnorCEMap: CEMap<typeof zadnorCEs> = {};

export const southHornCEs = {
  // entries from `data` starting with `3.`
  'calamityBound': data['3.5'],
} as const;

export const southHornCEMap: CEMap<typeof southHornCEs> = {
  '32F': 'calamityBound',
};

Other solutions are fine too, the ultimate goal is to source the name of the CE and its base "zone" from XIVAPI somehow.

[jacob-keller]

Unfortunately, XIVAPI has a bug with this data sheet where the Unkown0 field is invalid. Once the XIVAPI bug is fixed, I envision something like:

Any idea if XIVAPI stuff is open source and I could look at what it takes to fix this?

[valarnin]

Any idea if XIVAPI stuff is open source and I could look at what it takes to fix this?

It's fixed as of a minute ago or so. https://v2.xivapi.com/api/sheet/DynamicEventSet/3:1

For future reference, the XIVAPI discord is at https://discord.gg/MFFVHWC

[jacob-keller]

I'll take a stab at implementing something hopefully in the next couple of days

[valarnin]

I'll do a more proper review later or tomorrow, just wanted to address this first.

[valarnin]

We shouldn't be passing a subrow ID if the sheet doesn't have subrow IDs. This works for now but it's unintended behavior.

Probably maxSubRow should be number | undefined, and this should instead be:

      if (maxRow > 0) {
        url += `&after=${maxRow}`;
        if (maxSubRow !== undefined)
          url += `:${maxSubRow}`;
      }

or something along those lines.

[jacob-keller]

Will fix.

[jacob-keller]

I have some additional followup work to extend the make_timeline.sh with support for inserting appropriate headers for the CE start, as well as a new --timeline_offset commandline argument to allow adjusting the initial timelinePosition value to make it easier to create CE timelines in the way that we have them laid out in the exploration zone timeline files. I'm planning to submit those after this merges.

[Legends0]

I think it worked for what I provided in #736 ?

[jacob-keller]

Updated to include the director ID for forked tower reported by Legends0.

[jacob-keller]

Does this need any changes to be acceptable?

[Legends0]

Some additional notes... there are more action control lines within forked tower: blood:
33B => Demon Tablet
33F => demon tablet defeated
33C => trash before dead stars cleared (I think one side can still be clearing trash when this happens?)
340 => Dead Stars defeated
33D => Bridges cleared
341 => Marble Dragon cleared
33E => Magitaur room unlocked

There might another for after Magitaur, probably 342. EDIT: There was nothing. I also labelled this in my forked tower pr.