Speed up preloads using unity-scene-repacker

[jakobhellermann]

Currently preloading loads the entire scene at startup, which is usually the slowest part of startup.

unity-scene-repacker lets us repack the game's scenes into AssetBundles which can be loaded instead.

This PR introduces a PreloadMode globalsetting, with the following modes:

• full-scene: the current behaviour
• repack-scene: repacks the scenes into an AssetBundle containing the exact referenced scenes, but filtered to only relevant objects. The preloading code stays pretty much the same
• repack-assets: repacks the scenes into an AssetBundle that contains the game objects separately, which can then be spawned using `bundle.LoadAsset("SceneName/path/to/object.prefab"). For compatibility with the current API, every object still gets spawned in the beginning and handed off to the mods.

Some rough benchmarking with a few mods enabled:

• 20s full-scene
• 9s repack-scene
• 5s repack-assets
  However, only full-scene and repack-scene modes support sceneHooks.

---

In this PR:

• new preloader strategies 7be2e86 a81730f
  • this include a new unityscenerepacker.dll (or libunityscenerepacker.{so,dylib} on linux/macOS)
• remove $(SolutionDir) from csproj f9f50d0 this enabled dotnet build Assembly-CSharp --runtime win-x64 which is required to copy the native dependencies.
• fix smoothing of progress bar f3f77be
  • previously, when progress was made too fast, it would never actually catch up
• add <CopyDir> csproj variable which you can set to your game to automatically copy files on build, simplifying the build -> run -> edit cycle (I can move this to another PR or delete this if y'all don't like this)

[jakobhellermann]

Draft because the version of https://github.com/jakobhellermann/unity-scene-repacker used is not yet released and currently contains some hardcoded data for Hollow Knight I want to remove first.

[jakobhellermann]

Test build: MAPI-windows.zip

[BadMagic100]

Practically speaking, I think having this as a mapi global setting is bad since the majority of users will see no benefits from the feature that way. I would propose the following approach instead:

• Possibility of different strategies per mod
• By default, all mods use repack-scene for a modest performance improvement
• Mods which don't implement PreloadSceneHooks can opt themselves in to repack-assets for even better performance. Perhaps also worth considering, since this is opt-in anyway, making the API break that was discussed in discord to just be handed the assetbundle for even better performance
• Optional: add a legacyPreloads global setting (default false) to allow players to revert to the current behavior instead of repack-scene by default. Would allow players to work around any incompatibilities

[PrashantMohta]

I think having an opt out ( setting it to enabled by default) will be good enough for players wanting to kill switch it.

for mods wanting to use scene hooks, they should be able to indicate somehow that they care to hook into some scenes and only those should be loaded in that case.

[jakobhellermann]

Practically speaking, I think having this as a mapi global setting is bad since the majority of users will see no benefits from the feature that way. I would propose the following approach instead:

Yes, it makes sense to have this enabled my default. I mostly did it this way first in case we wanted to do some testing before rolling it out, in case there are any issues that come up. E.g., I haven't tested this on macOS, and I wonder if the native dependency immediately works on apple silicon.

Mods which don't implement PreloadSceneHooks can opt themselves in to repack-assets for even better performance.

Since we know what mods request scene hooks, we could also do that automatically. Use repack-scene for those that need it, and repack-assets otherwise.

[BadMagic100]

This name implies to me that it's using FullScene, not RepackScene

[jakobhellermann]

I changed it to

IEnumerator DoPreloadAssets(...) {
  if(uses scenehooks) return DoPreloadRepackedScenes(...);
}
IEnumerator DoPreloadRepackedScenes(...) {
  // repack bundle
  DoPreloadScenes(..., "mapi_preload_assetbundle");
}
IEnumerator DoPreloadScenes(..., sceneprefix = "")

[jakobhellermann]

When packing the scenes into an assetbundle, we need to have information about the layout of the serialized data of the game's MonoBehaviours, so that we can patch up some layout changes.

Here I'm just bundling them into the dll (they're 200KiB big).

But the question is, does the modding API support earlier game versions as well?
In that case, this would have to be done differently:

1. include a dump for each game version
2. generate the dumps at runtime:
   • either inside unityscenerepacker.dll using TypeTreeGeneratorAPI, but that would be an extra 18MiB dll we have to distribute
   • or generate the type trees here in the modding API using https://github.com/nesrak1/AssetsTools.NET, but that would have more complexity in this project

[Yurihaia]

But the question is, does the modding API support earlier game versions as well?

we only support the current version really. there are builds for some earlier patches but they don't really need these types of new features (and the API itself is wildly different)