-
Notifications
You must be signed in to change notification settings - Fork 18
Description
Recording discussion from yesterday: We might want to store some files in this repo in compressed form (e.g., model weights of a potential Transformers.js workload). The reduced file size (up to 40% reduction for said model weights) would be appreciated not only in this repository, but also when importing it in other projects, say, a browser's/engine's source tree.
It makes sense to choose a compression format that is widely supported by clients, i.e., gzip (potentially using zopfli or brotli and then serve these files as-is by the webserver, instead of recompressing or using a compression format that needs to be handled in JS/Wasm explicitly.
We anyways need a web server with some light modifications for serving the right CORS/COEP headers for Wasm multi-threaded workloads. Steps:
[ ] simple Python server based on http.server
[ ] correct MIME type for Wasm
[ ] add CORS/COEP headers
[ ] serve .gz files as-is if client has Accept-Encoding: gzip set
[ ] add scripts to update+recompress relevant files (e.g., Transformers.js or TF.js model weights) on rebuild
[ ] put all of this in util/ dir or something
[ ] optional: add configs for other web servers (Node.js, Apache?)