Skip to content
This repository has been archived by the owner on Feb 14, 2020. It is now read-only.

Upload handling #16

Open
sr229 opened this issue Dec 24, 2018 · 1 comment
Open

Upload handling #16

sr229 opened this issue Dec 24, 2018 · 1 comment
Labels
enhancement New feature or request REST Issues relating to the REST-level implementation
Milestone

Comments

@sr229
Copy link
Contributor

sr229 commented Dec 24, 2018

Supposedly this will be in par with our supporter system. Since we'll be allowing users to store their mods inside the same server Sayonika runs. But this presents serious scaling challenges to give such privelege.

I, however came up with a flow that emphasizes efficient use of storage while keeping with our scaling constraints. To do this we do the following:

On POST

  • We parse the ZIP from the multipart, and compress it in GZIP. GZIPs are notoriously known to keep things small, from a 226MB image to a 12MB archive.
  • Store it on S3 with its own file Snowflake as its name.

On GET

Now the download URL will be something like this https://sayonika.moe/api/v1/download/<mod_id>.

However, we only have two supported ways to distributed compressed GZIPs.

on Sayonika Frontend

  • Frontend is responsible to decompress the GZIP, and return it as a blob, then call the download as a blob stream.
    • This is quite useful for low-bandwidth users.

on Third Party

  • Third party is liable to decompress the GZIP by themselves, API shall return the GZIPped file.

Limits

To enforce limits, there would be only ONE FILE with ZIP extension per mod. All subsequent ZIPs will be treated as a new version of the file and will be saved, the old GZIPped archive will be overwritten.

@sr229 sr229 added enhancement New feature or request REST Issues relating to the REST-level implementation labels Dec 24, 2018
@sr229 sr229 added this to the Post v1 milestone Dec 24, 2018
@sr229
Copy link
Contributor Author

sr229 commented Feb 27, 2019

Please do #17 and #33 first before this.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement New feature or request REST Issues relating to the REST-level implementation
Projects
None yet
Development

No branches or pull requests

1 participant