Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

@tus/s3-store: add support to specify maxMultipartParts #711

Closed
2 tasks done
mitjap opened this issue Feb 5, 2025 · 0 comments · Fixed by #712
Closed
2 tasks done

@tus/s3-store: add support to specify maxMultipartParts #711

mitjap opened this issue Feb 5, 2025 · 0 comments · Fixed by #712

Comments

@mitjap
Copy link
Collaborator

mitjap commented Feb 5, 2025

Initial checklist

  • I understand this is a feature request and questions should be posted in the Community Forum
  • I searched issues and couldn’t find anything (or linked relevant results below)

Problem

Scaleway's S3-compatile Object Storage has different number of parts limit. While AWS S3's limit is 10.000, Scaleway's is 1.000. I've reported a bug to them but I don't expect anything to change at their side.

This is a problem because @tus/s3-store calculates required part size for given file size to make sure it does not exceed this limit.

Server rejects part upload with HTTP status 400 and message InvalidArgument: Part number must be an integer between 1 and 1000, inclusive.

AWS S3 documentation:

Part numbers can be any number from 1 to 10,000, inclusive.
Soruce: https://docs.aws.amazon.com/AmazonS3/latest/API/API_UploadPart.html

Scaleway Object Storage documentation:

The following technical specifications apply to multipart uploads: 1 to 1000 parts per object
Source: https://www.scaleway.com/en/docs/object-storage/api-cli/multipart-uploads/#uploading-a-part

Solution

I would like to specify to @tus/s3-store what limits are imposed by the server, in this case maxMultipartParts.

Alternatives

I can reduce the impact of this problem by increasing minPartSize to a larger value (e.g. 50MB).

@mitjap mitjap changed the title @tus/s3-store: add support to specify S3 server configuration @tus/s3-store: add support to specify maxMultipartParts Feb 5, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant