Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

@tus/s3-store: add maxMultipartParts option #712

Merged
merged 4 commits into from
Feb 6, 2025

Conversation

mitjap
Copy link
Collaborator

@mitjap mitjap commented Feb 5, 2025

This PR closes #711.

I added additional option maxMultipartParts which allows user to define non-standard S3 server limitations.

Also some tests for completeness.

Summary by CodeRabbit

  • New Features

    • Introduced a new configurable option for multipart uploads, allowing users to set a custom maximum parts limit (defaulting to 10,000) for improved flexibility with S3-compatible storage providers.
  • Tests

    • Expanded the test suite to verify the default and custom behaviors of the new multipart upload configuration.

@mitjap mitjap requested a review from Murderlon February 5, 2025 21:25
Copy link

changeset-bot bot commented Feb 5, 2025

🦋 Changeset detected

Latest commit: f3ee9d2

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
@tus/s3-store Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

This comment was marked as resolved.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 2da6a84 and b8c3605.

📒 Files selected for processing (3)
  • .changeset/tricky-emus-fail.md (1 hunks)
  • packages/s3-store/src/index.ts (3 hunks)
  • packages/s3-store/test/index.ts (1 hunks)
🧰 Additional context used
🪛 Biome (1.9.4)
packages/s3-store/test/index.ts

[error] 281-281: Don't focus the test.

The 'only' method is often used for debugging or during implementation. It should be removed before deploying to production.
Consider removing 'only' to ensure all tests are executed.
Unsafe fix: Remove focus from test.

(lint/suspicious/noFocusedTests)


[error] 281-286: This function expression can be turned into an arrow function.

Function expressions that don't use this can be turned into arrow functions.
Safe fix: Use an arrow function instead.

(lint/complexity/useArrowFunction)


[error] 288-288: Don't focus the test.

The 'only' method is often used for debugging or during implementation. It should be removed before deploying to production.
Consider removing 'only' to ensure all tests are executed.
Unsafe fix: Remove focus from test.

(lint/suspicious/noFocusedTests)


[error] 288-295: This function expression can be turned into an arrow function.

Function expressions that don't use this can be turned into arrow functions.
Safe fix: Use an arrow function instead.

(lint/complexity/useArrowFunction)

🪛 GitHub Actions: CI
packages/s3-store/src/index.ts

[error] 110-110: Formatter would have printed the following content: const { maxMultipartParts, partSize, minPartSize, s3ClientConfig } = options

🔇 Additional comments (3)
packages/s3-store/src/index.ts (2)

104-104: LGTM!

The property declaration is correct. Removing the as const assertion is appropriate since the value can be modified through the constructor.


522-528: LGTM!

The logic for calculating optimal part size based on maxMultipartParts is correct and well-documented. It ensures that:

  1. If the upload size is smaller than the preferred part size, it uses a single part.
  2. If the upload fits within maxMultipartParts using the preferred part size, it uses the preferred size.
  3. Otherwise, it calculates the minimum part size needed to fit within maxMultipartParts.
.changeset/tricky-emus-fail.md (1)

1-6: LGTM!

The changeset correctly indicates a minor version bump for adding a new feature. The description clearly explains the purpose of the maxMultipartParts option.

Copy link
Member

@Murderlon Murderlon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Sorry for the noisy coderabbit review, it's an experiment and this is the first PR it reviews.

@Murderlon Murderlon merged commit 7db2f17 into tus:main Feb 6, 2025
4 checks passed
@mitjap
Copy link
Collaborator Author

mitjap commented Feb 6, 2025

It is noisy, but parts of it make sense. Just when I figured I need to remove .only() in tests I saw that it warned me about this, so this is a good thing. It also suggested some formating changes which would not align with selected formatter.

Are test set to fail if it encounters focused tests? (using .only() keyword)?

@mitjap mitjap deleted the fix-scaleway-parts-limitation branch February 6, 2025 09:39
@Murderlon
Copy link
Member

Are test set to fail if it encounters focused tests? (using .only() keyword)?

The linting step (Biome) should fail on this yes

Murderlon added a commit that referenced this pull request Feb 6, 2025
* 'main' of https://github.com/tus/tus-node-server:
  @tus/s3-store: add `maxMultipartParts` option (#712)
  [ci] release (#701)
  @tus/s3-store: add `minPartSize` option (#703)
@github-actions github-actions bot mentioned this pull request Feb 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

@tus/s3-store: add support to specify maxMultipartParts
2 participants