Register `s3.client.<client>.disable_chunked_encoding` in repository-s3 node settings #20161

Gautam-aman · 2025-12-03T18:27:42Z

Description

This PR registers the s3.client.<client>.disable_chunked_encoding setting in the repository-s3 plugin.

Currently, configuring the setting in opensearch.yml such as:

s3.client.default.disable_chunked_encoding: true

results in a startup failure : unknown setting [s3.client.default.disable_chunked_encoding]

even though DISABLE_CHUNKED_ENCODING is defined in S3ClientSettings and supported at repository level.

Root Cause : 
DISABLE_CHUNKED_ENCODING was not included in the list returned by S3RepositoryPlugin#getSettings(), so the setting was never registered as a valid node setting.

Fix:
Added S3ClientSettings.DISABLE_CHUNKED_ENCODING to S3RepositoryPlugin#getSettings().
Added a test assertion in S3RepositoryPluginTests#testGetExecutorBuilders to ensure the setting is registered.

Testing
./gradlew :plugins:repository-s3:check passes successfully.

Impact : 
Users can now safely configure s3.client.default.disable_chunked_encoding in opensearch.yml for S3-compatible storage that does not support AWS chunked encoding (e.g., Oracle Object Storage, MinIO, Ceph, Cloudflare R2, etc.).


<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit

* **New Features**
  * Exposes additional S3 client settings: request timeout, connection timeout, connection TTL, max connections, max sync connections, connection acquisition timeout, max pending connection acquires, and a chunked-encoding toggle.

* **Tests**
  * Test suite updated to assert the expanded set of S3 configuration settings.

* **Other**
  * Internal equality/hash behavior adjusted; public APIs unchanged.

<sub>✏️ Tip: You can customize this high-level summary in your review settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

coderabbitai · 2025-12-03T18:28:06Z

Walkthrough

Added multiple S3 client connection and timeout settings to the plugin settings list, updated unit tests to assert those settings, and adjusted S3ClientSettings equality/hash/refine logic to include maxSyncConnections and use primitive comparison for disableChunkedEncoding.

Changes

Cohort / File(s)	Summary
Plugin settings list `plugins/repository-s3/src/main/java/org/opensearch/repositories/s3/S3RepositoryPlugin.java`	Extended `getSettings()` to expose additional S3 client configuration settings: `REQUEST_TIMEOUT_SETTING`, `CONNECTION_TIMEOUT_SETTING`, `CONNECTION_TTL_SETTING`, `MAX_CONNECTIONS_SETTING`, `MAX_SYNC_CONNECTIONS_SETTING`, `CONNECTION_ACQUISITION_TIMEOUT`, `MAX_PENDING_CONNECTION_ACQUIRES`, `DISABLE_CHUNKED_ENCODING` (inserted after `SIGNER_OVERRIDE` and before `REGION`). Comment updated to "s3 client configuration settings".
Tests `plugins/repository-s3/src/test/java/org/opensearch/repositories/s3/S3RepositoryPluginTests.java`	`testGetExecutorBuilders` expanded to assert presence of the newly exposed settings, including `S3ClientSettings.DISABLE_CHUNKED_ENCODING`, `MAX_SYNC_CONNECTIONS_SETTING`, `CONNECTION_ACQUISITION_TIMEOUT`, `MAX_PENDING_CONNECTION_ACQUIRES`, and the connection/timeout settings.
Client settings equality/hash/refine `plugins/repository-s3/src/main/java/org/opensearch/repositories/s3/S3ClientSettings.java`	`refine()` now compares `maxSyncConnections` when deciding reuse; `equals()` updated to include `maxSyncConnections` and use primitive equality for `disableChunkedEncoding`; `hashCode()` updated to include `maxSyncConnections`.
Docs formatting `plugins/repository-s3/README.md`	Minor formatting: added a blank line in the Optional environment variables section.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Verify S3RepositoryPlugin.getSettings() ordering and exact exposed settings.
Confirm test assertions in S3RepositoryPluginTests match the added settings.
Review S3ClientSettings.refine()/equals()/hashCode() for correctness and equals/hashCode contract compliance.

Poem

🐇 I hopped through configs, tidy and bright,
Timeouts and limits now lined up right,
Tests counted carrots one by one,
Hashes and equals, balanced and done,
A happy little hop — the change feels light.

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 16.67% which is insufficient. The required threshold is 80.00%.	You can run `@coderabbitai generate docstrings` to improve docstring coverage.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and specifically describes the main change: registering the disable_chunked_encoding setting in repository-s3 node settings.
Description check	✅ Passed	The description includes the required sections: clear description of what the change achieves, explanation of the problem and fix, testing information, and impact statement.

✨ Finishing touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Post copyable unit tests in a comment

📜 Recent review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 6737526 and 92d39fe.

📒 Files selected for processing (1)

plugins/repository-s3/README.md (1 hunks)

✅ Files skipped from review due to trivial changes (1)

plugins/repository-s3/README.md

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)

GitHub Check: gradle-check

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

coderabbitai

Actionable comments posted: 0

🧹 Nitpick comments (2)

plugins/repository-s3/src/test/java/org/opensearch/repositories/s3/S3RepositoryPluginTests.java (1)

78-79: Keep the assertion, but avoid PR-specific comment wording

The new assertion correctly verifies that DISABLE_CHUNKED_ENCODING is registered in the plugin settings. The inline comment (“New assertion for your bug fix”) is tied to this PR and doesn’t convey long-term intent; consider either removing it or replacing it with something descriptive like “ensure DISABLE_CHUNKED_ENCODING is registered as a node setting”.

plugins/repository-s3/src/main/java/org/opensearch/repositories/s3/S3RepositoryPlugin.java (1)

366-367: Setting registration is correct; clean up the inline bug-fix comment

Including S3ClientSettings.DISABLE_CHUNKED_ENCODING in getSettings() is the right fix to register this node setting. The comment “Fixed the bug in this line” is tied to this PR and doesn’t describe behavior; consider removing it or replacing it with a brief description of what the setting does (e.g., that it controls disabling HTTP chunked encoding for S3 clients).

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between b7f013f and bfa670a.

📒 Files selected for processing (2)

plugins/repository-s3/src/main/java/org/opensearch/repositories/s3/S3RepositoryPlugin.java (1 hunks)
plugins/repository-s3/src/test/java/org/opensearch/repositories/s3/S3RepositoryPluginTests.java (1 hunks)

github-actions · 2025-12-03T18:58:37Z

❌ Gradle check result for bfa670a: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

coderabbitai

Actionable comments posted: 0

🧹 Nitpick comments (1)

plugins/repository-s3/src/main/java/org/opensearch/repositories/s3/S3RepositoryPlugin.java (1)
366-368: DISABLE_CHUNKED_ENCODING registration is correct; consider cleaning up the comment

The addition of S3ClientSettings.DISABLE_CHUNKED_ENCODING to getSettings() is exactly what’s needed for the node setting to be registered; behavior-wise this looks good.

The inline comment // Fixed the bug in this line is not very descriptive and will age quickly. I’d either drop it or replace it with a comment about what the setting does instead of the historical note.

For example, to just remove the historical comment:
-            S3ClientSettings.SIGNER_OVERRIDE,
-            // Fixed the bug in this line
-            S3ClientSettings.DISABLE_CHUNKED_ENCODING,
+            S3ClientSettings.SIGNER_OVERRIDE,
+            S3ClientSettings.DISABLE_CHUNKED_ENCODING,
Or, if you want a descriptive comment:
-            S3ClientSettings.SIGNER_OVERRIDE,
-            // Fixed the bug in this line
-            S3ClientSettings.DISABLE_CHUNKED_ENCODING,
+            S3ClientSettings.SIGNER_OVERRIDE,
+            // Allows disabling AWS SDK chunked encoding for S3-compatible storage
+            S3ClientSettings.DISABLE_CHUNKED_ENCODING,

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between bfa670a and 217b8b7.

📒 Files selected for processing (2)

plugins/repository-s3/src/main/java/org/opensearch/repositories/s3/S3RepositoryPlugin.java (1 hunks)
plugins/repository-s3/src/test/java/org/opensearch/repositories/s3/S3RepositoryPluginTests.java (1 hunks)

🚧 Files skipped from review as they are similar to previous changes (1)

plugins/repository-s3/src/test/java/org/opensearch/repositories/s3/S3RepositoryPluginTests.java

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)

GitHub Check: gradle-check

github-actions · 2025-12-03T19:36:06Z

❌ Gradle check result for 217b8b7: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2025-12-04T05:09:00Z

❌ Gradle check result for 8d42f07: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

… settings Signed-off-by: Aman Gautam <[email protected]>

Gautam-aman · 2025-12-04T05:46:46Z

CI didn't start due to a Jenkins trigger 403 / jq parsing error.
Requesting to retrigger gradle-check manually. Thank you!

github-actions · 2025-12-04T07:20:19Z

❌ Gradle check result for 068a2bd: null

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Gautam-aman · 2025-12-04T07:35:55Z

I ran the suite locally via ./gradlew :plugins:repository-s3:internalClusterTest and it passes successfully.

The CI failure appears to be due to the Jenkins trigger issue (403 / jq parsing).
Requesting maintainers to manually retrigger gradle-check. Thank you!

Gautam-aman · 2025-12-04T14:47:50Z

Thanks for the suggestion! 👍
I will extend this PR to expose all missing S3 client settings (not only disable_chunked_encoding) as recommended.

I’m working on adding:

REQUEST_TIMEOUT_SETTING
CONNECTION_TIMEOUT_SETTING
CONNECTION_TTL_SETTING
MAX_CONNECTIONS_SETTING
MAX_SYNC_CONNECTIONS_SETTING
CONNECTION_ACQUISITION_TIMEOUT
MAX_PENDING_CONNECTION_ACQUIRES
DISABLE_CHUNKED_ENCODING

Will push an update shortly.

Signed-off-by: Aman Gautam <[email protected]>

coderabbitai

Actionable comments posted: 0

🧹 Nitpick comments (1)

plugins/repository-s3/src/main/java/org/opensearch/repositories/s3/S3RepositoryPlugin.java (1)
367-367: Consider revising the comment for accuracy.

The comment "required new settings" is misleading—these settings already exist in S3ClientSettings but were not previously registered. Consider a more accurate comment such as "additional s3 client configuration settings" or "previously unregistered s3 client settings."

Apply this diff to improve the comment:
-            // required new settings
+            // additional s3 client configuration settings

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 068a2bd and 7251e94.

📒 Files selected for processing (3)

plugins/repository-s3/src/main/java/org/opensearch/repositories/s3/S3ClientSettings.java (2 hunks)
plugins/repository-s3/src/main/java/org/opensearch/repositories/s3/S3RepositoryPlugin.java (2 hunks)
plugins/repository-s3/src/test/java/org/opensearch/repositories/s3/S3RepositoryPluginTests.java (1 hunks)

🚧 Files skipped from review as they are similar to previous changes (2)

plugins/repository-s3/src/main/java/org/opensearch/repositories/s3/S3ClientSettings.java
plugins/repository-s3/src/test/java/org/opensearch/repositories/s3/S3RepositoryPluginTests.java

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)

GitHub Check: gradle-check

🔇 Additional comments (1)

plugins/repository-s3/src/main/java/org/opensearch/repositories/s3/S3RepositoryPlugin.java (1)

368-375: Verify that all settings exist in S3ClientSettings.

The addition of these eight settings is essential for allowing users to configure them in opensearch.yml. Ensure that all these setting constants (REQUEST_TIMEOUT_SETTING, CONNECTION_TIMEOUT_SETTING, CONNECTION_TTL_SETTING, MAX_CONNECTIONS_SETTING, MAX_SYNC_CONNECTIONS_SETTING, CONNECTION_ACQUISITION_TIMEOUT, MAX_PENDING_CONNECTION_ACQUIRES, DISABLE_CHUNKED_ENCODING) are properly defined in S3ClientSettings to prevent compilation or runtime issues.

Signed-off-by: Aman Gautam <[email protected]>

github-actions · 2025-12-04T15:38:13Z

❌ Gradle check result for aeceb9a: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: Aman Gautam <[email protected]>

github-actions · 2025-12-04T18:08:32Z

❌ Gradle check result for 12ac7d1: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: Aman Gautam <[email protected]>

github-actions · 2025-12-04T19:08:36Z

❌ Gradle check result for 07f99f1: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

…istered settings Signed-off-by: Aman Gautam <[email protected]>

coderabbitai

Actionable comments posted: 1

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 07f99f1 and 1d67079.

📒 Files selected for processing (1)

plugins/repository-s3/src/main/java/org/opensearch/repositories/s3/S3RepositoryPlugin.java (2 hunks)

🧰 Additional context used

🧬 Code graph analysis (1)

plugins/repository-s3/src/main/java/org/opensearch/repositories/s3/S3RepositoryPlugin.java (1)

plugins/repository-s3/src/main/java/org/opensearch/repositories/s3/S3ClientSettings.java (1)

S3ClientSettings (66-771)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)

GitHub Check: gradle-check

🔇 Additional comments (1)

plugins/repository-s3/src/main/java/org/opensearch/repositories/s3/S3RepositoryPlugin.java (1)

346-371: Unable to verify this review comment due to repository access constraints. The comment makes several specific claims that require codebase examination:

Claims 14 settings were removed from getSettings() (based on an "AI summary" not provided)

Claims 8 settings were promised in PR objectives but only 4 were added

References specific setting names as removed

The provided code snippet shows only the final state of getSettings() without a diff showing what was removed. To properly verify, I would need to examine:

The full git diff showing removed vs. added settings

The actual PR description/objectives

The complete S3ClientSettings.java and S3Repository.java files

Whether the allegedly removed settings are actually defined elsewhere in the codebase

Without access to the actual codebase, I cannot confirm:

Whether these settings were actually removed

Whether they are still accessible through other means

The accuracy of the "8 promised settings" claim

Whether this truly constitutes a breaking change for production systems

The review comment contains actionable concerns, but the factual claims require codebase verification to assess validity.

plugins/repository-s3/src/main/java/org/opensearch/repositories/s3/S3RepositoryPlugin.java

github-actions · 2025-12-04T19:35:02Z

❌ Gradle check result for 1d67079: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

…OUT, TTL, MAX_CONNECTIONS)

coderabbitai

Actionable comments posted: 0

🧹 Nitpick comments (1)

plugins/repository-s3/src/main/java/org/opensearch/repositories/s3/S3RepositoryPlugin.java (1)

348-376: New S3 client settings registration is correct and complete; minor optional comment tweak

The expanded getSettings() block correctly registers all eight S3 client configuration settings (REQUEST_TIMEOUT_SETTING, CONNECTION_TIMEOUT_SETTING, CONNECTION_TTL_SETTING, MAX_CONNECTIONS_SETTING, MAX_SYNC_CONNECTIONS_SETTING, CONNECTION_ACQUISITION_TIMEOUT, MAX_PENDING_CONNECTION_ACQUIRES, DISABLE_CHUNKED_ENCODING) and keeps them grouped logically between the existing base client settings and region/role settings. I don’t see any functional or wiring issues here.

One minor style nit: the comment // additional s3 client configuration settings added in this PR is PR-specific and may age poorly in the codebase. Consider shortening it to something PR-agnostic like // additional S3 client configuration settings so it remains accurate after this PR lands.

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 1d67079 and 6737526.

📒 Files selected for processing (1)

plugins/repository-s3/src/main/java/org/opensearch/repositories/s3/S3RepositoryPlugin.java (3 hunks)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)

GitHub Check: gradle-check

github-actions · 2025-12-05T12:17:48Z

❌ Gradle check result for 6737526: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2025-12-05T12:32:21Z

❌ Gradle check result for 92d39fe: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Gautam-aman · 2025-12-05T12:37:01Z

All S3 plugin tests are passing locally and on CI before cleanup.
The final failure occurs during Jenkins cleanup (docker prune / workspace wipe),
which indicates a flaky infra failure unrelated to the code change.

Requesting a CI re-run. 🙏

Gautam-aman requested a review from a team as a code owner December 3, 2025 18:27

coderabbitai bot reviewed Dec 3, 2025

View reviewed changes

Gautam-aman force-pushed the fix-disable-chunked-encoding branch from bfa670a to 217b8b7 Compare December 3, 2025 19:03

coderabbitai bot reviewed Dec 3, 2025

View reviewed changes

Gautam-aman force-pushed the fix-disable-chunked-encoding branch from 217b8b7 to 8d42f07 Compare December 4, 2025 04:39

Register s3.client.*.disable_chunked_encoding in repository-s3 plugin…

068a2bd

… settings Signed-off-by: Aman Gautam <[email protected]>

Gautam-aman force-pushed the fix-disable-chunked-encoding branch from 8d42f07 to 068a2bd Compare December 4, 2025 05:17

Expose all missing S3 client settings in repository-s3 plugin

7251e94

Signed-off-by: Aman Gautam <[email protected]>

coderabbitai bot reviewed Dec 4, 2025

View reviewed changes

Update comment for clarity

aeceb9a

Signed-off-by: Aman Gautam <[email protected]>

Include max_sync_connections in S3 client refine logic

12ac7d1

Signed-off-by: Aman Gautam <[email protected]>

Fix S3 settings order to prevent CI timeout on docker fixtures

07f99f1

Signed-off-by: Aman Gautam <[email protected]>

Fix duplicate S3 client settings registration by removing already reg…

1d67079

…istered settings Signed-off-by: Aman Gautam <[email protected]>

coderabbitai bot reviewed Dec 4, 2025

View reviewed changes

plugins/repository-s3/src/main/java/org/opensearch/repositories/s3/S3RepositoryPlugin.java Outdated Show resolved Hide resolved

Register missing S3 client settings (REQUEST_TIMEOUT, CONNECTION_TIME…

6737526

…OUT, TTL, MAX_CONNECTIONS)

coderabbitai bot reviewed Dec 5, 2025

View reviewed changes

CI trigger: retrigger check-result

92d39fe

Register s3.client.<client>.disable_chunked_encoding in repository-s3 node settings #20161

Are you sure you want to change the base?

Register s3.client.<client>.disable_chunked_encoding in repository-s3 node settings #20161

Conversation

Gautam-aman commented Dec 3, 2025 • edited by coderabbitai bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Uh oh!

coderabbitai bot commented Dec 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Estimated code review effort

Poem

Pre-merge checks and finishing touches

Uh oh!

coderabbitai bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Dec 3, 2025

Uh oh!

coderabbitai bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Dec 3, 2025

Uh oh!

github-actions bot commented Dec 4, 2025

Uh oh!

Gautam-aman commented Dec 4, 2025

Uh oh!

github-actions bot commented Dec 4, 2025

Uh oh!

Gautam-aman commented Dec 4, 2025

Uh oh!

Gautam-aman commented Dec 4, 2025

Uh oh!

coderabbitai bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Dec 4, 2025

Uh oh!

github-actions bot commented Dec 4, 2025

Uh oh!

github-actions bot commented Dec 4, 2025

Uh oh!

coderabbitai bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actions bot commented Dec 4, 2025

Uh oh!

coderabbitai bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Dec 5, 2025

Uh oh!

github-actions bot commented Dec 5, 2025

Uh oh!

Gautam-aman commented Dec 5, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Register `s3.client.<client>.disable_chunked_encoding` in repository-s3 node settings #20161

Register `s3.client.<client>.disable_chunked_encoding` in repository-s3 node settings #20161

Gautam-aman commented Dec 3, 2025 •

edited by coderabbitai bot

Loading

coderabbitai bot commented Dec 3, 2025 •

edited

Loading