Update AWS Dependencies (Welcome to the Jungle) #1019
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
michaeldjeffrey
requested review from
madninja,
macpie,
bbalser,
jeffgrunewald and
kurotych
kurotych
approved these changes
Jul 18, 2025
michaeldjeffrey
force-pushed
the
mj/update-aws-deps
branch
from
9275b33 to
76219a1
Compare
macpie
approved these changes
Jul 22, 2025
|
Given the issue with building a release with the updated aws dependencies (mainly |
It might be larger than the other error variants enough that clippy is complaining about it.
I think it would be a good idea to roll this into file-store at some point to remove the need to keep them in line.
Path Style requests are being slowly deprecated. Fortunately, the s3_config still provides a way to override this setting. This is mostly for testing locally with minio or localstack, as you would need to setup some DNS rules for your localhost to get virtual-hosted style paths to work. https://docs.aws.amazon.com/AmazonS3/latest/userguide/VirtualHosting.html
This commit exists with both version so they can be compared in history, and in to show that there was a time where it was known that both functions generated the same token.
- remove old aws crates - consolidate signing errors - remove invalid auth token error, we construct the url almost statically
and update clippy warnings
Group them all together to try and reduce the amount of version hunting we need to do next time we need to update a dep.
We can remove this lint once # hyperium/tonic#2253 has a PR branch and is merged. And we can update from tonic `0.10` to `0.13.x`.
It has been removed in updated version of aws sdks, there are ways to handle it, but they're much more involved. We can test with updated versions of loading credentials, and it we're still getting timeout failures for price tracker, we can consider adding it back in with the new style.
We have an array, and try_from works on a slice of u8.
tonic is no longer exported from helium-proto, we define it as a dep, and it picks up it's version from helium-lib.
it will pick up it's version from the re-export of http-serde
michaeldjeffrey
force-pushed
the
mj/update-aws-deps
branch
from
94cc34e to
b9a3a5f
Compare
|
Latest set of updates caused by
|
- redundant return statements - confusing elided lifetime - unused test struct
This has a version of cc that is not bugged.
|
In effort to get these changes landed before the full switchover of iot to kubernetes. Builds are now using the |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Still to be done
Merge Update crypto deps helium-crypto-rs#74Thread it through https://github.com/helium/helium-wallet-rsVersion Updates
aws-config0.51.01.8.2aws-sdk-sts0.21.01.76.0aws-credential-types0.54.11.2.4aws-smithy-http0.51.0aws-types0.51.01.3.7aws-sig-auth0.51.0aws-sigv40.51.11.3.3aws-sdk-s30.21.01.98.0aws-smithy-types-convert0.60.9Helium Crypto
Updating AWS deps was initially fraught with danger because of crypto libraries used for signing. Mainly the
signaturecrate that provided theSignaturetrait.See: helium/helium-crypto-rs#74
Clippy Updates
1.86.0aws-sdk-stshas a msvr of1.86.0.rust-toolchain.tomlwas updated, along withcoverage-point-calculatorandcustom-tracingfor new clippy warnings.Those were
path.next_back()andstd::iter::repeat_n().1.87.0Box.file_store::Error::other()constructor1.88.0String formatting updates.
Clippy enforces inlining a variable into
format!when possible.clippy.tomlcargo
1.87.0added a clippy lint for large err variants in results (default: 128).https://rust-lang.github.io/rust-clippy/master/index.html#/large_err
Until Tonic updates, I've set clippy to report on variants 1 byte larger than
tonic::Status(176 bytes).We can remove this lint once hyperium/tonic#2253 has a PR branch and is merged. And we can update from tonic
0.10to0.13.x.DB Store
generate_rds_iam_token()needed to be updated to useaws-sigv4instead ofaws-sig-auth.This commit (64632a3) was the point in time where I had both versions running to ensure they produced the same iam token.
File Store
features=localand path style enforcementThe
localfeature flag no longer includesaws-types.Credentialsare now provided by their ownaws-sdk-*crate.The
localfeature now enforces path style access to s3 buckets.Path style access is a slow deprecation being replaced by DNS resolution for buckets.
However, when running against a localstack or minio, unless you want to setup some custom DNS rules on in your dev environment, path style access is still the way to go.
Boxing AWS Errors
Clippy has started complaining about Errors and Results that have large variants.
The compiler doesn't know what error might be throw so has to allocate enough space for the largest possible error.
One of the ways around this, is to
Boxpotentially large errors.into_stream_03x()Somewhere around
aws-sdk-s3@1the paginated version ofListObjectsV2lost the ability to be turned into a flat stream.aws-smithy-types-convertwas provided as a way to bring that back.smithy-lang/smithy-rs#3299 explains the bizarre naming.