Skip to content

Remove maps built on each call of some conversion functions. #3377

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

svart-riddare
Copy link

The said map should have been static as a 'TODO' in the code explains. Since the static keyword was removed when the std::map was changed to an Aws::Map, the map is rebuilt at each calls, which is a lot of work, and calls to malloc/free. Although the search is now linear instead of logarithmic, the number of entries is quite small and it's always better than rebuilding the map at each call.

This saves about 300 mallocs per PutObject request for example.

Issue #, if available:

Description of changes:

Check all that applies:

  • Did a review by yourself.
  • Added proper tests to cover this PR. (If tests are not applicable, explain.)
  • Checked if this PR is a breaking (APIs have been changed) change.
  • Checked if this PR will not introduce cross-platform inconsistent behavior.
  • Checked if this PR would require a ReadMe/Wiki update.

Check which platforms you have built SDK on to verify the correctness of this PR.

  • Linux
  • Windows
  • Android
  • MacOS
  • IOS
  • Other Platforms

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

static const char HTTP_CLIENT_METRICS_UPLOAD_SPEED[] = "UploadSpeed";
static const char HTTP_CLIENT_METRICS_UNKNOWN[] = "Unknown";

using namespace Aws::Utils;
HttpClientMetricsType GetHttpClientMetricTypeByName(const Aws::String& name)
static const Aws::Array<std::pair<HttpClientMetricsType, const char*>, 12> httpClientMetricsNames =
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure which casing I should use.

Copy link
Contributor

@sbiscigl sbiscigl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One comment to use std::find_if to preserve old behavior, but outside of that, really appriciate you taking the time to address a TODO that we left. sorry that we left it, and thanks for taking the time to do it.

}
return Aws::String(it->second.c_str());
assert(httpClientMetricsNames[static_cast<int>(type)].first == type);
return Aws::String(httpClientMetricsNames[static_cast<int>(type)].second);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would prefer you to use the std::find lookup i.e.

const auto metricName = std::find_if(httpClientMetricsNames.begin(),
  httpClientMetricsNames.end(),
  [&type](const std::pair<HttpClientMetricsType, const char *>& nameEntry) -> bool { return nameEntry.first == type; });
if(metricName == httpClientMetricsNames.end()) {
  return HTTP_CLIENT_METRICS_UNKNOWN;
}
return metricName->second;

as this preserves the behavior from before. i recognize that the map logically doesnt have any illegal access, but this makes sure that there is a defined behavior for when none are found when assertions are disabled.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, I was unsure whether you wanted to prevent an array out of bound index, or to prevent someone adding some entries in the constant table not in the correct order.

I have added a check for this first case; I can of course put an std::find if you want to prevent against the second case.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

or to prevent someone adding some entries in the constant table not in the correct order.

yeah my main concern is that someone edits this out of order, creates a binary incompatibility and it shows up in downstream package managers. I think we can change it ourselves to std::find if we find that is the case down the road, and leave the PR as is for now

@svart-riddare svart-riddare force-pushed the remove-unexpected-memory-allocations branch from 87cc114 to 8eb2e6c Compare April 14, 2025 17:08
The said map should have been static as a 'TODO' in the code explains. Since the static keyword was removed when the `std::map` was changed to an `Aws::Map`, the map is rebuilt at each calls, which is a lot of work, and calls to `malloc`/`free`. Although the search is now linear instead of logarithmic, the number of entries is quite small and it's always better than rebuilding the map at each call.

This saves about 300 mallocs per PutObject request for example.
@sbiscigl sbiscigl force-pushed the remove-unexpected-memory-allocations branch from 8eb2e6c to 332d9f0 Compare April 15, 2025 14:06
@SergeyRyabinin SergeyRyabinin merged commit 4825e10 into aws:main Apr 15, 2025
3 of 4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants