🐛 Concurrent S3 bucket creation attempt

[giancarloromeo]

What do these changes do?

This PR makes the setup of S3 in storage more robust, especially when trying to create the Bucket.

It would be ideal to move the creation out of the service, in a script, and not in the service logic.

Related issue/s

• fixes The sto-worker service fails to start due to concurrent S3 bucket creation during startup #8043

How to test

Dev-ops

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 68.30%. Comparing base (0fc43f7) to head (f6b5fef).

❗ There is a different number of reports uploaded between BASE (0fc43f7) and HEAD (f6b5fef). Click for more details.

HEAD has 31 uploads less than BASE

Flag	BASE (0fc43f7)	HEAD (f6b5fef)
unittests	32	1

Additional details and impacted files

@@             Coverage Diff             @@
##           master    #8045       +/-   ##
===========================================
- Coverage   88.13%   68.30%   -19.83%     
===========================================
  Files        1811      753     -1058     
  Lines       69732    35235    -34497     
  Branches     1258      176     -1082     
===========================================
- Hits        61459    24068    -37391     
- Misses       7909    11109     +3200     
+ Partials      364       58      -306     

Flag	Coverage Δ
integrationtests	64.34% <ø> (+0.05%)	⬆️
unittests	86.39% <100.00%> (-0.33%)	⬇️

Components	Coverage Δ
api	∅ <ø> (∅)
pkg_aws_library	∅ <ø> (∅)
pkg_celery_library	∅ <ø> (∅)
pkg_dask_task_models_library	∅ <ø> (∅)
pkg_models_library	∅ <ø> (∅)
pkg_notifications_library	∅ <ø> (∅)
pkg_postgres_database	∅ <ø> (∅)
pkg_service_integration	∅ <ø> (∅)
pkg_service_library	∅ <ø> (∅)
pkg_settings_library	∅ <ø> (∅)
pkg_simcore_sdk	76.86% <ø> (-8.13%)	⬇️
agent	∅ <ø> (∅)
api_server	∅ <ø> (∅)
autoscaling	∅ <ø> (∅)
catalog	∅ <ø> (∅)
clusters_keeper	∅ <ø> (∅)
dask_sidecar	∅ <ø> (∅)
datcore_adapter	∅ <ø> (∅)
director	∅ <ø> (∅)
director_v2	77.77% <ø> (-13.18%)	⬇️
dynamic_scheduler	∅ <ø> (∅)
dynamic_sidecar	88.33% <ø> (-1.77%)	⬇️
efs_guardian	∅ <ø> (∅)
invitations	∅ <ø> (∅)
payments	∅ <ø> (∅)
resource_usage_tracker	∅ <ø> (∅)
storage	86.39% <100.00%> (+0.02%)	⬆️
webclient	∅ <ø> (∅)
webserver	59.08% <ø> (-29.60%)	⬇️

---

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0fc43f7...f6b5fef. Read the comment docs.

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[sanderegg]

Please check my comments.

Also I do not agree with the "it should be done in a separate script". Why?

[giancarloromeo]

@Mergifyio queue

[mergify]

queue

🛑 The pull request has been removed from the queue default

The following conditions don't match anymore:

• any of: [🔀 queue conditions]
  • all of: [📌 queue conditions of queue default]
    • any of: [🛡 GitHub branch protection]
      • check-neutral = deploy to dockerhub
      • check-skipped = deploy to dockerhub
      • check-success = deploy to dockerhub
    • any of: [🛡 GitHub branch protection]
      • check-neutral = system-tests
      • check-skipped = system-tests
      • check-success = system-tests
    • any of: [🛡 GitHub branch protection]
      • check-neutral = integration-tests
      • check-skipped = integration-tests
      • check-success = integration-tests

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[mergify]

This pull request has been removed from the queue for the following reason: checks failed.

The merge conditions cannot be satisfied due to failing checks:

You may have to fix your CI before adding the pull request to the queue again.
If you update this pull request, to fix the CI, it will automatically be requeued once the queue conditions match again.
If you think this was a flaky issue instead, you can requeue the pull request, without updating it, by posting a @mergifyio requeue comment.