🚀 Pre-release master -> staging_Imparable1

[YuryHrytsuk]

What kind of pre-release?

master branch

Sprint Name

Cheops

Pre-release version

1

Commit SHA

TBD

Planned date

TBD

Did the commit CI succeed?

• The commit CI succeeded.

Motivation

• Weekly staging release

What Changed

• 🐛 [Frontend] Fix: Support group accessRights by @odeimaiz in 🐛 [Frontend] Fix: Support group accessRights #8420
• ✨Dask sidecar: add threads multiplier (⚠️ Devops) by @sanderegg in ✨Dask sidecar: add threads multiplier (⚠️ Devops) #8419
• 🎨 Updates all aiohttp state application keys from string-based keys to type-safe web.AppKey instances by @pcrespov in 🎨 Updates all aiohttp state application keys from string-based keys to type-safe web.AppKey instances #8405
• ♻️🐛🎨 Renaming and redesign of celery task metadata by @bisgaard-itis in ♻️🐛🎨 Renaming and redesign of celery task metadata #8388
• 🐛 [Frontend] Fix: filter tags with empty text by @odeimaiz in 🐛 [Frontend] Fix: filter tags with empty text #8428
• 🎨 [Frontend] Book a call: select a topic first by @odeimaiz in 🎨 [Frontend] Book a call: select a topic first #8422
• ♻️ webserver: Enhance Action Confirmation Token Logic (🚨🗃️) by @pcrespov in ♻️ webserver: Enhance Action Confirmation Token Logic (🚨🗃️) #8150
• ♻️ Refactor: migrate more aiohttp app keys to type-safe web.AppKey by @pcrespov in ♻️ Refactor: migrate more aiohttp app keys to type-safe web.AppKey #8424
• ✨ nicegui rendering aids by @GitHK in ✨ nicegui rendering aids #8427
• ♻️ Refactor groups/classifiers and scicrunch to use asyncpg with service/repository separation by @pcrespov in ♻️ Refactor groups/classifiers and scicrunch to use asyncpg with service/repository separation #8433
• 🐛 fix loss of logs (on project close/websocket disconnect) & add wallet permission check on wallet (on project open) 🚨 by @matusdrobuliak66 in 🐛 fix loss of logs (on project close/websocket disconnect) & add wallet permission check on wallet (on project open) 🚨 #8426
• 🎨 Align map load test with ui workflow by @bisgaard-itis in 🎨 Align map load test with ui workflow #8417
• ✨ Adding generic scheduling capability to dynamic-scheduler [part 1/2] by @GitHK in ✨ Adding generic scheduling capability to dynamic-scheduler [part 1/2] #8307
• 🎨 Increase function rpc call timeouts to 30sec by @bisgaard-itis in 🎨 Increase function rpc call timeouts to 30sec #8440
• 🐛 Fix performance issue when listing files filtering on projects by @bisgaard-itis in 🐛 Fix performance issue when listing files filtering on projects #8443
• 🎨 TaskID -> TaskKey by @bisgaard-itis in 🎨 TaskID -> TaskKey #8409
• 🔒️ Potential fix for code scanning alert no. 95: Clear-text logging of sensitive information by @pcrespov in 🔒️ Potential fix for code scanning alert no. 95: Clear-text logging of sensitive information #8442
• 🎨 New rpc client in the api-server to avoid cross-service calls with the web-server variants (🚨) by @pcrespov in 🎨 New rpc client in the api-server to avoid cross-service calls with the web-server variants (🚨) #8431
• ⬆️ Maintenance: upgrades python tests and tool requirements by @pcrespov in ⬆️ Maintenance: upgrades python tests and tool requirements #8435
• 🐛 Unified the redis_namespace for LRT ⚠️🚨 by @GitHK in 🐛 Unified the redis_namespace for LRT ⚠️🚨 #8450
• 🎨✅ [Maintenance] Add mock handler factory for LicensesRpcApi to fix pact tests by @pcrespov in 🎨✅ [Maintenance] Add mock handler factory for LicensesRpcApi to fix pact tests #8455
• 🎨 Removed noisy logger by @GitHK in 🎨 Removed noisy logger #8429
• ♻️ Refactor: migrate more aiohttp app keys to type-safe web.AppKey (follow up) by @pcrespov in ♻️ Refactor: migrate more aiohttp app keys to type-safe web.AppKey (follow up) #8452
• 🐛 Fix: enforce consistent WEBSERVER_RPC_NAMESPACE in docker-compose to prevent deploy bug by @pcrespov in 🐛 Fix: enforce consistent WEBSERVER_RPC_NAMESPACE in docker-compose to prevent deploy bug #8463
• 🐛 Fix listing function to avoid duplicates by @wvangeit in 🐛 Fix listing function to avoid duplicates  #8460
• ♻️ [Maintenance] Refactor Payments Domain to Use asyncpg Instead of aiopg by @pcrespov in ♻️ [Maintenance] Refactor Payments Domain to Use asyncpg Instead of aiopg #8462
• 🐛 [Frontend] Fix: Open My Account in s4llite by @odeimaiz in 🐛 [Frontend] Fix: Open My Account in s4llite #8468
• 🎨 [Frontend] Support center: Enhance conversation list item by @odeimaiz in 🎨 [Frontend] Support center: Enhance conversation list item #8464
• 🐛 Fix issue with not consumed rabbitmq message by @matusdrobuliak66 in 🐛 Fix issue with not consumed rabbitmq message #8472
• ♻️ [Frontend] Refactor: Message model by @odeimaiz in ♻️ [Frontend] Refactor: Message model #8471
• ✅ [Maintenance] Fixes api-server/tests/unit/pact_broker testing by @pcrespov in ✅ [Maintenance] Fixes api-server/tests/unit/pact_broker testing #8473
• ♻️ [Maintenance] Refactor Studies-dispatcher Domain to Use asyncpg Instead of aiopg by @pcrespov in ♻️ [Maintenance] Refactor Studies-dispatcher Domain to Use asyncpg Instead of aiopg #8467
• 🐛 Agent should be able to read files in directory now by @GitHK in 🐛 Agent should be able to read files in directory now  #8466
• 🎨 feat(api-server): Add TTL caching to authentication repositories 🚨 by @pcrespov in 🎨 feat(api-server): Add TTL caching to authentication repositories 🚨 #8474
• 🐛 [Frontend] Fix: Trash pagination by @odeimaiz in 🐛 [Frontend] Fix: Trash pagination #8478
• ✨ Add Celery task manager to Web Server ⚠️ by @giancarloromeo in ✨ Add Celery task manager to Web Server ⚠️ #8436
• 🐛 [Frontend] Fix: Sim4Life Escalate to Support by @odeimaiz in 🐛 [Frontend] Fix: Sim4Life Escalate to Support #8479
• ✨ dynamic-scheduler can chain operations by @GitHK in ✨ dynamic-scheduler can chain operations #8446
• 🎨 e2e Metamodeling: Improve project creation by @odeimaiz in 🎨 e2e Metamodeling: Improve project creation #8477
• Split functions repo in different files ♻️ by @wvangeit in Split functions repo in different files ♻️ #8483
• 🎨 [Frontend] Enh: Tasks UI by @odeimaiz in 🎨 [Frontend] Enh: Tasks UI #8485
• 🐛 fix unsubscription of project logs for multiple replicas by @matusdrobuliak66 in 🐛 fix unsubscription of project logs for multiple replicas #8458
• 🐛 [Frontend] Fix: invalidate cache after emptying trash by @odeimaiz in 🐛 [Frontend] Fix: invalidate cache after emptying trash #8492
• 🎨 Adds missing features to generic_scheduler by @GitHK in 🎨 Adds missing features to generic_scheduler #8490
• 🚨✨ Implement tracing sampling strategy (🚧 devops 🚧) by @bisgaard-itis in 🚨✨ Implement tracing sampling strategy (🚧 devops 🚧) #8421
• 🐛 Fixes webserver-api error on get_project_services when project services are missing in the catalog by allowing partial failure by @pcrespov in 🐛 Fixes webserver-api error on get_project_services when project services are missing in the catalog by allowing partial failure #8486
• Follow up to osparc.io incident by @bisgaard-itis in Follow up to osparc.io incident #8491
• 🎨 [e2e] Improve MMUX test by @odeimaiz in 🎨 [e2e] Improve MMUX test #8494
• 🎨 adds required initial operation context key by @GitHK in 🎨 adds required initial operation context key #8495
• 🐛 Ensure function execute permission check is performed only once in map endpoint by @bisgaard-itis in 🐛 Ensure function execute permission check is performed only once in map endpoint #8499
• 🎨 [Frontend] New Feature: Transfer Projects by @odeimaiz in 🎨 [Frontend] New Feature: Transfer Projects #8497
• 🐛 Prevent showing full s3 url with credentials in logs by @wvangeit in 🐛 Prevent showing full s3 url with credentials in logs  #8493
• 🐛 webserver: Fix 5XX error in GET /v0/projects/{project_id}/nodes/-/services when project has no nodes by @pcrespov in 🐛 webserver: Fix 5XX error in GET /v0/projects/{project_id}/nodes/-/services when project has no nodes #8501
• 🔨 [Maintenance] Refactor docker-compose.yml to Improve Maintainability via Environment Variable Grouping by @pcrespov in 🔨 [Maintenance] Refactor docker-compose.yml to Improve Maintainability via Environment Variable Grouping #8503
• 🎨 [Support center] reopen closed Fogbugz cases (🗃️) by @matusdrobuliak66 in 🎨 [Support center] reopen closed Fogbugz cases (🗃️) #8482
• Bump astral-sh/setup-uv from 6 to 7 by @dependabot[bot] in Bump astral-sh/setup-uv from 6 to 7 #8509
• ✨ Add ordering to function jobs inside a function job collection by @wvangeit in ✨ Add ordering to function jobs inside a function job collection  #8487
• ✨ Introduce streamed files search by @giancarloromeo in ✨ Introduce streamed files search #8202
• 🎨 [Frontend] Support Center: read/unread conversations by @odeimaiz in 🎨 [Frontend] Support Center: read/unread conversations #8489
• ⬆️ Unblock Mypy version in autoscaling + some more updates by @sanderegg in ⬆️ Unblock Mypy version in autoscaling + some more updates #8517
• 🐛 Fixed issues with cancellation by @GitHK in 🐛 Fixed issues with cancellation #8521
• 🐛 Fixed failing director-v2 when monitoring services by @GitHK in 🐛 Fixed failing director-v2 when monitoring services #8513
• ✨ Introduce chatbot client (⚠️) by @matusdrobuliak66 in ✨ Introduce chatbot client (⚠️) #8516
• 🎨 Check study and solver job status before returning output by @wvangeit in 🎨 Check study and solver job status before returning output #8511
• ⬆️ Upgrade UV to 0.9.x version everywhere by @sanderegg in ⬆️ Upgrade UV to 0.9.x version everywhere #8519
• Autoscaling for dynamic services: Performance improvements for cold starts (⚠️ devops) 🚨🚨🚨 by @sanderegg in Autoscaling for dynamic services: Performance improvements for cold starts (⚠️ devops) 🚨🚨🚨 #8375
• 🎨 Allow case-insensitive filename matching by @giancarloromeo in 🎨 Allow case-insensitive filename matching #8522
• ♻️E2E: Fix DEBUG logging not outputing to the logs by @sanderegg in ♻️E2E: Fix DEBUG logging not outputing to the logs #8523
• Fix/sntz tgs by @odeimaiz in Fix/sntz tgs #8524
• 🐛 Fix: initialize 3rd party libs by @odeimaiz in 🐛 Fix: initialize 3rd party libs #8527
• 🐛 Use Celery prefork pool for CPU-bound tasks by @giancarloromeo in 🐛 Use Celery prefork pool for CPU-bound tasks #8500
• 🐛 [Frontend] Fix: Pass the initial value by @odeimaiz in 🐛 [Frontend] Fix: Pass the initial value #8529
• ✨ AI answers user questions in support center (🗃️⚠️🚨) by @matusdrobuliak66 in ✨ AI answers user questions in support center (🗃️⚠️🚨) #8525
• ✨ [Frontend] Search Files by @odeimaiz in ✨ [Frontend] Search Files #8520
• 🐛Storage: Listing path query grows with number of projects unbounded by @sanderegg in 🐛Storage: Listing path query grows with number of projects unbounded #8542
• 🎨E2E: ensure info logs are shown when testing playwright e2e by @sanderegg in 🎨E2E: ensure info logs are shown when testing playwright e2e #8543
• 🎨🔒️ Add string_types Module for Strongly Validated API String Fields by @pcrespov in 🎨🔒️ Add string_types Module for Strongly Validated API String Fields #8526
• 🎨🐛 [Frontend] PO Center improvements by @odeimaiz in 🎨🐛 [Frontend] PO Center improvements #8537
• Bump actions/setup-node from 5.0.0 to 6.0.0 by @dependabot[bot] in Bump actions/setup-node from 5.0.0 to 6.0.0 #8534
• Bump github/codeql-action from 3 to 4 by @dependabot[bot] in Bump github/codeql-action from 3 to 4 #8508
• 🎨 No more long running http requests while stopping services by @GitHK in 🎨 No more long running http requests while stopping services #8531
• 🔨 Make devel-mode work again and importing in main modules ⚠️ by @pcrespov in 🔨 Make devel-mode work again and importing in main modules ⚠️  #8515
• 🐛 chatbot followups (🗃️) by @matusdrobuliak66 in 🐛 chatbot followups (🗃️) #8541
• 🐛 [BugFix] Properly handle incorrect passwords for existing accounts during product registration by @pcrespov in 🐛 [BugFix] Properly handle incorrect passwords for existing accounts during product registration #8548
• 🐛 [Frontend] Fix: no-read default accessRights for the chatbot user by @odeimaiz in 🐛 [Frontend] Fix: no-read default accessRights for the chatbot user #8551
• 🎨E2E: improve logging of websocket messages by @sanderegg in 🎨E2E: improve logging of websocket messages #8550
• 🐛 [Frontend] Fix: catch error /group/{gid}/users by @odeimaiz in 🐛 [Frontend] Fix: catch error /group/{gid}/users #8552
• 🎨E2E: Adds missing prefix for websocket messages by @sanderegg in 🎨E2E: Adds missing prefix for websocket messages #8554
• ♻️ RabbitMQ: Make sure exclusive queues are not set as durable (⚠️ devops) by @sanderegg in ♻️ RabbitMQ: Make sure exclusive queues are not set as durable (⚠️ devops) #8546
• 🎨Computational backend: Make sure the number of threads of a dask-worker is computed for autoscaling 🚨🚨🚨 by @sanderegg in 🎨Computational backend: Make sure the number of threads of a dask-worker is computed for autoscaling 🚨🚨🚨 #8423
• 🐛 LRT redis cleanup ⚠️🚨 by @GitHK in 🐛 LRT redis cleanup ⚠️🚨 #8539
• 🐛Autoscaling: fixes unknown passing type to dask-scheduler by @sanderegg in 🐛Autoscaling: fixes unknown passing type to dask-scheduler #8556
• 🚑️ Revert 🎨Computational backend: Make sure the number of threads of a dask-worker is computed for autoscaling 🚨🚨🚨 #8423 + 🐛Autoscaling: fixes unknown passing type to dask-scheduler #8556 until later fix and allow for staging release by @sanderegg in 🚑️ Revert #8423 + #8556 until later fix and allow for staging release #8557
• 🎨 Batch api-server->wb-api-server requests in map endpoint in the api-server by @bisgaard-itis in 🎨 Batch api-server->wb-api-server requests in map endpoint in the api-server #8514
• 🎨 Allows guest users to run a project form a template with outputs pushing enabled by @GitHK in 🎨 Allows guest users to run a project form a template with outputs pushing enabled 🗃️ #8555

Devops check ⚠️ devops

• Update docker engine version Update docker engine to major version 26 osparc-ops-environments#1198
  • dalco stag
  • nih stag
• Deploy rabbit in dalco stag Self-hosted Cluster RabbitMQ osparc-ops-environments#1176 --> blocked / not ready YH
• Add dockerhub dv-2 creds Add dockerhub creds env for director-v2 osparc-ops-environments#1208
• Clean up redis 🐛 LRT redis cleanup ⚠️🚨 #8539
• remove (clean) old docker api network 🔨 Make devel-mode work again and importing in main modules ⚠️  #8515 @YuryHrytsuk
• Manually remove Rabbit queues ♻️ RabbitMQ: Make sure exclusive queues are not set as durable (⚠️ devops) #8546 @YuryHrytsuk @sanderegg --> nothing to do YH

e2e testing check 🧪

No response

Summary 📝

• make release-staging name=<sprint_name> version=<version> git_sha=<commit_sha>
  • https://github.com/ITISFoundation/osparc-simcore/releases/new?prerelease=1&target=<commit_sha>&tag=staging_<sprint_name><version>&title=Staging%20<sprint_name><version>
• Draft pre-release
• Announce (add redis key maintenance in every concerned deployment)

  {"start": "2023-02-01T12:30:00.000Z", "end": "2023-02-01T13:00:00.000Z", "reason": "Release ResistanceIsFutile9 "}

• Announce release in Mattermost

  :loud_sound:  Maintenance scheduled for **NAMED_DAY DD. MM from START_TIME - END_TIME**.
  =========================================================================
  
  @all Be aware that you will automatically be logged out and your projects stopped and saved during the maintenance time. Affected:
  *   [https://staging.osparc.io](https://staging.osparc.io/)
  *   [https://https://staging.s4l-lite.io/](https://https://staging.s4l-lite.io//)
  
  and on premises:
  *   [https://osparc-staging.speag.com](https://osparc-staging.speag.com/)
  *   [https://tip-staging.speag.com](https://tip-staging.speag.com/)
  *   [https://s4l-staging.speag.com](https://s4l-staging.speag.com/)
  *   [https://s4l-lite-staging.speag.com](https://s4l-lite-staging.speag.com/)
  
  
  Reason: Scheduled staging-release of STAGING_NAME_AND_VERSION.
  
  Thanks for your understanding and sorry for the inconveniences,
  
  Your friendly oSparc Team
  
  

Releasing

• Release (release draft)
• Check Release CI
• Check hanging sidecars. Helper command to run in director-v2 CLI simcore-service-director-v2 close-and-save-service <uuid>
• Check deployed
  • aws deploy
  • dalco deploy
• Delete announcement
• Check e2e runs
• Announce

https://github.com/ITISFoundation/osparc-simcore/releases/tag/staging_<sprint_name><version>

[matusdrobuliak66]

• @GitHK 🐛 Unified the redis_namespace for LRT ⚠️🚨 #8450 / 🐛 LRT redis cleanup ⚠️🚨 #8539
• @bisgaard-itis please manual test in staging TRACING_SAMPLING_PROBABILITY=0.1
• @sanderegg merge Autoscaling for dynamic services: Performance improvements for cold starts (⚠️ devops) 🚨🚨🚨 #8375

After release:

• @matusdrobuliak66 ✨ AI answers user questions in support center (🗃️⚠️🚨) #8525 (DB change)

[YuryHrytsuk]

@matusdrobuliak66 I am done. Feel free to close after you are done with After release activities mentioned above