Skip to content

Improved Alert Handling #688

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 16 commits into from
Feb 19, 2025
Merged

Improved Alert Handling #688

merged 16 commits into from
Feb 19, 2025

Conversation

wk9874
Copy link
Collaborator

@wk9874 wk9874 commented Feb 13, 2025

Improved Alert Handling

Improves the alert deduplication - a new query parameter 'deduplicate' can be provided to the post request, which will return a 409 CONFLICT if that alert definition already exists on the server.

Made it so that this is the default behaviour, assigning a self._params = {'deduplicate': True} in the newmethod of each alert type. Could not add it to theinitof the alert as this was then overriden by theself_params = {}` in the BaseObject init.

Also added a new attach_to_run parameter to all create_xyz_alert methods in the Run class - if disabled, the alert will be created on the server but not linked to any run. Because of this, I also removed the check_run_initialized decorator from these methods, and moved it to the attach_to_run method. The benefit of this is you can now start a run in disabled mode to define alerts on the server without creating a whole new run to do so. This is used in the integrations repo to validate and define alerts up front, before then using add_alert to attach relevent ones to simulation, epoch or evaluation runs. The old way to do it would have only allowed the alerts to be validated when created and attached to a run, meaning eg you might spend hours training a model and then simvue crashes because the alert you defined for your evaluation run was invalid.

Also spotted a couple of other bugs:

  • The getter for RunObject.alerts returned a generator, but the setter expected a list - changed these to both be lists for consistency.

  • In add_alert, there was a line set(run.sv_obj.alerts + [id], where id was already a list - so removed the extraneous square brackets

  • add_alert would never work when passed a list of names since it still expected Alert.get to return the old dict format (instead of the current tuple format) - fixed this by adding [1]'s so that it would look at the alert element of the tuple

  • get_alert_details() will only work for online runs, so added a check if used when offline and raises an exception. Also added an 'if offline' check to run.alerts so that it will just return the list of alert IDs, instead of doing a dict comprehension from the server returned info

  • add_alerts now always passes a full list of alerts which should be attached to the run at any time to the server endpoint, and the server endpoint has been changed to overwrite the list instead of appending to it (to stop duplicate alerts being added)

  • Removed _attach_alert_to_run() - now just uses add_alerts

  • Added test for add_alerts functionality

Closes #678
Closes #693
Closes #694
Closes #695
Closes #696
Closes #630 since create_xyz_alert can now be used with a disabled run and attach_to_run = False to validate and create alerts up front
Closes #135

@wk9874 wk9874 requested a review from kzscisoft February 13, 2025 17:38
@wk9874 wk9874 changed the base branch from dev to v2.0 February 13, 2025 17:38
@wk9874 wk9874 marked this pull request as draft February 14, 2025 17:23
@wk9874 wk9874 marked this pull request as ready for review February 17, 2025 14:22
@wk9874 wk9874 changed the title Add improved Alert deduplication Improved Alert Handling Feb 18, 2025
@wk9874 wk9874 linked an issue Feb 18, 2025 that may be closed by this pull request
@wk9874 wk9874 mentioned this pull request Feb 18, 2025
3 tasks
@wk9874 wk9874 linked an issue Feb 19, 2025 that may be closed by this pull request
@wk9874 wk9874 added this to the v2.0.0-rc1 milestone Feb 19, 2025
@kzscisoft kzscisoft self-requested a review February 19, 2025 13:52
Copy link
Collaborator

@kzscisoft kzscisoft left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please solve conflict

@kzscisoft kzscisoft assigned kzscisoft and wk9874 and unassigned kzscisoft Feb 19, 2025
@wk9874 wk9874 merged commit 3639a4f into v2.0 Feb 19, 2025
1 check passed
@kzscisoft kzscisoft deleted the wk9874/alert_deduplication branch May 12, 2025 07:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants