Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SacessOptimizer: More efficient saving of intermediate results #1529

Merged
merged 1 commit into from
Dec 10, 2024

Conversation

dweindl
Copy link
Member

@dweindl dweindl commented Nov 27, 2024

Don't write old local optima again and again in every iteration to save time and to avoid huge fragmented files.

Some other smaller modifications/fixes:

  • make sacess_history plotting more robust.
  • Set SacessOptimizer.exit_flag

@dweindl dweindl self-assigned this Nov 27, 2024
@codecov-commenter
Copy link

codecov-commenter commented Nov 27, 2024

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

Attention: Patch coverage is 84.61538% with 4 lines in your changes missing coverage. Please review.

Project coverage is 84.33%. Comparing base (25c2f84) to head (41e91a8).

Files with missing lines Patch % Lines
pypesto/visualize/optimizer_history.py 50.00% 3 Missing ⚠️
pypesto/optimize/ess/sacess.py 94.73% 1 Missing ⚠️

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #1529      +/-   ##
===========================================
- Coverage    84.35%   84.33%   -0.02%     
===========================================
  Files          163      163              
  Lines        14013    14034      +21     
===========================================
+ Hits         11820    11836      +16     
- Misses        2193     2198       +5     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@dweindl dweindl force-pushed the sacess_tmpfiles branch 2 times, most recently from 0220e10 to 08dc47f Compare November 28, 2024 12:03
@dweindl dweindl marked this pull request as ready for review November 28, 2024 12:05
@dweindl
Copy link
Member Author

dweindl commented Dec 9, 2024

Hi, could somebody please review?

@@ -683,20 +692,14 @@ def run(
# perform one ESS iteration
ess._do_iteration()

if self._tmp_result_file:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another hyper parameter like trace_save_iter for ESS?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not really. This class is not intended to be used directly, only through SacessOptimizer. In that case, this will always evaluate to True. It's rather for unit testing purposes where one doesn't want/need any intermediary result files.

Copy link
Member Author

@dweindl dweindl Dec 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or was that a request to make saving intermediate result optional in SacessOptimizer? (Can be done, but I'd do that separately then).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes @dweindl sorry, shaky mouse. Don't see it be super urgent to add this, but agreed, add separately.

Copy link
Contributor

@stephanmg stephanmg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remaining looks good. Thanks!

Don't write old local optima again and again in every iteration to save time and avoid huge fragmented files.

Some other smaller modifications/fixes:
* make sacess_history plotting more robust.
* Set SacessOptimizer.exit_flag
@dweindl dweindl enabled auto-merge December 9, 2024 10:22
@dweindl dweindl disabled auto-merge December 10, 2024 09:15
@dweindl dweindl merged commit 2e5b8de into ICB-DCM:develop Dec 10, 2024
18 checks passed
@dweindl dweindl deleted the sacess_tmpfiles branch December 10, 2024 09:16
This was referenced Jan 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants