1290 add ABM visualization from paper to memilio plot #1291

xsaschako · 2025-05-27T22:20:54Z

Changes and Information

Please briefly list the changes (main added features, changed items, or corrected bugs) made:

add visualization for the abm runs.

If need be, add additional information and what the reviewer should look out for in particular:

Merge Request - Guideline Checklist

Please check our git workflow. Use the draft feature if the Pull Request is not yet ready to review.

Checks by code author

Every addressed issue is linked (use the "Closes #ISSUE" keyword below)
New code adheres to coding guidelines
No large data files have been added (files should in sum not exceed 100 KB, avoid PDFs, Word docs, etc.)
Tests are added for new functionality and a local test run was successful (with and without OpenMP)
Appropriate documentation for new functionality has been added (Doxygen in the code and Markdown files if necessary)
Proper attention to licenses, especially no new third-party software with conflicting license has been added
(For ABM development) Checked benchmark results and ran and posted a local test above from before and after development to ensure performance is monitored.

Checks by code reviewer(s)

Corresponding issue(s) is/are linked and addressed
Code is clean of development artifacts (no deactivated or commented code lines, no debugging printouts, etc.)
Appropriate unit tests have been added, CI passes, code coverage and performance is acceptable (did not decrease)
No large data files added in the whole history of commits(files should in sum not exceed 100 KB, avoid PDFs, Word docs, etc.)
On merge, add 2-5 lines with the changes (main added features, changed items, or corrected bugs) to the merge-commit-message. This can be taken from the briefly-list-the-changes above (best case) or the separate commit messages (worst case).

…on runs

…es script

…rove function structure, and add new plotting capabilities for location types.

Copilot

Pull Request Overview

This PR adds new visualization modules to support ABM simulation output analysis for infection states and ICU/death comparisons. Key changes include:

Introduction of plotAbmInfectionStates.py for visualizing infection states and spatial location types.
Introduction of plotAbmICUAndDeadComp.py for comparing ICU occupancy and deaths between simulation results and real-world data.
Addition of helper functions for file loading and axis formatting to support the new visualizations.

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
pycode/memilio-plot/memilio/plot/plotAbmInfectionStates.py	New module for plotting infection states and location types from simulation HDF5 data.
pycode/memilio-plot/memilio/plot/plotAbmICUAndDeadComp.py	New module for plotting and comparing ICU occupancy and cumulative deaths with real-world data.

Comments suppressed due to low confidence (2)

pycode/memilio-plot/memilio/plot/plotAbmICUAndDeadComp.py:71

The variable 'age_grous_string' appears to be misspelled. Consider renaming it to 'age_groups_string' for clarity.

age_grous_string = ['Age 0-4', 'Age 5-14', 'Age 15-34', 'Age 35-59', 'Age 60-79', 'Age 80+'] # Adjust as needed

pycode/memilio-plot/memilio/plot/plotAbmICUAndDeadComp.py:30

Consider using a context manager (e.g., 'with h5py.File(..., 'r') as f:') when opening HDF5 files to ensure they are properly closed after reading.

f_p50 = h5py.File(path+"/infection_state_per_age_group/0/p50/Results.h5", 'r')

pycode/memilio-plot/memilio/plot/plotAbmICUAndDeadComp.py

… documentation, and improve function argument handling for better usability

…st cases for loading H5 results and plotting functions

…ments for clarity

…ll_requires

…th comparison plots

codecov · 2025-05-27T23:05:21Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 97.28%. Comparing base (83c57ad) to head (392ec54).

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #1291   +/-   ##
=======================================
  Coverage   97.28%   97.28%           
=======================================
  Files         168      168           
  Lines       14858    14858           
=======================================
  Hits        14454    14454           
  Misses        404      404

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

…arguments for better flexibility

mknaranja · 2025-05-30T10:42:32Z

Thanks for bringing this close to our previous plot style from the assessment paper. @jubicker Could you have a look for the detailed review?

pycode/memilio-plot/memilio/plot/plotAbmInfectionStates.py

pycode/memilio-plot/memilio/plot_test/test_plot_plotAbmInfectionStates.py

…ilio-plot

jubicker · 2025-06-16T14:06:54Z

pycode/memilio-plot/memilio/plot/plotAbmInfectionStates.py

+#      * @brief Log the TimeSeries of the number of Person%s in an #InfectionState.
+#      * @param[in] sim The simulation of the abm.
+#      * @return A pair of the TimePoint and the TimeSeries of the number of Person%s in an #InfectionState.
+#      */


The @return description is still wrong

jubicker · 2025-06-16T14:11:17Z

pycode/memilio-plot/memilio/plot/plotAbmInfectionStates.py

+        xtick_step=150,
+        y05=None, y95=None, show_90=False):
+    """ Plots infection states with percentiles and improved styling. """
+    plt.figure('Infection_states')


Still missing

jubicker · 2025-06-16T14:12:56Z

pycode/memilio-plot/memilio/plot_test/test_plot_plotAbmInfectionStates.py

+            abm.plot_infections_loc_types_average('dummy_path')
+            assert mock_ax.plot.called
+            assert mock_ax.set_xticks.called
+            assert mock_ax.set_xticklabels.called


jubicker · 2025-06-16T14:16:56Z

pycode/memilio-plot/memilio/plot_test/test_plot_plotAbmInfectionStates.py

+        # Check that plot_infection_states was called with correct data structure
+        states_call_args = mock_states.call_args
+        self.assertIsNotNone(states_call_args)
+        self.assertEqual(len(states_call_args[0]), 7)  # x, y50, y25, y75


I don't get the comment. There are for arguments but the assertion says 7

jubicker · 2025-06-16T14:17:12Z

pycode/memilio-plot/memilio/plot_test/test_plot_plotAbmInfectionStates.py

+        indiv_call_args = mock_indiv.call_args
+        self.assertIsNotNone(indiv_call_args)
+        # x, p50_bs, p25_bs, p75_bs
+        self.assertEqual(len(indiv_call_args[0]), 5)


Same here as above

jubicker · 2025-06-16T14:19:34Z

pycode/memilio-plot/memilio/plot_test/test_plot_plotAbmInfectionStates.py

+                            "fill_between should be called for confidence intervals")
+            fill_calls = mock_ax.fill_between.call_args_list
+            # Should have calls for both 50% and 90% confidence intervals if show_90=True
+            self.assertGreater(len(fill_calls), 0)


You do not check if it was called for the 90% confidence intervall. Please check if it was called exactly 2 times

jubicker · 2025-06-16T14:20:56Z

pycode/memilio-plot/memilio/plot_test/test_plot_plotAbmInfectionStates.py

+            expected_ticks = np.arange(0, len(x), 2)  # xtick_step=2
+            np.testing.assert_array_equal(xticks_call, expected_ticks)
+
+            # Verify xticklabels are set correctly


You only test whether the xtick labels have the correct size not if they are called correctly

jubicker · 2025-06-16T14:21:38Z

pycode/memilio-plot/memilio/plot_test/test_plot_plotAbmInfectionStates.py

+                show90=True
+            )
+
+            # Verify subplot was called with correct dimensions


Comment does not suit to code

Changed it a bit

jubicker · 2025-06-23T06:46:05Z

There is one CI check failing (Which is not failing on the main).

…ilio-plot

xsaschako added 3 commits May 27, 2025 21:02

Add scripts for analyzing infection states and ICU data from simulati…

840303d

…on runs

Remove unused imports and enable plotting functions in infection stat…

c7ffab3

…es script

Refactor infection states plotting script: enhance documentation, imp…

e986d96

…rove function structure, and add new plotting capabilities for location types.

xsaschako requested review from reneSchm and Copilot May 27, 2025 22:20

xsaschako self-assigned this May 27, 2025

xsaschako linked an issue May 27, 2025 that may be closed by this pull request

Add ABM visualization from paper to MEmilio-plot #1290

Open

2 tasks

Copilot AI reviewed May 27, 2025

View reviewed changes

pycode/memilio-plot/memilio/plot/plotAbmICUAndDeadComp.py Outdated Show resolved Hide resolved

xsaschako added 6 commits May 28, 2025 00:23

Refactor plotAbmInfectionStates.py: update authorship, enhance module…

6e4c6fe

… documentation, and improve function argument handling for better usability

Add unit tests for plotAbmInfectionStates: implement comprehensive te…

663a189

…st cases for loading H5 results and plotting functions

Refactor setup.py: enhance PylintCommand documentation and update com…

28c3187

…ments for clarity

Fix comment in setup.py: update example for Excel file types in insta…

2ba672e

…ll_requires

Remove plotAbmICUAndDeadComp.py: delete unused script for ICU and dea…

baa272b

…th comparison plots

formatting

20b651e

xsaschako added 2 commits May 28, 2025 20:47

Implement martins suggestions

706aea0

Refactor plotAbmInfectionStates.py: update paths to use command line …

ab2e0e8

…arguments for better flexibility

kilianvolmer changed the title ~~1290 add abm visualization from paper to memilio plot~~ 1290 add ABM visualization from paper to memilio plot Jun 2, 2025

xsaschako requested review from jubicker and removed request for reneSchm June 9, 2025 08:58

jubicker reviewed Jun 12, 2025

View reviewed changes

xsaschako added 6 commits June 14, 2025 12:02

Julias suggestions p1

e4df6cb

julia review p2

c5cf7dc

julia review pt3

f5dffd6

Merge branch 'main' into 1290-add-abm-visualization-from-paper-to-mem…

f124f04

…ilio-plot

format

75e1105

fix tests

ddb9cb4

jubicker reviewed Jun 16, 2025

View reviewed changes

julia review pt5

c2433fc

xsaschako requested a review from jubicker June 21, 2025 14:04

xsaschako added 2 commits June 21, 2025 16:09

julia review pt7

606ba7f

fix: improve error message for missing file in fake filesystem

e316b6a

xsaschako added 3 commits June 23, 2025 12:26

Merge branch 'main' into 1290-add-abm-visualization-from-paper-to-mem…

a446792

…ilio-plot

Update test_surrogatemodel_ode_secir_groups.py

8a002c7

Update test_surrogatemodel_ode_secir_groups.py

392ec54

1290 add ABM visualization from paper to memilio plot #1291

Are you sure you want to change the base?

1290 add ABM visualization from paper to memilio plot #1291

Uh oh!

Conversation

xsaschako commented May 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes and Information

Merge Request - Guideline Checklist

Checks by code author

Checks by code reviewer(s)

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

codecov bot commented May 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

mknaranja commented May 30, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

xsaschako Jun 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jubicker commented Jun 23, 2025

Uh oh!

Uh oh!

xsaschako commented May 27, 2025 •

edited

Loading

codecov bot commented May 27, 2025 •

edited

Loading

xsaschako Jun 21, 2025 •

edited

Loading