Update cset-workflow GUI organisation with clearer labelling for "General setup", "Cycling and Models", diagnostic-type and evaluation-type specific option panels

[ukmo-huw-lewis]

Reviewed and refined rose GUI layout.

• Most changes connected to updating meta/rose-meta.conf.jinja2 and meta/diagnostics/rose-meta.conf to give a more ordered flow to GUI panels and ordering of items.
• Have checked for the triggering of related items in GUI and working as expected.
• Have condensed the x3 options popping up for case aggregation to series of T/F buttons for users to select which versions of aggregation requested. Aim was to use new includes/case_aggregation_options.cylc to translate rose-suite.conf inputs to the previous env variables used in the other includes/*.cylc files. May be tempting to create new issue to further rationalise number of includes/ files, but will leave that for later proposal.

Contribution checklist

Aim to have all relevant checks ticked off before merging. See the developer's guide for more detail.

• Documentation has been updated to reflect change.
• New code has tests, and affected old tests have been updated.
• All tests and CI checks pass.
• Ensured the pull request title is descriptive.
• Conda lock files have been updated if dependencies have changed.
• Attributed any Generative AI, such as GitHub Copilot, used in this PR.
• Marked the PR as ready to review.

[github-actions]

[ukmo-huw-lewis]

Testing:

Sample outputs can be seen at https://wwwspice/~huw.lewis/CSET/CSET_WAfr_RAL3LFRic/
I would still like to check that all outputs correspond to requested rose-suite.conf settings (particularly on aggregation aspects).

If wishing to explore rose GUI locally, try rose edit -C /home/users/huw.lewis/CSET/CSET/cset-workflow

Sample screenshots (compare with originals in issue):

Top panel view:

Diagnostic options home panel (would allow expansion to include future aspects like specifying subarea selection, regridding etc):

Separating out "surface fields", "pressure fields", "model level fields" panels:
(suggestion that "Quicklooks" was getting overly-long, and down-playing importance or future extensions of 'quicklook')

Capturing new panel for any diagnostics in "other" as implying more bespoke diagnostic methods, that propose should assume to have their own GUI panel rather than invite users to look through potentially long future list of disparate options connected to different methods etc:

Documentation
Please advise if any documentation changes would be required if details currently captured in docs have changed as result of proposed gui changes.

[ukmo-huw-lewis]

Recognise likely conflicts with #1303, but think proposed changes may make visibility of radio buttons for histograms (and other future additions to plotting options) easier to navigate and implement etc.

[ukmo-huw-lewis]

My cset-workflow/run19 testing (with James W Africa data) failed on following tasks:

• Histogram_series_agg_m1_lightning
• Histogram_series_agg_m1_reflectivity (and for model m2)
• Histogram_series_agg_m1_surface_micro_rainfall_rate (and for model m2)
• Histogram_series_agg_m1_surface_micro_snowfall_rate (and for model m2)
• Histogram_series_agg_m2_high_type_cloud_area_fraction
• Histogram_series_agg_m2_low_type_cloud_area_fraction

All due to "Histogram can only plot a single cube" errors.
Appears that read cube was picking up 2 input cubes for these variables, likely with different time processing (e.g. inst + mean). Flagging here in case that error introduced through GUI changes and will need rectifying.

[ukmo-huw-lewis]

Sense check test GUI requests vs output:

PLOT_SPATIAL_SURFACE_MODEL_FIELD_CASE_AGGREGATION: by lead time + hour of day no aggregation produced - will need to check logic

SPATIAL_SURFACE_DIFFERENCE_CASE_AGGREGATION: by lead time + hour of day no aggregation produced - will need to check logic

DOMAIN_MEAN_SURFACE_TIME_SERIES_CASE_AGGREGATION: by lead time + hour of day no aggregation produced - will need to check logic

DOMAIN_SURFACE_HISTOGRAM_SERIES_FIELD_CASE_AGGREGATION: by lead time + all cases some aggregation output - looks like it's the "all cases" option that's worked here

Pressure level fields and profiles similarly lacking case aggregation outputs just now. I suspect something not following through from my proposed includes/case_aggregation_options.cylc (noting I wanted to keep that away from the main flow.cylc file if I could, or may be preferable just to apply some of this logic in relevant includes/ file if looks more complex than needed?)

UPDATE: Have now pushed translation of rose-suite.conf T/F statements into flow.cylc and appear to be picked up correctly (see following change update).

[ukmo-huw-lewis]

Sense check test GUI requests vs output 2:

Sample outputs: https://wwwspice/~huw.lewis/CSET/CSET_WAfr_RAL3LFRic/

PLOT_SPATIAL_SURFACE_MODEL_FIELD_CASE_AGGREGATION: by lead time + hour of day --> aggregation produced by lead time and hour of day as requested

PRESSURE_LEVEL SPATIAL --> produced by lead time + hour of day as requested

SPATIAL_SURFACE_DIFFERENCE_CASE_AGGREGATION: by lead time + hour of day --> produced both as requested

DOMAIN_MEAN_SURFACE_TIME_SERIES_CASE_AGGREGATION: by lead time + hour of day --> both produced as requested, though appear under 'Time Series' tab in CSET webpage rather than 'Surface Time Series'

DOMAIN_SURFACE_HISTOGRAM_SERIES_FIELD_CASE_AGGREGATION: by lead time + all cases some aggregation output --> both generated as requested

All appears to have functioned correctly

[ukmo-huw-lewis]

Note opt rose-suite.conf files will need updating to reflect any changes to workflow options etc, but want to ensure we bottom out approach first before making those changes.

[jfrost-mo]

I haven't gone into too much detail on the minutia yet, but I wanted to write down my thoughts on the overall design.

I really like the improvements to the output. The ordering is significantly nicer and the additional categorisation of diagnostics definitely improves their discoverability. Having multiple checkboxes on a single line is a novel way of selecting multiple different kinds of aggregation without making the list too long.

I can't say I'm much of a fan of the translation table in the flow.cylc. I worry we have too much Jinja2 logic already, without adding configuration parsing too. On the other hand this is very simple logic, so probably doesn't hurt testability too much, and does provide a very real benefit. I also can't think of another way to do this that is any nicer.

I'm not sure if we need to keep the toggle for showing the case aggregation options now we have more granular categories. It was implemented as a stopgap measure when the quicklooks section became overwhelming, but this redesign has solved that problem more fundamentally.

This may be slight scope creep, but it would be good to provide help text for the diagnostics, though I do wonder if this would be better in the web documentation instead (or in addition). Also nicer titles could be set.

Finally aligning the cylc task name with the recipe name is a very sensible change, but I wonder if we could go further. If we aligned the name of the include file with the recipe it would pave the way to being able to auto-generate them. We could even use the "Category" information in the recipe to construct the configuration. This is probably something for a subsequent pull request though.

[jfrost-mo]

Happy to approve these changes. The only change that needs to happen before merging is to edit the pull request title to be a concise description of the changes in this PR, so it can be pulled through to use in the changelog.