Skip to content

Conversation

rcannood
Copy link
Member

@rcannood rcannood commented Aug 14, 2025

Describe your changes

This PR adds a results processing script. It has been generalised a bit and has been moved into the common repo. To try it out:

$ common/scripts/render_results_report --help
Usage: render_results_report [OPTIONS] MODE

A script to render benchmark reports from results data.
Automatically detects AWS CLI availability and falls back to Docker if needed.

MODES:
    local <path>        Process results from a local directory
    s3 <s3_path>        Process results from a specific S3 path
    latest              Process the latest results from S3
    select              Show selection window of different runs

OPTIONS:
    -h, --help          Show this help message
    -o, --output DIR    Output directory (default: output/report)
    -t, --task NAME     Task name (auto-detected from _viash.yaml if not provided)
    -v, --verbose       Enable verbose output
    --dry-run           Show commands that would be executed without running them

EXAMPLES:
    render_results_report latest
    render_results_report s3 s3://openproblems-data/resources/task_cyto_batch_integration/results/run_20241010
    render_results_report local /path/to/local/results
    render_results_report select
    render_results_report -o custom_output latest

If aws cli isn't installed, the aws cli calls are run via Docker.

Checklist before requesting a review

  • I have performed a self-review of my code

  • Check the correct box. Does this PR contain:

    • Breaking changes
    • New functionality
    • Major changes
    • Minor changes
    • Bug fixes
  • Proposed changes are described in the CHANGELOG.md

  • CI Tests succeed and look good!

@rcannood rcannood marked this pull request as ready for review September 25, 2025 08:10
@rcannood rcannood requested a review from ghar1821 September 25, 2025 08:11
@rcannood
Copy link
Member Author

rcannood commented Sep 25, 2025

This should work, but only assuming:

  • that the user has aws cli installed
  • that the last benchmark run has finished successfully

If either of these assumptions are not true, you could define the run id manually.

Edit: Nvm! I fixed this ;)

@rcannood rcannood changed the title [WIP] Add results processing script Add results processing script Sep 25, 2025
@LuLeom
Copy link
Contributor

LuLeom commented Sep 25, 2025

@ghar1821 does it work for you?

@rcannood It is not a problem with the script itself, rather a problem with fetching the submodule updates.

@LuLeom
Copy link
Contributor

LuLeom commented Oct 8, 2025

@rcannood tried to git pull from add_results_processing branch.
A script render_report.sh seems to be there, but it is saved in the scripts/ folder and it tries to run a render_results_report which is not there.

From the repo's root directory:

image

The reason why it fails it's because nothing is fetched in common/scripts/
image

@ghar1821
Copy link
Contributor

ghar1821 commented Oct 15, 2025

@rcannood, the select option can't fetch any runs.

bash common/scripts/render_results_report select  
Fetching available runs...
common/scripts/render_results_report: line 179: readarray: command not found

1) 
2) Quit
Select a run (1-2): 2
Selection cancelled.
Operation cancelled by user.

Does this only work for a full run? i.e., with all methods and metrics? I just pull results from one partial run and i couldn't get any tables to render.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants