ci: show all OS test results in merged Allure report

[maxandersen]

TL;DR: allure merge in main only show windows because environemnt.properties collide when merging. this moves to utilize allure's default support for handling multiple test runs by passing multiple dirs.

p.s. would NEVER or at least in any sane time have figured this out without a research agent :)

Why

The merged Allure report on main builds currently shows only Windows metadata at the top level (Environment widget). Mac/Linux test data is actually present (UUID-named result JSONs do not collide), but everything humans see on the overview page comes from a single environment.properties file — and during artifact merge the Windows copy overwrote Ubuntu's and macOS's.

Root cause

download-artifact with merge-multiple: true flattens every per-OS artifact into one directory. Fixed-name files (environment.properties, executor.json, etc.) collide; alphabetically Windows is last and wins. This also destroyed the per-launch boundary Allure uses to aggregate metadata across multiple runs.

Fix

Allure already supports this scenario natively: allure generate takes multiple results directories and treats each as a separate "launch". The Environment widget aggregates keys across launches (e.g. OS → Linux amd64, Mac OS X aarch64, Windows 10 amd64) and the Executors widget shows one row per launch.

• build.gradle — allureRawResultElements is now configurable via -PallureMergeDirs=dir1,dir2,.... Default (single build/allure-results) unchanged for per-job runs.
• build.gradle — generateAllureEnvironment also writes an executor.json per job (name, type: github, buildName, buildUrl, buildOrder) so the Executors widget links back to each matrix job.
• .github/workflows/step-ci-build.yml — merge-test-reports now downloads each per-OS artifact into its own subdir (no merge-multiple) and passes them to allureReport via -PallureMergeDirs.

Net diff: +26 / -4 across two files.

What you'll see on the merged report after this PR

• Environment widget: OS aggregates across launches; shared keys (GitCommit, ProjectVersion, CI) collapse to one entry.
• Executors widget: one row per matrix cell, each linking back to its GitHub Actions job. Because jbang's matrix has many cells (Ubuntu × Java 8/11/17/21/25 for IT, plus unit + native-image), this widget will be longish but informative — labels include OS + Java version for disambiguation.
• All per-test data preserved.

Verification

Locally simulated 3-OS merge: confirmed the embedded environment.json aggregates OS values and executors.json contains three executor entries. CI run on this PR will be the real check — open the Merged Allure Report link from the test-results-summary step summary.

How to verify on this PR

1. Wait for the ci-build workflow to finish.
2. Open the run's summary → "Merged Allure Report" link.
3. Overview page should show:
   • Environment widget with multi-value entries for OS/Java/RUNNER_OS.
   • Executors widget with one row per matrix cell, links back to each job.

Background research

Allure 2's intended pattern for cross-host reports is "pass multiple dirs to allure generate". See Allure1EnvironmentPlugin (groups env keys across launches), ExecutorPlugin (one entry per launch dir), and AllureReport.kt in the gradle plugin (passes every rawResults entry as a separate CLI arg).

[coderabbitai]

Important

Review skipped

Auto reviews are limited based on label configuration.

🏷️ Required labels (at least one) (1)

• ai-review

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: d1c09f3a-39e5-4123-96be-b8bf7a5d4c70

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}