Cue-swap probe dataset for multi-turn red-team scorers

[connerlambden]

Helium Model Worldview includes structured cue-swap pairs (name swap, label swap, topic swap) that could seed PyRIT multi-turn adversarial scenarios.

304 prompts with per-model flip rates on Hugging Face. Safety split shows wide refusal/compliance spread across frontier models.

Dataset: https://huggingface.co/datasets/HeliumTrades/helium-model-worldview-benchmark
Scoring guide: https://heliumtrades.com/benchmarks/

[connerlambden]

Closing — this was opened too broadly during initial outreach and does not fit microsoft/PyRIT's scope (PyRIT maintainers prefer contributed datasets via PR/discussion with concrete scenarios.).

We are focusing contributions on:

• Proper PRs to curated awesome benchmark lists (with a "why this belongs here" line)
• Real integrations: Add Helium Market Resolution and Model Worldview tasks EleutherAI/lm-evaluation-harness#3906, Add Helium Model Worldview cue-swap example promptfoo/promptfoo#9950, Add Helium Market Resolution benchmark open-compass/opencompass#2507

Sorry for the noise. — Helium Trades team