Skip to content

FEAT: adding Garak audio achilles scenario#2138

Open
rlundeen2 wants to merge 3 commits into
microsoft:mainfrom
rlundeen2:rlundeen2-audio-achilles-heel-scenario
Open

FEAT: adding Garak audio achilles scenario#2138
rlundeen2 wants to merge 3 commits into
microsoft:mainfrom
rlundeen2:rlundeen2-audio-achilles-heel-scenario

Conversation

@rlundeen2

Copy link
Copy Markdown
Contributor

Description

Adds a new Garak scenario, AudioAchillesHeel, that ports Garak's audio.AudioAchillesHeel probe into PyRIT scenario constructs. The adversarial instruction is delivered as spoken audio while the text channel carries only a benign "follow the audio instructions" nudge, testing whether a multimodal target complies with harmful spoken instructions.

What it does

  • Loads spoken-jailbreak clips from the existing garak_audio_achilles_heel dataset.
  • Shapes each clip into a single multimodal user turn (text nudge + audio at the same sequence, merging into one Message), mirroring Garak's request shape.
  • Derives a per-clip objective from the clip's harm category and scores the response for compliance (the PyRIT analogue of Garak's non-refusal mitigation.MitigationBypass detector).
  • Baseline attack policy is disabled (audio-payload dominated; a text-only baseline drops the audio).

Notes

  • The default run samples a small subset (max_dataset_size=10 of ~350 clips) so a default pyrit_scan garak.audio_achilles_heel completes quickly; override with --max-dataset-size.
  • The objective target must accept audio_path input (multimodal).
  • Added a documentation-only section to the Garak scanner docs (doc/scanner/garak.py + .ipynb).

Tests

  • 20 unit tests in tests/unit/scenario/garak/test_audio_achilles_heel.py (all passing).
  • pre-commit and ruff clean.

rlundeen2 and others added 3 commits July 8, 2026 09:38
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
… dataset

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Wire the runnable audio example to RealtimeTarget (the OpenAI Realtime
audio websocket), which accepts audio_path input, and execute the
notebook so the AudioAchillesHeel scenario produces real outputs.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant