v0.4.0

What's Changed

1. New Advanced Attack Techniques: Expanded orchestrators with advanced attack techniques, including PAIR, tree of attacks, and crescendo strategies.
2. New Targets: Crucible target, Prompt Shield Target, Azure OpenAI GPT-4o target
3. New Converters: Added Tense, Emoji, image to text, and Character Space converters.
4. New Scorers: Scale Scorer, Prompt Shield, and True/False Inverter Scorer
5. Automatic Scoring & Memory Labels: Introduced automatic scoring in the PromptSendingOrchestrator. Added support for scoring with user-provided memory labels.
6. Delegation SAS Authentication: Supported delegation SAS authentication for secure interactions with Azure Blob Storage targets.
7. Improved Resiliency: Enhanced the resiliency of targets, converters, and orchestrators with robust error handling mechanisms.
8. Bug Fixes & Performance: Various bug fixes, added support for Python 3.12, speedup unit tests
9. Fetch functionality: Introduced functionality to fetch adversarial datasets, such as SecLists, XStest etc.,
10. Updated Demo Codes: Replaced demo code examples with the GPT-4o target.

Full List of Changes

• FIX: Fixing policheck bug by @rlundeen2 in https://github.com/Azure/PyRIT/pull/261
• release v0.3.0 by @jbolor21 in https://github.com/Azure/PyRIT/pull/265
• DOC: Adding Guidance on Incorporating Research by @rlundeen2 in https://github.com/Azure/PyRIT/pull/268
• FEAT: Adding Tense Converter by @rlundeen2 in https://github.com/Azure/PyRIT/pull/273
• [FEAT] Add Scoring to PromptSendingOrchestrator by @nina-msft in https://github.com/Azure/PyRIT/pull/262
• FIX Fixed mypy Type Failures by @elgertam in https://github.com/Azure/PyRIT/pull/269
• FEAT: Adding Crucible Target by @rlundeen2 in https://github.com/Azure/PyRIT/pull/277
• FIX ValueError with Azure TTS Target in Single Turn Conversation Using PromptSendingOrchestrator by @nina-msft in https://github.com/Azure/PyRIT/pull/278
• FEAT: Converter Tokens by @rlundeen2 in https://github.com/Azure/PyRIT/pull/279
• [FIX] Add flake8-copyright check to pre-commit hooks by @nina-msft in https://github.com/Azure/PyRIT/pull/281
• FIX Exclude Morse Converter from Flake8 Precommit by @nina-msft in https://github.com/Azure/PyRIT/pull/284
• [DRAFT] [FIX] Replace Orchestrator ID with UUID by @nina-msft in https://github.com/Azure/PyRIT/pull/285
• DOC update citation for past tense paper by @romanlutz in https://github.com/Azure/PyRIT/pull/288
• FEAT Add scale scorer by @romanlutz in https://github.com/Azure/PyRIT/pull/274
• FEAT Add Delegation SAS-Based Auth, Update Storage Plugins, and Async Blob Download by @rdheekonda in https://github.com/Azure/PyRIT/pull/286
• FEAT add (back) Gandalf scorer by @romanlutz in https://github.com/Azure/PyRIT/pull/287
• MAINT clean up copyright by @romanlutz in https://github.com/Azure/PyRIT/pull/297
• FEAT: Add Error Handling to AML Chat Target by @nina-msft in https://github.com/Azure/PyRIT/pull/299
• FIX: bug with multi-modal image responses by @rlundeen2 in https://github.com/Azure/PyRIT/pull/301
• MAINT: Improving some LLM Converters by @rlundeen2 in https://github.com/Azure/PyRIT/pull/300
• [FIX][Issue #302] update language version enforcement to fix black-pre-commit installation incompatibility by @shivenchawla in https://github.com/Azure/PyRIT/pull/303
• FEAT return ID in conversation duplication code by @romanlutz in https://github.com/Azure/PyRIT/pull/296
• [FEAT] Implement PAIR by @dlmgary in https://github.com/Azure/PyRIT/pull/255
• FEAT add float scale threshold scorer by @romanlutz in https://github.com/Azure/PyRIT/pull/294
• FEAT: Add GPT4-o chat target by @shivenchawla in https://github.com/Azure/PyRIT/pull/293
• FEAT: Adding Emoji Converter by @rlundeen2 in https://github.com/Azure/PyRIT/pull/306
• DOC: Doc Reorg by @rlundeen2 in https://github.com/Azure/PyRIT/pull/304
• MAINT: Removing asyncio sleep by @rlundeen2 in https://github.com/Azure/PyRIT/pull/309
• MAINT add support for Python 3.12 and fix tests that started breaking by @romanlutz in https://github.com/Azure/PyRIT/pull/305
• FEAT Add print_conversation method to prompt sending orchestrator by @romanlutz in https://github.com/Azure/PyRIT/pull/312
• FEAT Add many-shot jailbreaking feature implementation by @KutalVolkan in https://github.com/Azure/PyRIT/pull/254
• FEAT: Add tree of attacks with pruning by @salmazainana in https://github.com/Azure/PyRIT/pull/210
• FEAT Add Space Converter by @rdheekonda in https://github.com/Azure/PyRIT/pull/316
• FEAT Add Flexible Memory Labels and Scoring to Orchestrators by @rdheekonda in https://github.com/Azure/PyRIT/pull/315
• FEAT: Crescendo Orchestrator by @SafwanA02 in https://github.com/Azure/PyRIT/pull/275
• Feat: Adding multi-turn promptSendingOrchestrator by @rlundeen2 in https://github.com/Azure/PyRIT/pull/317
• DOC Fix README.md link by @romanlutz in https://github.com/Azure/PyRIT/pull/319
• MAINT: Fixing data serializer ability to properly raise errors by @rlundeen2 in https://github.com/Azure/PyRIT/pull/318
• FEAT: Add fetch function for SecLists AI LLM Bias Testing datasets (#267) by @KutalVolkan in https://github.com/Azure/PyRIT/pull/280
• FEAT: Adding true_false inverter scorer by @rlundeen2 in https://github.com/Azure/PyRIT/pull/321
• FIX: fixing check links by @rlundeen2 in https://github.com/Azure/PyRIT/pull/323
• FEAT: Add Exception Handling to Azure TTS Target by @nina-msft in https://github.com/Azure/PyRIT/pull/322
• DOC - replacing gpt4 with gpt4o in example notebooks by @jsong468 in https://github.com/Azure/PyRIT/pull/313
• [MAINT] Changing Examples from stop signs by @jbolor21 in https://github.com/Azure/PyRIT/pull/325
• FEAT Prompt Shield by @ValbuenaVC in https://github.com/Azure/PyRIT/pull/271
• FEAT: add xstest dataset by @KutalVolkan in https://github.com/Azure/PyRIT/pull/320
• [FEAT] Created add_image_text_converter and unit tests by @jsong468 in https://github.com/Azure/PyRIT/pull/328
• DOC: Adding Notebook to document re-sending previous prompts by @rlundeen2 in https://github.com/Azure/PyRIT/pull/332
• MAINT: speeding up crescendo tests by @rlundeen2 in https://github.com/Azure/PyRIT/pull/333
• FIX Move pillow from dev to core dependency by @rdheekonda in https://github.com/Azure/PyRIT/pull/334
• FIX add sample image classifier file by @jbolor21 in https://github.com/Azure/PyRIT/pull/336
• FEAT: Add deterministic flag and custom substitutions to LeetspeakConverter by @KutalVolkan in https://github.com/Azure/PyRIT/pull/329
• MAINT Remove Duplicate Module by @rdheekonda in https://github.com/Azure/PyRIT/pull/337
• MAINT Restructure pyrit.models module and prune by @romanlutz in https://github.com/Azure/PyRIT/pull/338
• [MAINT] Speeding up unit tests by @jbolor21 in https://github.com/Azure/PyRIT/pull/335
• FIX Crescendo backtrack with same orchestrator ID and handling responses with markdown syntax by @romanlutz in https://github.com/Azure/PyRIT/pull/340

New Contributors

• @shivenchawla made their first contribution in https://github.com/Azure/PyRIT/pull/303
• @KutalVolkan made their first contribution in https://github.com/Azure/PyRIT/pull/254
• @salmazainana made their first contribution in https://github.com/Azure/PyRIT/pull/210
• @jsong468 made their first contribution in https://github.com/Azure/PyRIT/pull/313
• @ValbuenaVC made their first contribution in https://github.com/Azure/PyRIT/pull/271

Full Changelog: Azure/PyRIT@v0.3.0...v0.4.0