v0.3.0

What's Changed

• New and improved scorers! Many new scorers have been added, and scorers can now be swapped out and made generic.
• Many new attack techniques and variations have been introduced. These include skeleton key, most of GPTFuzz, adding text to images, repeated token attack, cipherchat, shorten/expand, tone, CodeChameleon, and more. A total of 13 new converters have been added!
• Framework improvements:
  • Ability to duplicate conversations for reuse (this makes implementation easier for attacks like PAIR/TAP/crescendo).
  • Converters can be added to LLM responses.
  • All framework calls are now async and parallelizable.
  • Error handling and intelligent automatic retries in targets (e.g., for network errors) and converters/scorers (e.g., for JSON deserialization).

Full list of Changes

• FEAT: Refactoring and Standardizing Scores and Scorers by @rlundeen2 in https://github.com/Azure/PyRIT/pull/190
• FIX: Making RESULTS_PATH be simple in pip packages by @rlundeen2 in https://github.com/Azure/PyRIT/pull/191
• FIX: Minor Self-Ask Scorer Improvements by @rlundeen2 in https://github.com/Azure/PyRIT/pull/194
• FEAT: Adding Scores to the Database by @rlundeen2 in https://github.com/Azure/PyRIT/pull/195
• MAINT use context manager in XPIA notebook by @romanlutz in https://github.com/Azure/PyRIT/pull/198
• FEAT: Update score_async to add score to database by @rlundeen2 in https://github.com/Azure/PyRIT/pull/200
• FEAT support duplicating memory when cloning orchestrators by @romanlutz in https://github.com/Azure/PyRIT/pull/177
• MAINT: Likert Scoring Tweaks to Reduce False Positives by @rlundeen2 in https://github.com/Azure/PyRIT/pull/201
• FEAT add CSV support by @romanlutz in https://github.com/Azure/PyRIT/pull/197
• FEAT: Adding Human in the Loop Scorer by @rlundeen2 in https://github.com/Azure/PyRIT/pull/202
• FEAT: Azure content filter scorer by @cseifert1 in https://github.com/Azure/PyRIT/pull/206
• FEAT Adding Image Converter: add text on image by @jbolor21 in https://github.com/Azure/PyRIT/pull/205
• FEAT: Score Prompts Orchestrator by @rlundeen2 in https://github.com/Azure/PyRIT/pull/208
• MAINT: Deprecated send_prompt methods by @mart123p in https://github.com/Azure/PyRIT/pull/204
• FEAT Add image generation example with red teaming orchestrator and unify existing orchestrator definitions by @romanlutz in https://github.com/Azure/PyRIT/pull/189
• FEAT: self ask conversation objective and verifier scorer for crescendo by @cseifert1 in https://github.com/Azure/PyRIT/pull/209
• FEAT: Centralize Exception Handling and Implement in GPTv Target by @rdheekonda in https://github.com/Azure/PyRIT/pull/207
• MAINT Making Prompt Converters Async by @jbolor21 in https://github.com/Azure/PyRIT/pull/211
• Update .env_example Typo "Azure Open AI"→"Azure OpenAI" by @hyoshioka0128 in https://github.com/Azure/PyRIT/pull/214
• MAINT: Small scoring updates by @rlundeen2 in https://github.com/Azure/PyRIT/pull/215
• MAINT: Adding pretty print functionality and small RTO updates by @rlundeen2 in https://github.com/Azure/PyRIT/pull/217
• DOC: Re-organizing documentation by @rlundeen2 in https://github.com/Azure/PyRIT/pull/219
• FEAT: Add Suffix Converter by @NaijingGuo in https://github.com/Azure/PyRIT/pull/212
• MAINT: Updating GPT-V to use new exception guide by @rlundeen2 in https://github.com/Azure/PyRIT/pull/220
• FEAT: Add nesting and prepend/append jailbreaks from papers by @jl8771 in https://github.com/Azure/PyRIT/pull/216
• MAINT Adding Error Handling to OpenAIChatInterface by @jbolor21 in https://github.com/Azure/PyRIT/pull/218
• MAINT Add Exception Handling to DALLE Target by @rdheekonda in https://github.com/Azure/PyRIT/pull/221
• FEAT: Add repeated token attack converter by @jl8771 in https://github.com/Azure/PyRIT/pull/224
• MAINT Resolve Install Issues and Add Multiline Text Wrapping in AddTextImageConverter by @rdheekonda in https://github.com/Azure/PyRIT/pull/230
• MAINT: PromptRequestPiece SHA setting update by @rlundeen2 in https://github.com/Azure/PyRIT/pull/231
• FEAT: Implements Crescendo-style attack based on system prompt. by @dlmgary in https://github.com/Azure/PyRIT/pull/237
• MAINT add notebook version disclaimer by @romanlutz in https://github.com/Azure/PyRIT/pull/234
• FEAT: Adding Converters to Output by @rlundeen2 in https://github.com/Azure/PyRIT/pull/236
• DOC: Reorganizing MemoryDocs by @rlundeen2 in https://github.com/Azure/PyRIT/pull/239
• Added complex code jailbreak template by @petebryan in https://github.com/Azure/PyRIT/pull/238
• FEAT: Add prompt converters for atbash, caesar, morse and cipherchat from paper by @jl8771 in https://github.com/Azure/PyRIT/pull/223
• MAINT add test instructions to release guide by @romanlutz in https://github.com/Azure/PyRIT/pull/232
• FIX: Fixing doc links by @rlundeen2 in https://github.com/Azure/PyRIT/pull/245
• FEAT: Adding Master Key Jailbreak by @SafwanA02 in https://github.com/Azure/PyRIT/pull/248
• MAINT Adding Error Handling Code for converters by @jbolor21 in https://github.com/Azure/PyRIT/pull/247
• FIX: Fixing score conversation history by @rlundeen2 in https://github.com/Azure/PyRIT/pull/251
• FEAT: Add shorten/expand converters by @jl8771 in https://github.com/Azure/PyRIT/pull/246
• FEAT: Add CodeChameleon converter by @jl8771 in https://github.com/Azure/PyRIT/pull/240
• FEAT: Adding Noise and Tone Converters by @rlundeen2 in https://github.com/Azure/PyRIT/pull/252
• FEAT: Add persuasion converter with 5 persuasion techniques by @jl8771 in https://github.com/Azure/PyRIT/pull/253
• FEAT Implementation of SQL Server connectivity by @elgertam in https://github.com/Azure/PyRIT/pull/227
• MAINT Error Handling for Scorers by @jbolor21 in https://github.com/Azure/PyRIT/pull/256
• FIX: Skeleton Key Orchestrator by @SafwanA02 in https://github.com/Azure/PyRIT/pull/260
• MAINT upgrading AOAI version by @jbolor21 in https://github.com/Azure/PyRIT/pull/264

New Contributors

• @mart123p made their first contribution in https://github.com/Azure/PyRIT/pull/204
• @hyoshioka0128 made their first contribution in https://github.com/Azure/PyRIT/pull/214
• @jl8771 made their first contribution in https://github.com/Azure/PyRIT/pull/216
• @SafwanA02 made their first contribution in https://github.com/Azure/PyRIT/pull/248
• @elgertam made their first contribution in https://github.com/Azure/PyRIT/pull/227

Full Changelog: Azure/PyRIT@v0.2.1...v0.3.0