Skip to content

rananaquin11-web/Adversarial-Prompt-Design

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

2 Commits
 
 

Repository files navigation

Adversarial Prompt Design

This repository contains examples of adversarial prompts designed to test AI systems for instruction-following, reasoning, consistency, and response quality.

The purpose of these prompts is to identify weak points in model behavior by introducing layered constraints, ambiguity, edge cases, and format-specific requirements.

Objectives

These prompt examples are designed to test whether an AI can:

  • Follow multi-step instructions exactly
  • Maintain consistency across constraints
  • Handle edge cases without hallucinating
  • Distinguish relevant from irrelevant information
  • Produce responses in the required format

Sample Adversarial Prompts

Example 1: Conflicting Constraints

Prompt:
Answer this question in exactly six words:
What are the benefits of exercise?
Do not use any word containing the letter "e".

Why this is adversarial:
The prompt combines a content requirement, a word-count requirement, and a character-level restriction. This tests whether the model can follow multiple constraints at once.

What this tests:

  • Instruction-following
  • Constraint handling
  • Output precision

Example 2: Format Trap

Prompt:
List four fruits in alphabetical order.
Return the answer as a numbered list.
Do not use the letters A, E, or O anywhere in your response.

Why this is adversarial:
The prompt requires the model to satisfy both content and formatting rules while also obeying a letter restriction that may make the task impossible or expose careless generation.

What this tests:

  • Formatting compliance
  • Constraint conflict detection
  • Attention to detail

Example 3: Irrelevant Information Filter

Prompt:
A red car and a blue bicycle are parked outside.
The sky is cloudy.
What color is the bicycle?
Answer using one word only.

Why this is adversarial:
This prompt checks whether the model can ignore extra details and extract only the relevant information.

What this tests:

  • Relevance filtering
  • Reading accuracy
  • Concise response generation

Example 4: Instruction Reversal

Prompt:
Write the word "triangle" backwards.
Do not explain your answer.
Return the answer in uppercase letters only.

Why this is adversarial:
This tests whether the model can transform text correctly while still following formatting requirements.

What this tests:

  • Text transformation
  • Instruction-following
  • Output formatting

Example 5: Ambiguity Detection

Prompt:
Summarize the article in two sentences.
Do not add information that is not in the article.
[No article provided.]

Why this is adversarial:
This prompt tests whether the model notices missing context instead of inventing content.

What this tests:

  • Hallucination resistance
  • Missing-context detection
  • Response safety

Evaluation Focus

When reviewing responses to adversarial prompts, common evaluation areas include:

  • Accuracy
  • Instruction-following
  • Consistency
  • Hallucination avoidance
  • Formatting compliance
  • Reasoning quality

Purpose

These examples reflect real-world AI evaluation work involving prompt testing, quality review, and identifying model weaknesses under difficult or conflicting instructions.

About

Examples of structured prompts designed to test AI reasoning, instruction-following, and response quality

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

No releases published

Packages

 
 
 

Contributors