Skip to content

Added runtime validation for Agent name field #998

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 25 commits into
base: main
Choose a base branch
from

Conversation

DanielHashmi
Copy link
Contributor

@DanielHashmi DanielHashmi commented Jul 3, 2025

Add __post_init__ validation to ensure Agent name is a string

The Agent class previously only used type hints for the name field without
runtime validation, allowing non-string values like integers to be passed.
This caused downstream errors during JSON serialization, tracing, and other
operations that expect the name to be a string.

Changes:

  • Add __post_init__ method to Agent class with isinstance check
  • Raise TypeError with descriptive message for non-string names
  • Validation occurs at instantiation time to fail fast

Fixes issue where Agent(name=1) would succeed but cause errors later
in the execution pipeline.

Fixes #996

…evelopers" appears twice in the classifiers list. Line Number 20 and Line Number 26 Fix: Removed the duplicate classifier entry
…he tracing module where one import uses an absolute path in line number 3 while all others use relative imports
@seratch
Copy link
Member

seratch commented Jul 8, 2025

Related to #996; if we add this, we may want to have validation logic for other arguments too @rm-openai any opinion?

@DanielHashmi
Copy link
Contributor Author

Thanks for the feedback! You're right that this could be part of a broader validation pattern. Looking at the Agent class, several other fields could benefit from similar runtime validation:

  • instructions: Should validate str, callable, or None
  • tools: Should validate list of Tool objects
  • handoffs: Should validate list of Agent/Handoff objects
  1. Should we implement validation incrementally or all at once?
  2. Prefer individual isinstance checks in __post_init__ or a structured validation framework?
  3. Any specific validation patterns you'd like as the SDK standard?

I'm happy if I can expand this PR to include other critical fields or use this as a foundation for incremental validation. What approach works best for the team?

@seratch
Copy link
Member

seratch commented Jul 10, 2025

Should we implement validation incrementally or all at once?

Agent class currently has 16 fields, so adding all of them may take some time. If you're fine with it, your contribution would be appreciated. Otherwise, we can later add validations to the rest of the fields after merging this PR only with name.

Prefer individual isinstance checks in post_init or a structured validation framework?

Simply writing all logic in __post_init__ should be fine for now. Perhaps, most code would be combination of if XXX is not None and if isinstance(V, T).

Any specific validation patterns you'd like as the SDK standard?

Checking if a value is not None and then check the type of the value would be enough. I don't think we should scan all the elements of an array and nested property values (i.e., no need to check internal values of ModelSettings, each element of handoffs etc.). We may want to do similar to other data classes, but in this PR, we can focus on the Agent class.

@seratch
Copy link
Member

seratch commented Jul 10, 2025

I just noticed @rm-openai started refactoring of the class - #1044 ; aligning with this change and/or working on this PR after the refactoring is done would be appreciated 🙏

@DanielHashmi
Copy link
Contributor Author

Ok! will work on this after refactoring 👍

@seratch
Copy link
Member

seratch commented Jul 16, 2025

@DanielHashmi The refactoring is done, and we've been receiving the same feedback from a few users. If you have time and interest to resume this PR work, it'd be appreciated! (Otherwise, I am happy to work on the rest 👋 )

@DanielHashmi
Copy link
Contributor Author

I have added validations and some tests, Can you review it and tell me what should be my next step?

@seratch seratch requested review from rm-openai and Copilot and removed request for Copilot July 16, 2025 22:54
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Adds runtime validation to the Agent class to catch invalid field types at instantiation and corresponding tests to verify those checks.

  • Introduce a __post_init__ method in Agent that raises TypeError for non-string name and other invalid field types.
  • Add unit tests in TestAgentValidation to cover key validation scenarios.
  • Validate fields like tool_use_behavior, hooks, tools, handoffs, and model_settings.

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
tests/test_agent_config.py Add TestAgentValidation to test various runtime field validations
src/agents/agent.py Implement __post_init__ with type checks on multiple Agent fields
Comments suppressed due to low confidence (2)

tests/test_agent_config.py:174

  • Several new validations in post_init (e.g., for mcp_servers, mcp_config, instructions, prompt, input_guardrails, output_guardrails, output_type, reset_tool_choice) lack corresponding tests. Consider adding tests to improve coverage and ensure these validations behave as expected.
class TestAgentValidation:

src/agents/agent.py:224

  • If AgentBase defines its own post_init, overriding it here without calling super().__post_init__() may skip base initialization logic. Consider invoking super().__post_init__() at the start of this method.
    def __post_init__(self):

Comment on lines +225 to +245
from typing import get_origin

if not isinstance(self.name, str):
raise TypeError(f"Agent name must be a string, got {type(self.name).__name__}")

if self.handoff_description is not None and not isinstance(self.handoff_description, str):
raise TypeError(
f"Agent handoff_description must be a string or None, "
f"got {type(self.handoff_description).__name__}"
)

if not isinstance(self.tools, list):
raise TypeError(f"Agent tools must be a list, got {type(self.tools).__name__}")

if not isinstance(self.mcp_servers, list):
raise TypeError(
f"Agent mcp_servers must be a list, got {type(self.mcp_servers).__name__}"
)

if not isinstance(self.mcp_config, dict):
raise TypeError(
Copy link
Preview

Copilot AI Jul 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] The post_init method contains many repetitive type checks; consider refactoring with a helper function or a mapping of field names to expected types to reduce duplication and improve readability.

Suggested change
from typing import get_origin
if not isinstance(self.name, str):
raise TypeError(f"Agent name must be a string, got {type(self.name).__name__}")
if self.handoff_description is not None and not isinstance(self.handoff_description, str):
raise TypeError(
f"Agent handoff_description must be a string or None, "
f"got {type(self.handoff_description).__name__}"
)
if not isinstance(self.tools, list):
raise TypeError(f"Agent tools must be a list, got {type(self.tools).__name__}")
if not isinstance(self.mcp_servers, list):
raise TypeError(
f"Agent mcp_servers must be a list, got {type(self.mcp_servers).__name__}"
)
if not isinstance(self.mcp_config, dict):
raise TypeError(
def validate_fields(field_mapping: dict[str, tuple[type, bool]]) -> None:
"""Helper function to validate fields based on a mapping of field names to types."""
for field_name, (expected_type, allow_none) in field_mapping.items():
value = getattr(self, field_name)
if value is None and allow_none:
continue
if not isinstance(value, expected_type):
raise TypeError(
f"Agent {field_name} must be a {expected_type.__name__}"
f"{' or None' if allow_none else ''}, got {type(value).__name__}"
)
field_mapping = {
"name": (str, False),
"handoff_description": (str, True),
"tools": (list, False),
"mcp_servers": (list, False),
"mcp_config": (dict, False),
}
validate_fields(field_mapping)
raise TypeError(

Copilot uses AI. Check for mistakes.

Copy link
Member

@seratch seratch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great to me
@rm-openai we've been receiving the same feedback about the lack of validation of inti params for Agent almost every day. Can you take a look at this change and include it in the upcoming release?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request feature:core
Projects
None yet
Development

Successfully merging this pull request may close these issues.

No validation on the Agent constructor arguments
2 participants