-
Notifications
You must be signed in to change notification settings - Fork 1
Description
Summary
Parameter descriptions defined in Pydantic Field() objects aren't extracted by the Python SDK or displayed in the eval parameter UI, making it unclear to users what each parameter does.
Impact
Workaround: No workaround - descriptions simply don't appear in the UI
Reproduction
Expected: Descriptions from Field(description="...") should appear below parameter names in the UI
Actual: No description text is shown anywhere in the parameter editing interface
Context
Component: Python SDK + Web App
Version: Current (all versions)
Example code:
class SystemPromptParam(BaseModel):
value: str = Field(
default="You are a helpful assistant.",
description="System prompt for the model" # ← Not shown in UI
)
Eval(
name="my-eval",
parameters={"system_prompt": SystemPromptParam},
...
)Investigation
Python SDK (src/braintrust/parameters.py:149): Has explicit TODO comment - doesn't extract description from Pydantic models:
result[name] = {
"type": "data",
"schema": _pydantic_to_json_schema(schema),
# TODO: Extract default and description from pydantic model
}UI expects a top-level description field on the ExtractedParameter interface.
UI renders the description conditionally if present:
{param.description && (
<div className="mt-1 text-xs text-primary-500">
{param.description}
</div>
)}Root cause: SDK sends type and nested schema, but doesn't extract default or description to top level where the UI expects them.
Theory: Need to extract description from Pydantic Field metadata and add to top-level parameter object, similar to how prompt-type parameters work.
Linear issue: https://linear.app/braintrustdata/issue/BRA-4209/python-sdk-parameter-descriptions-from-pydantic-field-not-displayed-in