Skip to content

Conversation

@colinbennettbrain
Copy link
Contributor

Summary

  • Fixes remote eval showing "Not connected" even when server is running and returning 200 OK
  • Root cause: Python SDK serialized None as null for missing description field, but frontend Zod schema uses .optional() which rejects null

In py/src/braintrust/parameters.py, the parameters_to_json_schema() function was unconditionally including description and default fields:

result[name] = {
    "type": "prompt",
    "default": schema.get("default"),      # None -> null
    "description": schema.get("description"),  # None -> null
}

The frontend Zod schema in js/dev/types.ts uses:
description: z.string().optional(),  // Accepts undefined, REJECTS null

When null is received, evaluatorDefinitionsSchema.safeParse() fails, causing the "Not connected" status.

Fix

Only include description and default fields when they have actual values.

Test plan

  • Added unit tests for parameters_to_json_schema()
  • Manual testing: Remote eval connects when parameter has no description

When a remote eval parameter doesn't have a `description` field set,
`schema.get("description")` returns `None`, which serializes as `null`
in the JSON response. The frontend Zod schema uses `.optional()` which
accepts `undefined` but rejects `null`, causing silent validation
failures and "Not connected" status.

Fix: Only include `description` and `default` fields in the serialized
output when they have actual values, avoiding `null` serialization.

Fixes Pylon case #10206

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants