fix: camelize JSON-schema required values for OpenAI strict structured outputs

[GuillenSanti]

Problem

Since the Mastra SDK upgrade (ai 5→6, @ai-sdk/azure 2→3, @mastra/core 1.4→1.45), agents using structured outputs (generate_object) started failing in production with:

Ai::Error: Invalid schema for response_format 'response':
In context=('properties','matches','items'), 'required' is required to be supplied
and to be an array including every key in properties. Missing 'transactionId'.

This is firing across many structured-output agents at once — banking transaction categorization, the finance OCR document scanner, the interactive GraphQL path, performance-review summaries, and more (1,000+ events / 300+ users).

Triggered by: factorialco/factorial-agent#1826 ("MASTRA UPGRADE", merged 2026-07-01) — https://github.com/factorialco/factorial-agent/pull/1826 — which bumped ai 5→6 / @ai-sdk/azure 2→3 / @mastra/core 1.4→1.45, enabling OpenAI strict structured outputs. The largest wave (banking + OCR Sidekiq jobs and more) escalated right after its deploy today; the same schema signature also appears via the interactive GraphQL path going back ~6 days. Either way the upgrade did not introduce the bug — it exposed this latent one, and the root cause + fix are identical for all of them.

Affected Sentry issues (same root cause)

Dashboard — all Invalid schema for response_format errors. Every one has the same signature: required must include every key in properties; Missing '<camelCaseField>'.

• Banking — BACKEND-5601 · Missing 'transactionId' · Banking::Jobs::Transactions::Categorize
• OCR invoices — BACKEND-5602 / BACKEND-53HX · Missing 'accountNumber' · Finance::EventHandlers::Invoices::ProcessProcessing — surfaced as BACKEND-55W1 "OCR agent is temporarily unavailable"
• Interactive / GraphQL (highest volume) — BACKEND-4YS6 (781 events / 314 users), BACKEND-4TJF, BACKEND-55KS, BACKEND-5600 · Missing 'accountNumber' · ApiPrivate::GraphqlController#execute
• Others — BACKEND-5604 Missing 'performanceReviewSummary', BACKEND-5605 Missing 'lastMeetingContext'

Root cause (a latent bug, not the upgrade)

Ai::StructToJsonSchema.convert builds the schema from a T::Struct with snake_case keys in both properties and required — internally consistent.

Ai::Clients::Mastra#deep_camelize_keys then runs deep_transform_keys, which camelizes hash KEYS but not array string VALUES:

• properties keys → camelCase (transaction_id → transactionId) ✅
• required values → stay snake_case (transaction_id) ❌ (they are array elements, not keys)

So the schema sent to OpenAI has always been internally inconsistent (properties camelCase, required snake_case).

Why it only breaks now: the pre-upgrade path did not enforce OpenAI strict structured outputs, so the mismatch was tolerated. The upgraded SDK enables strict mode, where OpenAI requires required to list every property key by exact name. The camel/snake mismatch now fails validation. The upgrade did not introduce the bug — it exposed a latent one.

Fix

After deep_transform_keys, recursively align every required array with the camelized property keys (camelize_schema_required!). The walk is recursive so it also handles nested and nilable (anyOf) structs — e.g. the OCR payment_details.account_number case (context=('properties','paymentDetails','anyOf','0'), Missing 'accountNumber'), not just flat top-level schemas.

Verification

• Reproduced deterministically (no OpenAI call) using the real production struct through the exact path (StructToJsonSchema.convert → deep_camelize_keys):
  • before: properties=[transactionId, categoryId], required=[transaction_id, category_id] → strict REJECTS
  • after: required=[transactionId, categoryId] → strict ACCEPTS
• Added two regression specs (flat + nested anyOf). Note the pre-existing "converts struct fields to camelCase" spec never asserted required — which is how this slipped through.
• bundle exec rspec → 85 examples, 0 failures · srb tc → no errors · rubocop → no offenses.

Impact

Fixes structured-output failures across all agents whose struct has multi-word (snake_case) fields — banking categorization, finance OCR, etc. This is a cross-domain production incident; after merge + release, bump the ai gem pin in the monorepo Gemfile.

🤖 Generated with Claude Code