Skip to content

[P1.39] LLM provider errors surface as raw JSON dicts on the first-run path #1110

Description

@frankbria

Found by the #614 cold-start walkthrough on a clean container, following only the README. Priority P1.39 — severity high, category ux. Blocks the beta conversion path.

Problem

The two most likely first-run failures — a bad API key and a dead model ID — both surface as a raw Python-repr of the provider's JSON error. This is the first thing a new user sees, at the first AI-backed command in the README quickstart.

Bad key:

$ cf prd generate
Using template: Standard PRD
Error: Error code: 401 - {'type': 'error', 'error': {'type':
'authentication_error', 'message': 'API key is invalid.'}, 'request_id': None}

Dead model:

$ cf prd generate
Using template: Standard PRD
Error: Error code: 404 - {'type': 'error', 'error': {'type': 'not_found_error',
'message': 'model: claude-3-5-haiku-20241022'}, 'request_id':
'req_011CdriPN8FW32UUNgw1FSFM'}

Neither tells the user what to do. The 401 does not name ANTHROPIC_API_KEY, does not say which provider was resolved, and does not point at cf env check. The 404 exposes an internal model constant the user never chose and cannot act on.

The message is produced by the catch-all except Exception as e: console.print(f"[red]Error:[/red] {e}") at the bottom of prd_generate (codeframe/cli/app.py:1695-1697), which stringifies whatever the SDK raised.

Why this matters more than a normal error-message nit

This is the conversion path. A user who installs from the README and hits an unreadable dict on their first command has no reason to believe the rest works. cf prd generate is the entry point to the THINK pillar, which is the product's differentiator.

Acceptance criteria

  • Provider auth failures (401) produce an actionable message naming the resolved provider, the env var that was read, and a next step (cf env check)
  • Provider model-not-found (404) produces a message naming the configured model and the CODEFRAME_*_MODEL override that controls it — not a raw dict
  • Rate limit (429) and overloaded (529) get the same treatment
  • The raw provider payload is still available under --verbose / in the event log, not lost
  • Applies to every LLM-backed command, not just cf prd generate (tasks generate, work start --execute, prd stress-test)
  • A test asserts the mapped message for at least the 401 and 404 cases

Evidence

  • codeframe/cli/app.py:1695-1697 — catch-all that stringifies the SDK exception
  • Cold-start transcripts: scripts/quickstart-cleanroom/artifacts-pypi-0.9.1/transcript.txt

Metadata

Metadata

Assignees

No one assigned

    Labels

    type:bugSomething is broken and needs fixingux

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions