Output format/framing drift after prompt/model changes — have you seen this?

[minogin]

I’m trying to sanity-check a problem I’ve seen in AI workflows.

Sometimes an intermediate LLM step keeps roughly the same semantic intent, but changes the output framing after a prompt/model/code change.

Example:

Before:

{"riskLevel":"MEDIUM","reason":"Ownership is not disclosed."}

After:

"{\"riskLevel\":\"LOW\",\"reason\":\"Supplier is active and clear.\"}"

Or:

REQUEST_MORE_INFORMATION

becomes:

The supplier requires more information before approval.

Both are understandable, but they may break parsers, routing logic, or downstream workflow assumptions.

Questions:

Have you seen this kind of output format/framing drift in real LLM workflows?

If yes, how do you usually detect it — logs, evals, tests, retries, schema validation, something else?

Was it actually painful, or just a minor annoyance?