Pre-1.0 Contract API hardening — deferred items from adversarial review

[roryford]

Tracking issue for Contract-API findings from an adversarial review (Jun 2026) that were not fixed in the first hardening pass. The three highest-severity, freeze-irreversible items (JSONSchemaValue integer fidelity, ToolResult.ErrorKind escape hatch, BackendCapabilities tolerant decode) are handled in the feat!: harden Contract wire types before 1.0 freeze PR. The items below survived red-team scrutiny but were judged lower-severity or design-trade decisions.

Each item notes whether it is freeze-blocking (must land before the 1.0 vocabulary freeze because it can't be fixed non-breakingly after) or deferrable (can ship post-1.0).

Freeze-relevant (decide before 1.0)

• EmbeddingBackend is thinly specified vs InferenceBackend (fix: stable reverse-scroll when prepending older messages #15). No capabilities, no thread-safety doc, dimensions undefined before loadModel, and embed(_:) has no stated guarantee that output vector count matches input count. EmbeddingError.dimensionMismatch partially mitigates. For a third-party-implementable protocol this contract is under-defined. Freeze-blocking if we want to add capabilities to the protocol later.
• Document the throw-vs-silent-ignore rule in GenerationConfig (Replace NSLock + DispatchQueue.main.async with actor and @MainActor Task #1). The rule is principled (capability-gated guarantees like grammar throw unsupportedGrammar; advisory hints like seed/minP/jsonMode degrade silently) but is only discoverable by reading individual field docs. State the rule once at the type level. Deferrable (doc-only).
• Vendor-specific sampler knobs on the shared GenerationConfig (Add XCTSkipIf hardware gates to hardware-dependent tests #2). llamaDRY/llamaXTC/llamaMirostatV2 (llama.cpp-only), yieldEveryNTokens (MLX-only), streamPrefillProgress (OpenAI-compat-only) are frozen into the "shared across all backends" type. Consider a backendSpecific side-channel before freeze. Freeze-blocking — can't relocate frozen public fields after 1.0.

Deferrable (post-1.0 safe)

• GenerationConfig Codable round-trip is silently lossy (Add XCTMeasure performance baselines for hot paths #3). jsonMode/thinkingMarkers/structuredOutput are intentionally runtime-only and dropped on decode (tested), but a Codable type that drops fields on round-trip is a footgun for new callers. Add a prominent type-level doc note, or split a PersistedGenerationConfig.
• GenerationStream idle-timeout throws CloudBackendError.timeout (chore(main): release 1.0.0 #9) from a wrapper usable by local backends too — leaky name. Consider a backend-neutral timeout error. Opt-in (idleTimeout defaults nil), so low urgency.
• BackendCapabilities has two public names for one capability (chore: pin release-as 0.1.1 to bootstrap Release Please versioning #11): streamsToolCallArguments + the computed alias streamsToolCallArgumentDeltas, both frozen. Consider deprecating one.
• Message (wire enum) can't represent tool/image turns (chore(main): release 0.1.1 #12). Intentional thin one-shot wire shape; richer turns go through the value/@Model types. Documenting it as a known scope boundary may suffice; a multimodal one-shot path would be a feature.

Reviewed and dismissed (no action — recorded for traceability)

• Add demo screenshot or animated GIF to README #5 (two error channels) — wrong: the contract pins the sync channel and BackendContractChecks enforces it.
• ci: add Release Please, PR lint, perf baselines, and CLAUDE.md #6/chore(main): release 1.0.0 #7/chore: bootstrap Release Please manifest at v0.1.0 #8/chore: add release-please-config.json and update CLAUDE.md worker instructions #16 — mitigated by existing contract tests / correct-as-designed (sync stopGeneration is tested; secureWipe no-op is correct for backends with no KV residue; the status-getter TOCTOU is defused by the single-generate queue + MainActor; the maxToolIterations triple-clamp is correct defense-in-depth).
• chore(main): release 1.0.0 #10 "positional init trap" — wrong: the memberwise init is fully labeled.

Source: adversarial Contract review + red-team verification pass, Jun 2026.

[roryford]

Adversarial review (Jun 2026) + partial close via #1887

A three-worker adversarial pass re-examined the three "freeze-relevant" items (#2 vendor knobs, #15 EmbeddingBackend.capabilities, #1 throw-vs-silent doc) and refuted the freeze-blocking framing for all three.

Decisive finding: ManifoldKit ships source stability only — there is no -enable-library-evolution / BUILD_LIBRARY_FOR_DISTRIBUTION anywhere. Consumers rebuild from source, so field layout / ABI irreversibility arguments don't apply. The only truly irreversible post-1.0 operation is removing an existing public symbol — and none of these items require that:

• Replace NSLock + DispatchQueue.main.async with actor and @MainActor Task #1 — pure doc; partially stated inline already (grammar MUST-throw). Shippable in any patch.
• fix: stable reverse-scroll when prepending older messages #15 — capabilities can be added post-1.0 via a protocol-extension default with zero conformer breakage (same pattern already used for InferenceBackend.manifest/secureWipe).
• Add XCTSkipIf hardware gates to hardware-dependent tests #2 — a backendSpecific container is additive anytime; the existing fields never need removal (deprecate-in-place is a permanent acceptable end state).

Done in #1887 (non-breaking, additive)

• Replace NSLock + DispatchQueue.main.async with actor and @MainActor Task #1: type-level throw-vs-silent-ignore doc block on GenerationConfig.
• fix: stable reverse-scroll when prepending older messages #15: new EmbeddingCapabilities value type + capabilities requirement (protocol-extension default) + tightened dimensions/embed contract docs.
• Add XCTSkipIf hardware gates to hardware-dependent tests #2: docs-only — tech-debt comment by the vendor knobs; fields left in place.

Deliberately NOT done — and why

Do not add a backendSpecific side-channel. Design red-team verdict: [String: any Codable] doesn't compile in Swift 6; [String: JSONSchemaValue] downgrades a compiler-checked schema to a stringly-typed one and still freezes the key convention forever; adding it without removing the typed fields creates a cross-repo two-sources-of-truth precedence trap. It launders the pollution rather than removing it.

"Relocate + remove the vendor fields now" was rejected as ~3× bigger than it looks: it's 8 public removals (the 3 llama knobs are mirrored by GenerationParameter enum cases in ManifoldHardware), with real read-sites in the companion repos (manifold-llama LlamaGenerationDriver.swift:78/89/100, manifold-mlx MLXGenerationDriver.swift:338) that MK's CI cannot build or gate, a hard version deadlock (companions pinned <0.51.0, can't reach a replacement API until MK publishes it), silent Codable data-loss on old persisted configs, and 8 permanent un-commentable api-breakage-allowlist entries.

Genuine follow-up (deferred, post-1.0, cross-repo feat!:)

The real architectural smell behind #2 is vendor pollution in the Contract kernel — ManifoldContract carries Llama*SamplerOptions. The right fix is relocating those structs into the manifold-llama companion (backend-owned config), not a stringly-typed map. That is a coordinated multi-repo breaking change and should be scheduled into a major, not rushed for the freeze.

Status: #1/#15/#2 reclassified as deferrable; doc/additive portions handled in #1887. Remaining work = the optional kernel-purity relocation above.