From 37c069d230e0456eb9f6dda309609ad4ea528786 Mon Sep 17 00:00:00 2001 From: Yi-Ting Chiu Date: Fri, 7 Aug 2026 16:43:52 +0000 Subject: [PATCH 1/5] =?UTF-8?q?docs:=20pin=20the=20trust=20model=20?= =?UTF-8?q?=E2=80=94=20plugins=20are=20trusted=20code,=20the=20security=20?= =?UTF-8?q?layer=20defends=20against=20accidents?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- AGENTS.md | 1 + 1 file changed, 1 insertion(+) diff --git a/AGENTS.md b/AGENTS.md index 64ab535a..cdd21ced 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -24,6 +24,7 @@ Standard ASR is a **Python library that defines and enforces a universal interfa - **Explicit > implicit.** Silent wrong results are the cardinal sin. When in doubt, fail loudly or emit a structured diagnostic — never silently degrade. When DX convenience and explicitness conflict, **correctness wins** (a loud error the developer can fix beats a silent wrong transcript). - **Standard-library rigor.** This is infrastructure others build on for 10 years. Types complete, boundaries sharp, error paths explicit, no implicit behavior. - **Security by default.** Credentials use `SecretStr`. URLs validated (HTTPS, no SSRF). Unsafe options require explicit opt-in. +- **Trust model: plugins are trusted code. The security layer defends against ACCIDENTS, not adversaries.** An installed engine plugin runs arbitrary in-process code — it can already read `os.environ` and open sockets, so no in-library boundary can contain a malicious plugin, and we do not build one. What we DO defend against is the honest mistake: a mis-pasted credential echoed back by pydantic's `input_value=...`, a secret riding inside an exception message into a log or a wire response. The whole defense is three cheap rules — scrub validation-error echoes, wrap credentials in `SecretStr`, never format a raw exception chain into operator logs or client responses. **Hard budget:** no proofs about third-party code, no introspection of pydantic/CPython internals, no machinery against hostile metaclasses / forged markings / subverted C slots — those attackers don't need our log path. A defense that needs its own review rounds to stay correct is a defect source, not a defense. Reviewers: do not file findings that assume an adversarial plugin author; they are out of scope by this definition. ## Rules From efb6963d8a4b4ca79728c4cf6fe2a706c03580cd Mon Sep 17 00:00:00 2001 From: Yi-Ting Chiu Date: Fri, 7 Aug 2026 16:44:08 +0000 Subject: [PATCH 2/5] feat: design-review sweep of contract, runtime, and toolchain One wave across the three layers -- they interlock (the WireJsonValue diagnostics domain, IC.6 declaration enforcement, and the redaction surface tie their type and behavior contracts together), so the layers land as one buildable, fully-tested change. contract: - Wire-visible extras are closed to JSON values with string keys at every depth; capability nodes reject separator-bearing keys so the dot-path query grammar stays a bijection. - Engine DECLARATION defects raise EngineContractError; strict-mode candidate-language rejections raise UnsupportedFeatureError. - TranscriptionResult.metadata removed; Segment.start/end nullable, with renderers that never silently drop text. runtime: - config: the input surface is closed (extra="forbid") at every depth by reading the core schema's effective policy; authored serialization hooks, undeclared value shapes, SerializeAsAny and exclude=True are refused at class definition; secret carriers pair with the secret marker; env values decode by the schema-derived codec (Json[T] terminal-raw); from_env explicit-wins is alias-aware. - redaction: the accident-model scrubber (trust model in AGENTS.md) -- validation errors rebuilt from type/loc/msg, loc masked by field-name shape, content-based echo detection, bounded chain summary, log_exception_safely. - streaming: supersede places replacements on a reading-order ledger; result() is the delivered stream's reduction; the sync session submits to its owned loop; the sync-call boundary is enforced from the EngineBase author hooks. - protocol_boundary (new): canonicalized type names for protocol and compliance error surfaces. - discovery: shadowed engine ids fail loud; a broken plugin is an engine fault. toolchain: - server: engine faults map to scrubbed 503/500 and never blame the caller; the metadata endpoints wrap the whole operation in one fault boundary starting at the model key; the wire projection keeps the framework's compact non-ASCII encoding; WS gets a closed config handshake, diagnostics delta frames, and error-event extra scrubbing; operator logs go through log_exception_safely. - cli: exit codes classify fault at the seam; every error line reports through the safe boundary; --strict renamed --strict-discovery. - doctor: satisfiability is exact (packaging 26.1 oracle). - compliance: one model's fault cannot deny other models their verdicts; the batch-only refusal is verified behaviorally; ConfigurationRequiredError narrows the credential skip; the gating probe pins TranscriptionSession. --- src/standard_asr/__init__.py | 11 +- src/standard_asr/audio/conversion.py | 37 +- src/standard_asr/audio/format.py | 3 +- src/standard_asr/audio/loader.py | 14 - src/standard_asr/compliance.py | 1461 +++++++++-- src/standard_asr/contract/capabilities.py | 228 +- src/standard_asr/contract/exceptions.py | 164 +- src/standard_asr/contract/identifiers.py | 36 +- src/standard_asr/contract/language.py | 75 +- src/standard_asr/contract/params.py | 26 +- src/standard_asr/contract/properties.py | 12 +- src/standard_asr/contract/results.py | 346 ++- src/standard_asr/engine.py | 14 +- src/standard_asr/plugins/discovery.py | 46 +- src/standard_asr/renderers.py | 437 +++- src/standard_asr/runtime/config.py | 1607 ++++++++++-- src/standard_asr/runtime/gating.py | 31 +- src/standard_asr/runtime/interface.py | 425 +++- src/standard_asr/runtime/protocol_boundary.py | 439 ++++ src/standard_asr/runtime/redaction.py | 296 ++- src/standard_asr/runtime/streaming.py | 948 ++++++- src/standard_asr/toolchain/cli.py | 873 +++++-- src/standard_asr/toolchain/doctor.py | 612 ++++- src/standard_asr/toolchain/server.py | 886 +++++-- tests/test_asr_interface.py | 372 ++- tests/test_audio_conversion.py | 16 + tests/test_capabilities.py | 202 ++ tests/test_cli.py | 2130 +++++++++++++++- tests/test_compliance.py | 2266 +++++++++++++++-- tests/test_config.py | 2037 ++++++++++++++- tests/test_discovery.py | 90 +- tests/test_doctor.py | 1073 +++++++- tests/test_error_redaction.py | 481 +++- tests/test_exceptions.py | 26 + tests/test_language.py | 101 +- tests/test_package_exports.py | 61 +- tests/test_param_gating.py | 25 +- tests/test_protocol_boundary.py | 532 ++++ tests/test_results.py | 818 +++++- tests/test_server.py | 2060 +++++++++++++-- tests/test_streaming.py | 1114 +++++++- 41 files changed, 20494 insertions(+), 1937 deletions(-) create mode 100644 src/standard_asr/runtime/protocol_boundary.py create mode 100644 tests/test_protocol_boundary.py diff --git a/src/standard_asr/__init__.py b/src/standard_asr/__init__.py index 52aba0a6..31a6ec3c 100644 --- a/src/standard_asr/__init__.py +++ b/src/standard_asr/__init__.py @@ -41,7 +41,9 @@ from standard_asr.contract.exceptions import ( AudioProcessingError, ConfigError, + ConfigurationRequiredError, DiscoveryError, + EngineContractError, EntrypointValidationError, FactoryLoadError, FFmpegNotFoundError, @@ -52,6 +54,7 @@ StandardASRError, StreamClosedError, StructuredError, + SubtitleRenderingError, TranscriptionError, UnsupportedFeatureError, ) @@ -62,6 +65,7 @@ WordTimestampGranularity, ) from standard_asr.contract.results import ( + DIAG_SEGMENT_TIMESTAMPS_UNAVAILABLE, ChannelResult, Diagnostic, Segment, @@ -69,7 +73,7 @@ Word, ) from standard_asr.plugins.discovery import ModelRegistry, ModelSpec, discover_models -from standard_asr.renderers import to_srt, to_vtt +from standard_asr.renderers import UnrenderablePolicy, to_srt, to_vtt from standard_asr.runtime.interface import StandardASR from standard_asr.runtime.streaming import ( StreamDeadlines, @@ -91,10 +95,13 @@ "AudioUrl", "ChannelResult", "ConfigError", + "ConfigurationRequiredError", + "DIAG_SEGMENT_TIMESTAMPS_UNAVAILABLE", "DIARIZE", "Diagnostic", "DiarizationRequest", "DiscoveryError", + "EngineContractError", "EntrypointValidationError", "FFmpegNotFoundError", "FFprobeNotFoundError", @@ -102,6 +109,7 @@ "IncompatibleAudioInputError", "InvalidProviderParamError", "InvalidSessionUseError", + "UnrenderablePolicy", "ModelRegistry", "ModelSpec", "RuntimeParams", @@ -111,6 +119,7 @@ "StreamClosedError", "StreamDeadlines", "StructuredError", + "SubtitleRenderingError", "SyncSession", "TranscriptionError", "TranscriptionEvent", diff --git a/src/standard_asr/audio/conversion.py b/src/standard_asr/audio/conversion.py index a5161508..f50c1f23 100644 --- a/src/standard_asr/audio/conversion.py +++ b/src/standard_asr/audio/conversion.py @@ -49,6 +49,17 @@ #: Canonical fallback sample rate when a bare array omits its rate. ASSUMED_SAMPLE_RATE = 16000 +#: Diagnostic codes the conversion pipeline emits. The spec names these codes +#: normatively (AI R3/R4/R6/R8), so -- like the ``DIAG_*`` constants in +#: :mod:`standard_asr.runtime.gating` and :mod:`standard_asr.contract.language` -- +#: each is a wire-visible contract with a single source of truth here; several +#: are emitted from more than one site in this module, where a repeated literal +#: could silently drift. +DIAG_AUDIO_CONVERSION = "audio_conversion" +DIAG_NON_FINITE_AUDIO = "non_finite_audio" +DIAG_RESAMPLED_WITH = "resampled_with" +DIAG_ASSUMED_SAMPLE_RATE = "assumed_sample_rate" + def _empty_diagnostics() -> list[Diagnostic]: """Return an empty diagnostics list (typed factory for dataclass default). @@ -359,7 +370,7 @@ def _prepare_encoded( diags.append( Diagnostic( level="warning", - code="audio_conversion", + code=DIAG_AUDIO_CONVERSION, message="Encoded array to WAV/16-bit PCM (lossy float->int16).", param="audio", provided="array", @@ -370,7 +381,7 @@ def _prepare_encoded( diags.append( Diagnostic( level="warning", - code="audio_conversion", + code=DIAG_AUDIO_CONVERSION, message="Downmixed multi-channel audio to mono for encoding.", param="audio", ) @@ -384,7 +395,7 @@ def _prepare_encoded( diags.append( Diagnostic( level="warning", - code="non_finite_audio", + code=DIAG_NON_FINITE_AUDIO, message=( f"Sanitized {result.sanitized_non_finite} non-finite " "sample(s) (NaN/Inf) to 0/+-1 during WAV encoding." @@ -453,7 +464,7 @@ def _diagnose_non_finite(array: NDArray[np.float32], diags: list[Diagnostic]) -> diags.append( Diagnostic( level="warning", - code="non_finite_audio", + code=DIAG_NON_FINITE_AUDIO, message=( f"Array delivery contains {bad} non-finite sample(s) (NaN/Inf); " "forwarded unchanged." @@ -611,7 +622,7 @@ def _prepare_array( diags.append( Diagnostic( level="info", - code="audio_conversion", + code=DIAG_AUDIO_CONVERSION, message=f"Decoded encoded audio to a waveform array at {native_sr} Hz.", param="audio", effective="array", @@ -692,7 +703,7 @@ def _apply_sample_rate( diags.append( Diagnostic( level="warning", - code="resampled_with", + code=DIAG_RESAMPLED_WITH, message=( f"Resampled {sample_rate} Hz -> {target} Hz with the built-in " "fallback resampler. Install standard-asr[audio] for a " @@ -707,7 +718,7 @@ def _apply_sample_rate( diags.append( Diagnostic( level="info", - code="resampled_with", + code=DIAG_RESAMPLED_WITH, message=f"Resampled {sample_rate} Hz -> {target} Hz (scipy resample_poly).", param="audio", # The rate transition lives in ``provided`` and the structured @@ -730,11 +741,19 @@ def _assumed_sample_rate_diag() -> Diagnostic: """ return Diagnostic( level="warning", - code="assumed_sample_rate", + code=DIAG_ASSUMED_SAMPLE_RATE, message=f"No sample rate provided; assumed {ASSUMED_SAMPLE_RATE} Hz.", param="audio", effective=ASSUMED_SAMPLE_RATE, ) -__all__ = ["ASSUMED_SAMPLE_RATE", "PreparedAudio", "execute_plan"] +__all__ = [ + "ASSUMED_SAMPLE_RATE", + "DIAG_ASSUMED_SAMPLE_RATE", + "DIAG_AUDIO_CONVERSION", + "DIAG_NON_FINITE_AUDIO", + "DIAG_RESAMPLED_WITH", + "PreparedAudio", + "execute_plan", +] diff --git a/src/standard_asr/audio/format.py b/src/standard_asr/audio/format.py index 2e4d21d5..014e165b 100644 --- a/src/standard_asr/audio/format.py +++ b/src/standard_asr/audio/format.py @@ -14,6 +14,7 @@ from __future__ import annotations from pydantic import BaseModel, ConfigDict, Field, field_validator +from pydantic_core import PydanticCustomError class AudioFormat(BaseModel): @@ -63,7 +64,7 @@ def _normalize_encoding(cls, value: str) -> str: """ cleaned = value.strip().lower() if not cleaned: - raise ValueError("encoding must not be blank.") + raise PydanticCustomError("standard_asr_encoding_blank", "encoding must not be blank.") return cleaned diff --git a/src/standard_asr/audio/loader.py b/src/standard_asr/audio/loader.py index a27a30f6..2c634ff8 100644 --- a/src/standard_asr/audio/loader.py +++ b/src/standard_asr/audio/loader.py @@ -1,20 +1,6 @@ # SPDX-FileCopyrightText: 2026 Standard Voice Contributors # SPDX-License-Identifier: Apache-2.0 -# Copyright 2025 The Standard ASR Authors - -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at - -# http://www.apache.org/licenses/LICENSE-2.0 - -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - """ Audio loading and normalization utilities for the Standard ASR audio contract. diff --git a/src/standard_asr/compliance.py b/src/standard_asr/compliance.py index f6276de9..bb8a7c09 100644 --- a/src/standard_asr/compliance.py +++ b/src/standard_asr/compliance.py @@ -6,9 +6,10 @@ from __future__ import annotations import inspect +import math import threading from dataclasses import dataclass -from typing import Callable, Iterable, Literal, Sequence +from typing import Callable, ClassVar, Iterable, Literal, Protocol, Sequence, cast import numpy as np from pydantic import BaseModel, ConfigDict, ValidationError @@ -23,6 +24,8 @@ ) from standard_asr.contract.exceptions import ( ConfigError, + ConfigurationRequiredError, + EngineContractError, EntrypointValidationError, InvalidProviderParamError, UnsupportedFeatureError, @@ -48,7 +51,17 @@ DIAG_UNSUPPORTED_PARAMETER_IGNORED, _count_tokens, # pyright: ignore[reportPrivateUsage] ) -from standard_asr.runtime.interface import EngineBase +from standard_asr.runtime.interface import ( + EngineBase, + StandardASR, + ensure_wire_format_supported, +) +from standard_asr.runtime.protocol_boundary import ( + safe_class_name, + safe_type_name, + sync_result_defect, +) +from standard_asr.runtime.redaction import safe_exception_summary, sanitized_validation_message from standard_asr.runtime.streaming import ( SyncSession, TranscriptionEvent, @@ -59,6 +72,9 @@ __all__ = [ "ComplianceIssue", "ComplianceReport", + "DEFAULT_SYNC_BRIDGE_TIMEOUT", + "SupportsCapabilities", + "SupportsWireRecommendation", "assert_prefix_invariant", "check_entrypoints", "check_event_sequence", @@ -67,6 +83,7 @@ "check_streaming_param_gating", "check_sync_bridge", "check_transcription_result", + "validate_bridge_timeout", ] #: Candidate (param-field, params-builder, capability-suffix) probes for an @@ -104,7 +121,7 @@ _SUB_CONSTRAINT_PROBE_MAX_TOKENS = 4096 -def _pick_sub_constraint_probe(engine: EngineBase) -> tuple[str, RuntimeParams, str] | None: +def _pick_sub_constraint_probe(engine: StandardASR) -> tuple[str, RuntimeParams, str] | None: """Build a probe violating a declared sub-constraint of a supported feature. Used when the engine supports every probe in :data:`_GATING_PROBES` at the @@ -128,9 +145,23 @@ def _pick_sub_constraint_probe(engine: EngineBase) -> tuple[str, RuntimeParams, Returns: A ``(field_name, params, expected_diagnostic_code)`` triple, or ``None`` - when the engine declares no violable sub-constraint. + when the engine declares no violable sub-constraint (or exposes no + readable capability tree to derive one from). """ - capabilities = engine.effective_capabilities + # ``effective_capabilities`` is an EngineBase convenience, NOT a StandardASR + # protocol member: a fully-compliant structural engine may omit it, and + # reading it bare turned that omission into an AttributeError the caller + # reported as gating_probe_selection_raised -- a false FAILURE of a + # compliant engine. Fall back to the protocol's ``declared_capabilities`` + # (EngineBase's effective_capabilities defaults to exactly that); with + # neither readable there is no sub-constraint to derive, which is a no-op + # pass, not an engine fault. A PRESENT-but-raising attribute still + # propagates to the caller's containment (a broken surface stays loud). + capabilities = getattr(engine, "effective_capabilities", None) + if not isinstance(capabilities, DeclaredCapabilities): + capabilities = getattr(engine, "declared_capabilities", None) + if not isinstance(capabilities, DeclaredCapabilities): + return None prompt = capabilities.node_at("streaming.guidance.prompt") if isinstance(prompt, PromptCap) and prompt.is_supported: max_tokens = prompt.constraints.max_tokens @@ -269,6 +300,7 @@ def check_entrypoints( *, strict_discovery: bool = False, instantiate: bool = True, + names: Iterable[str] | None = None, ) -> ComplianceReport: """Validate that discovered entry points conform to expectations. @@ -295,7 +327,27 @@ def check_entrypoints( strict_discovery: Treat invalid entry-point names as a hard discovery error (reported as an error issue here, never raised). Default ``False``. Engine-identity collisions are reported as errors regardless. - instantiate: If ``True``, instantiate zero-arg factories and verify metadata. + instantiate: If ``True``, instantiate zero-arg factories and verify + the instance surface -- including one BEHAVIORAL probe: on an + engine declaring no streaming axis, ``start_transcription()`` is + called once with no arguments and MUST raise + ``UnsupportedFeatureError`` (a compliant engine refuses at the + capability gate before constructing anything; a returned session + is never entered, but a non-compliant implementation may still + run arbitrary author code in the method body). ``False`` skips + instantiation and the probe with it. + names: Restrict the PER-ENGINE checks to these model keys; ``None`` + checks every discovered engine. The registry-global invariants + (RuntimeParams closedness, engine-identity collisions, + no-entry-points) always evaluate the whole environment -- they + are environment facts, not per-engine verdicts. Pass the user's + named subset here rather than filtering the report afterwards: + the instance checks EXECUTE engine code (construction, a + ``supports()`` sweep, the ``start_transcription()`` refusal + probe -- a model load, for a cloud engine potentially a billable + call), a side effect that must not be paid on a co-installed + plugin the caller never named, for a verdict they are never + shown. Returns: Compliance report summarizing findings. @@ -316,7 +368,10 @@ def check_entrypoints( ComplianceIssue( level="error", code="entrypoint_invalid", - message=(f"Strict discovery rejected one or more entry points: {exc}"), + message=( + "Strict discovery rejected one or more entry points: " + f"{safe_exception_summary(exc)}" + ), model=None, ) ) @@ -367,7 +422,12 @@ def check_entrypoints( ) return ComplianceReport(registry=registry, issues=issues) - for name in registry.names(): + # The per-engine loop honors the caller's named scope; an unknown name is + # the caller's own lookup error to surface (the CLI resolves its names + # against this same registry), not silently "checked". + named = None if names is None else set(names) + selected = [n for n in registry.names() if named is None or n in named] + for name in selected: _check_engine(registry, name, instantiate=instantiate, issues=issues) return ComplianceReport(registry=registry, issues=issues) @@ -404,7 +464,8 @@ def _check_engine( level="error", code="engine_check_crashed", message=( - f"Checking engine {name!r} raised {exc!r}; an engine's public " + f"Checking engine {name!r} raised " + f"{safe_exception_summary(exc)}; an engine's public " "surface (properties / config / capabilities) MUST be readable " "without raising during a compliance check." ), @@ -461,23 +522,71 @@ def _check_engine_unguarded( try: instance = factory() - except (ConfigError, ValidationError) as exc: - # A credentialed engine's zero-arg factory raises when the required - # credential is absent (explicit config > env > raise). On a clean CI with + except ConfigurationRequiredError as exc: + # A credentialed engine's zero-arg factory raises this when the required + # credential is absent (explicit config > env > raise; from_env raises + # the narrow subtype automatically). On a clean CI with # no env vars set this is the *correct* behavior, so it MUST NOT be a # compliance error -- otherwise the verdict would depend on the runtime's # credential state rather than the plugin. Report it as a warning skip and # point at the env var; pass --no-instantiate or set the credential to run # the full instance-level checks. + # `exc` is typically from_env's sanitized ConfigurationRequiredError, + # but an engine building config another way raises its own -- embed + # through the total safe renderer, never repr(). issues.append( ComplianceIssue( level="warning", code="factory_requires_config", message=( "Skipped instantiation: the factory requires configuration not " - f"present in this environment ({exc!r}). Set the engine's " - "STANDARD_ASR__ environment variable (e.g. an API " - "key) or pass an explicit config to run the full instance checks." + f"present in this environment ({safe_exception_summary(exc)}). " + "Set the engine's " + "STANDARD_ASR___ environment variable (double " + "underscore between engine and field, per env_var_name; e.g. an " + "API key) or pass an explicit config to run the full instance " + "checks." + ), + model=name, + ) + ) + return + except (ConfigError, ValidationError) as exc: + # Any OTHER config/validation failure is a defect, not a missing + # credential: an invalid supplied value, an internally inconsistent + # declaration, or a factory building a broken internal model. Waiving + # these as "requires config" let a broken plugin read as + # green-with-warning -- the skip is reserved for the narrow + # ConfigurationRequiredError (absence), which from_env raises + # automatically; an engine building config another way must raise it + # itself for the missing-credential state. + # + # ONE total boundary for the embedded text: a raw ValidationError + # reaches here un-wrapped (the factory is called directly, not through + # ModelRegistry.create's sanitizing wrap) and its repr echoes the + # offending input_value; an engine-authored ConfigError may have + # interpolated a chained ValidationError's echo into its own message + # (raise ConfigError(f"bad: {ve}") from ve), which repr() re-leaks + # and a hostile __repr__ turns into a second crash site. + # safe_exception_summary handles all of it: sanitized loc/msg for the + # ValidationError, marked sanitized wrappers kept, everything else + # withheld, total under hostile __str__/__repr__. + defect = ( + sanitized_validation_message(exc, prefix="ValidationError") + if isinstance(exc, ValidationError) + else safe_exception_summary(exc) + ) + issues.append( + ComplianceIssue( + level="error", + code="factory_config_invalid", + message=( + f"Factory invocation failed with a configuration/validation " + f"defect ({defect}). If this state is actually 'required " + "configuration absent from the environment' (e.g. a missing " + "credential), raise ConfigurationRequiredError instead " + "(BaseConfig.from_env does so automatically); compliance " + "skips that state rather than failing it." ), model=name, ) @@ -488,7 +597,7 @@ def _check_engine_unguarded( ComplianceIssue( level="error", code="entrypoint_factory_failed", - message=f"Factory invocation failed with {exc!r}.", + message=f"Factory invocation failed with {safe_exception_summary(exc)}.", model=name, ) ) @@ -499,6 +608,266 @@ def _check_engine_unguarded( _check_instance_properties(instance, spec, name, issues) _check_instance_config(instance, name, issues) _check_instance_capabilities(instance, name, issues) + _check_supports_contract(instance, name, issues) + _check_instance_wire_format(instance, name, issues) + + +def _check_instance_wire_format( + instance: object, + name: str, + issues: list[ComplianceIssue], +) -> None: + """Round-trip ``recommended_wire_format()`` for EVERY constructed engine. + + The protocol member is unconditionally required (spec §3.1: the + recommendation is Properties-pure and capability-blind), so its + self-consistency round-trip holds for every engine — batch-only included. + Gating it on a streaming axis (as the CLI once did) let a batch-only + engine ship a raising, wrong-typed, or self-inconsistent implementation + that every consumer of the member would then trip over. Runs here, at the + entrypoint layer, so one ``compliance run`` exercises it exactly once per + engine. + + Args: + instance: The instantiated engine. + name: The model key (for issue attribution). + issues: The mutable list of issues to append to. + """ + member = getattr(instance, "recommended_wire_format", None) + if not callable(member) or inspect.iscoroutinefunction(member): + # Absence and the `async def` modality are already reported by the + # surface checks; calling here would crash redundantly or manufacture + # the very coroutine the modality check exists to prevent. + return + if not isinstance(getattr(instance, "properties", None), BaseProperties): + # The round-trip validates the format against the engine's Properties; + # a missing/invalid Properties is already reported by the properties + # checks, and running the round-trip against it would only add noise. + return + # Runtime-verified above (callable member + typed properties); the cast + # only names what was just checked. + engine = cast(SupportsWireRecommendation, instance) + issues.extend(_wire_format_round_trip_issues(engine, model=name)) + + +def _check_supports_contract( + instance: object, + name: str, + issues: list[ComplianceIssue], +) -> None: + """Verify ``supports()`` semantics: shape, fail-closed unknowns, tree agreement. + + Every capability negotiation in the ecosystem consumes ``supports()``, + so both a wrong return SHAPE (a truthy non-bool reads as "supported" + everywhere; an awaitable is truthy AND a leaked coroutine) and a wrong + ANSWER (a hand-written ``supports()`` diverging from the capability tree + it is defined to query -- spec R5) are silent wrong capability verdicts. + Three layers, cheapest first, each pure metadata (no session is opened): + + 1. **Shape probe** (the canonical ``streaming_input`` path): synchronous, + real ``bool``. A broken shape stops here -- sweeping a malformed + ``supports()`` over the whole tree would flood one defect into dozens + of issues. + 2. **Unknown path fail-closed** (spec R5: a missing path returns + ``False``, without raising): a sentinel path guaranteed to name no + real capability MUST answer literal ``False`` + (``supports_not_fail_closed``). + 3. **Equivalence sweep**: for every queryable node path + (:meth:`~standard_asr.contract.capabilities.DeclaredCapabilities.iter_queryable_paths` + -- supported nodes, unsupported nodes, containers, constraint + submodels, ``x_*`` subtrees) the answer MUST be identical to the + engine's own capability tree's answer. The baseline is + ``effective_capabilities`` when it is a valid tree (what + ``EngineBase.supports`` itself queries and what R5's "current + usability" means), else ``declared_capabilities``, else the sweep is + skipped (an invalid tree is already reported by the capabilities + checks). Mismatches aggregate into ONE + ``supports_disagrees_with_capabilities`` issue naming the first few + paths and the totals -- a systematically wrong implementation must not + flood the report. + + Args: + instance: The instantiated engine to probe. + name: The model key (for issue attribution). + issues: The mutable list of issues to append to. + """ + supports = getattr(instance, "supports", None) + if not callable(supports) or inspect.iscoroutinefunction(supports): + # Absence and the `async def` modality are already reported by the + # surface checks; calling here would crash redundantly or manufacture + # the very coroutine the modality check exists to prevent. + return + try: + value = supports("streaming_input") + except Exception as exc: # noqa: BLE001 - contained per-engine, run continues + issues.append( + ComplianceIssue( + level="error", + code="supports_raised", + message=( + f"supports('streaming_input') raised " + f"{safe_exception_summary(exc)}; the capability " + "surface must answer a dot-path query without raising " + "(consumers fail closed on it, so a raising supports() reads " + "as 'nothing supported')." + ), + model=name, + ) + ) + return + if _sync_member_violation(value, "supports()", name, issues, expected_type=bool): + return + _check_supports_unknown_path(supports, name, issues) + _check_supports_tree_agreement(instance, supports, name, issues) + + +#: A dot-path guaranteed to name no real capability: its first segment is not a +#: capability-tree field and (not being ``x_*``-prefixed) can never resolve into +#: the extension namespace either, so the fail-closed contract (spec R5) pins +#: the answer to a literal ``False`` for every compliant engine. +_SUPPORTS_UNKNOWN_PROBE = "standard_asr_compliance.nonexistent_capability_probe" + +#: Cap on the mismatch paths named inline by ``supports_disagrees_with_capabilities`` +#: (the totals always report the full extent). +_SUPPORTS_MISMATCH_DISPLAY_LIMIT = 10 + + +def _check_supports_unknown_path( + supports: Callable[[str], object], + name: str, + issues: list[ComplianceIssue], +) -> None: + """Probe an unknown capability path: the answer MUST be literal ``False``. + + The reported description never embeds an arbitrary return value's + ``repr`` (the sync-call boundary's own rule -- an engine-fabricated + object could smuggle payload text into the report): a ``bool`` shows its + value, anything else shows its type only. + + Args: + supports: The engine's (shape-verified) ``supports`` callable. + name: The model key (for issue attribution). + issues: The mutable list of issues to append to. + """ + try: + answer = supports(_SUPPORTS_UNKNOWN_PROBE) + except Exception as exc: # noqa: BLE001 - contained per-engine, run continues + described = f"raised {safe_type_name(exc)}" + else: + if answer is False: + return + defect = sync_result_defect(answer) + if defect is not None: + # A stray coroutine has been closed by the boundary; report the + # modality honestly rather than repr-ing a dead coroutine object. + described = ( + "answered an awaitable" + if defect.kind == "awaitable" + else f"returned a result the sync boundary could not classify ({defect.clause})" + ) + elif answer is True: + described = "answered True" + else: + described = f"answered a {safe_type_name(answer)} (value withheld)" + issues.append( + ComplianceIssue( + level="error", + code="supports_not_fail_closed", + message=( + f"supports() {described} for an unknown capability path; " + "the capability model is fail-closed (spec R5): a path that " + "does not exist in the tree MUST answer literal False, without " + "raising. Anything else makes applications negotiate features " + "the engine never declared." + ), + model=name, + ) + ) + + +def _check_supports_tree_agreement( + instance: object, + supports: Callable[[str], object], + name: str, + issues: list[ComplianceIssue], +) -> None: + """Sweep every queryable path: ``supports()`` MUST agree with the tree. + + Args: + instance: The instantiated engine (for the baseline trees). + supports: The engine's (shape-verified) ``supports`` callable. + name: The model key (for issue attribution). + issues: The mutable list of issues to append to. + """ + baseline = _supports_baseline_tree(instance) + if baseline is None: + # An absent/invalid capability tree is already reported by the + # capabilities checks; sweeping against it would only manufacture a + # cascade of noise on top of the real defect. + return + mismatches: list[str] = [] + total = 0 + for path in baseline.iter_queryable_paths(): + total += 1 + expected = baseline.supports(path) + try: + actual = supports(path) + except Exception as exc: # noqa: BLE001 - contained per-engine, run continues + mismatches.append(f"{path} (raised {safe_type_name(exc)})") + continue + defect = sync_result_defect(actual, expected_type=bool) + if defect is not None: + mismatches.append(f"{path} ({defect})") + continue + if actual is not expected: + mismatches.append(f"{path} (answered {actual!r}, tree says {expected!r})") + if not mismatches: + return + shown = mismatches[:_SUPPORTS_MISMATCH_DISPLAY_LIMIT] + overflow = len(mismatches) - len(shown) + suffix = f"; ... and {overflow} more" if overflow else "" + issues.append( + ComplianceIssue( + level="error", + code="supports_disagrees_with_capabilities", + message=( + f"supports() disagrees with the engine's capability tree on " + f"{len(mismatches)} of {total} queryable paths: " + f"{'; '.join(shown)}{suffix}. supports() is defined as a direct " + "query of the effective capability tree (spec R5; " + "EngineBase.supports IS effective_capabilities.supports) -- " + "every capability negotiation reads these answers, so a " + "divergence is a silent wrong verdict on every consumer." + ), + model=name, + ) + ) + + +def _supports_baseline_tree(instance: object) -> DeclaredCapabilities | None: + """Pick the tree ``supports()`` is expected to answer from, defensively. + + ``effective_capabilities`` (an ``EngineBase`` convenience, not a protocol + member) wins when present and valid -- it is what ``EngineBase.supports`` + queries and what spec R5's "current usability" means; a structural engine + without it falls back to the protocol's ``declared_capabilities`` (what + ``EngineBase`` defaults effective to). Anything invalid yields ``None`` + (the capability checks own reporting that). + + Args: + instance: The instantiated engine. + + Returns: + The baseline tree, or ``None`` when no valid tree is reachable. + """ + for attr in ("effective_capabilities", "declared_capabilities"): + try: + tree = getattr(instance, attr, None) + except Exception: # noqa: BLE001 - a raising convenience property + continue + if isinstance(tree, DeclaredCapabilities): + return tree + return None def _check_instance_properties( @@ -547,11 +916,16 @@ def _check_instance_properties( try: type(properties).model_validate(properties.model_dump()) except ValidationError as exc: + # str(exc) echoes the offending input_value; the message lands in + # terminals/CI logs, so render the sanitized loc/msg summary instead. issues.append( ComplianceIssue( level="error", code="properties_revalidation_failed", - message=f"Instance properties fail re-validation: {exc}", + message=sanitized_validation_message( + exc, + prefix="Instance properties fail re-validation", + ), model=name, ) ) @@ -592,9 +966,9 @@ def _check_instance_config( code="config_type_mismatch", message=( "Instance config is not an instance of the declared " - f"config_type ({config.__class__.__name__!r} is not a " - f"{declared_config_type.__name__!r}); the schema published " - "for UIs would not match the config actually consumed." + f"config_type ({safe_type_name(cast('object', config))!r} is not a " + f"{safe_class_name(cast('type', declared_config_type))!r}); the schema " + "published for UIs would not match the config actually consumed." ), model=name, ) @@ -638,7 +1012,7 @@ def _check_instance_capabilities( level="error", code="effective_capabilities_raised", message=( - f"Reading effective_capabilities raised {exc!r}; the " + f"Reading effective_capabilities raised {safe_exception_summary(exc)}; the " "property MUST return a DeclaredCapabilities (or None) " "without raising." ), @@ -670,7 +1044,7 @@ def _check_instance_capabilities( code="effective_capabilities_wrong_type", message=( "effective_capabilities is not a DeclaredCapabilities " - f"(got {type(effective).__name__!r}); it MUST be a " + f"(got {safe_type_name(effective)!r}); it MUST be a " "DeclaredCapabilities (or None) so the effective ⊆ " "declared invariant can be verified." ), @@ -703,12 +1077,22 @@ def _check_language_axis_config( if isinstance(instance, EngineBase): try: instance._validate_language_config() # pyright: ignore[reportPrivateUsage] - except (ConfigError, ValueError) as exc: + except (ConfigError, EngineContractError, ValueError) as exc: + # EngineContractError covers the DECLARATION side of the same + # runtime validation (a missing IC.6 default, a malformed + # declared tag); ConfigError/ValueError the value side. A raw + # ValidationError IS a ValueError and its str() echoes the + # offending input; scrub it before the message reaches CI logs. + detail = ( + sanitized_validation_message(exc, prefix="ValidationError") + if isinstance(exc, ValidationError) + else str(exc) + ) issues.append( ComplianceIssue( level="error", code="language_config_invalid", - message=f"Language config is invalid; every transcribe will fail: {exc}", + message=f"Language config is invalid; every transcribe will fail: {detail}", model=name, ) ) @@ -736,9 +1120,25 @@ def _check_language_axis_config( #: Public callables every compliant engine MUST expose unconditionally -#: (StandardASR protocol). ``start_transcription`` is required only -#: when the engine declares a streaming axis -- handled separately below. -_ALWAYS_REQUIRED_METHODS: tuple[str, ...] = ("transcribe", "transcribe_async", "supports") +#: (StandardASR protocol -- the COMPLETE public surface, batch-only included). +_ALWAYS_REQUIRED_METHODS: tuple[str, ...] = ( + "transcribe", + "transcribe_async", + # ALWAYS present per the protocol ("start_transcription is always + # present; a batch-only engine raises UnsupportedFeatureError from it"): + # the protocol's whole point is that callers type an engine as StandardASR + # and call the streaming entry point without a cast or hasattr probe. A + # batch-only engine that OMITS the method hands those callers an + # AttributeError instead of the standardized fail-closed rejection -- + # certifying that shape would let compliance pass an object that does not + # satisfy the very protocol it certifies. + "start_transcription", + "supports", + # Part of the StandardASR protocol: the documented first step of the + # streaming journey, derivable from Properties (EngineBase provides it for + # free; a structural engine must implement it). + "recommended_wire_format", +) def _check_required_surface( @@ -748,14 +1148,17 @@ def _check_required_surface( ) -> None: """Verify the engine exposes the full required public surface. - Every engine MUST expose the unconditional batch/query surface - (:meth:`transcribe`, :meth:`transcribe_async`, :meth:`supports`); a missing - member is a compliance **error**, not a silent accept. ``start_transcription`` - is required **only** when the engine declares a streaming axis - (``streaming_input`` or ``streaming_output``) -- a batch-only engine - legitimately omits it. The ``properties``/``declared_capabilities`` - attributes are verified by the caller's type checks; this helper covers the - callable methods and the conditional streaming entry point. + Every engine MUST expose the unconditional surface pinned by + :data:`_ALWAYS_REQUIRED_METHODS` -- :meth:`transcribe`, + :meth:`transcribe_async`, :meth:`start_transcription` (ALWAYS present per + the ``StandardASR`` protocol; a batch-only engine raises + ``UnsupportedFeatureError`` from it rather than omitting it, so protocol- + typed callers never hit an ``AttributeError``), :meth:`supports`, and + :meth:`recommended_wire_format` (derivable from Properties even for + batch-only engines); a missing member is a compliance **error**, not a + silent accept. The ``properties``/``declared_capabilities`` attributes are + verified by the caller's type checks; this helper covers the callable + methods plus the streaming-declaration consistency check below. For an :class:`EngineBase` engine the streaming requirement uses the same :meth:`~standard_asr.runtime.interface.EngineBase._overrides_streaming` predicate @@ -772,7 +1175,8 @@ def _check_required_surface( issues: The mutable list of issues to append to. """ for method in _ALWAYS_REQUIRED_METHODS: - if not callable(getattr(instance, method, None)): + attr = getattr(instance, method, None) + if not callable(attr): issues.append( ComplianceIssue( level="error", @@ -784,50 +1188,248 @@ def _check_required_surface( model=name, ) ) - - # ``start_transcription`` is required iff the engine declares streaming. Read - # the declared axes defensively: a malformed ``declared_capabilities`` (its - # own error is raised elsewhere) simply means we cannot assert a streaming - # requirement here, so we do not over-report. - declared = getattr(instance, "declared_capabilities", None) - declares_streaming = isinstance(declared, DeclaredCapabilities) and ( - declared.supports("streaming_input") or declared.supports("streaming_output") - ) - if not declares_streaming: - return - if isinstance(instance, EngineBase): - # The base template always provides start_transcription, so presence is - # not enough: the engine must override the _start_transcription hook, or - # the runtime raises UnsupportedFeatureError at session establishment. - if not instance._overrides_streaming(): # pyright: ignore[reportPrivateUsage] + elif method != "transcribe_async" and inspect.iscoroutinefunction(attr): + # Modality is part of the surface: every member except + # transcribe_async is SYNCHRONOUS (async behavior lives in + # transcribe_async and inside the returned session). An + # `async def` implementation hands protocol-typed callers a + # coroutine where a result is pinned -- and every behavioral + # probe would otherwise manufacture never-awaited coroutines + # (RuntimeWarnings under warnings-as-errors) exercising it. issues.append( ComplianceIssue( level="error", - code="streaming_declared_not_implemented", + code="protocol_member_not_synchronous", message=( - "Instance declares a streaming axis (streaming_input / " - "streaming_output) but does not implement the streaming hook " - "(_start_transcription); start_transcription would raise " - "UnsupportedFeatureError at runtime (fail-closed: a declared " - "capability is a promise)." + f"{method!r} is an `async def`; the StandardASR " + "protocol pins it as a SYNCHRONOUS member (async " + "behavior lives in transcribe_async and inside the " + "returned session)." ), model=name, ) ) + + # Presence is unconditional (checked above for every engine); what remains + # is the streaming-declaration CONSISTENCY check. Read the declared axes + # defensively: a malformed ``declared_capabilities`` (its own error is + # raised elsewhere) simply means we cannot assert anything here, so we do + # not over-report. + declared = getattr(instance, "declared_capabilities", None) + declares_streaming = isinstance(declared, DeclaredCapabilities) and ( + declared.supports("streaming_input") or declared.supports("streaming_output") + ) + if not declares_streaming: + if isinstance(declared, DeclaredCapabilities): + # Presence alone does not verify the protocol's batch-only + # promise; the refusal probe below does. Skipped when + # declared_capabilities is unreadable (we cannot know the engine + # is batch-only, and its own error is reported elsewhere). + _check_batch_only_streaming_refusal(instance, name, issues) return - if not callable(getattr(instance, "start_transcription", None)): + if isinstance(instance, EngineBase) and not instance._overrides_streaming(): # pyright: ignore[reportPrivateUsage] + # The base template always provides start_transcription, so presence is + # not enough for a streaming-DECLARING EngineBase engine: it must + # override the _start_transcription hook, or the runtime raises + # UnsupportedFeatureError at session establishment. issues.append( ComplianceIssue( level="error", - code="missing_start_transcription", + code="streaming_declared_not_implemented", message=( "Instance declares a streaming axis (streaming_input / " - "streaming_output) but is missing a callable " - "'start_transcription' method (required by the StandardASR protocol)." + "streaming_output) but does not implement the streaming hook " + "(_start_transcription); start_transcription would raise " + "UnsupportedFeatureError at runtime (fail-closed: a declared " + "capability is a promise)." + ), + model=name, + ) + ) + + +def _sync_member_violation( + value: object, + member: str, + model: str | None, + issues: list[ComplianceIssue], + *, + expected_type: type | tuple[type, ...] | None = None, +) -> bool: + """Contain and report a sync protocol member's wrong-shaped return value. + + THE single guard every behavioral probe applies after calling a + SYNCHRONOUS ``StandardASR`` member (``transcribe`` / + ``start_transcription`` / ``supports`` / ``recommended_wire_format`` -- + async behavior lives in ``transcribe_async`` and inside the returned + session). Two defect shapes are contained: + + * **Awaitable**: an ``async def`` implementation (or a sync wrapper + delegating to one) hands back an awaitable that (a) is the wrong + result type for every consumer and (b) becomes a never-awaited + coroutine polluting the run with a ``RuntimeWarning`` under + warnings-as-errors. A bare coroutine is CLOSED so nothing leaks; + reported as ``protocol_member_not_synchronous``. + * **Wrong type** (when ``expected_type`` is given): a value outside the + member's pinned return type -- the canonical case is ``supports()`` + returning a truthy non-bool (``"false"``, an object), which every + truthiness-based consumer would silently misread as "supported", a + wrong capability negotiation. Reported as + ``protocol_member_wrong_return_type``. The check is strict + ``isinstance`` (a ``numpy.bool_`` is NOT a ``bool``): the protocol + pins the type, so quacking is not compliance -- return a real ``bool``. + * **Unclassifiable**: the boundary's own classification introspection + raised against the value's type metadata (a hostile metaclass, a + broken ``__class__`` property). No consumer can safely classify such + a result, so that IS the defect -- reported as + ``protocol_member_unclassifiable_result`` with the boundary's honest + clause, never by re-inspecting the value here. + + Any defect gets one stable code instead of masquerading as whatever + verdict the probe would have drawn from the malformed value. + + Args: + value: The member's return value. + member: Display name of the member (for the message). + model: The model key to attribute the issue to, or ``None``. + issues: The mutable issue list to append to. + expected_type: The member's pinned return type(s), or ``None`` to + check only synchronicity. + + Returns: + ``True`` when the value violated the contract (reported and, for a + coroutine, closed); ``False`` for a normal conforming result. + """ + defect = sync_result_defect(value, expected_type=expected_type) + if defect is None: + return False + # The VERDICT selects the issue code; the value is never re-inspected + # here (its metadata already trained containment once). + if defect.kind == "awaitable": + issues.append( + ComplianceIssue( + level="error", + code="protocol_member_not_synchronous", + message=( + f"{member} returned an awaitable; the StandardASR protocol pins " + "it as a SYNCHRONOUS member (async behavior lives in " + "transcribe_async and inside the returned session), so " + "protocol-typed callers can never await its result." + ), + model=model, + ) + ) + return True + if defect.kind == "unclassifiable": + issues.append( + ComplianceIssue( + level="error", + code="protocol_member_unclassifiable_result", + message=( + f"{member} {defect}; the sync-return boundary itself could not " + "reach a verdict because the value's own type metadata raised " + "under inspection. A result no consumer can safely classify " + "violates the contract outright (fail-closed), independent of " + "what the value might have been." + ), + model=model, + ) + ) + return True + issues.append( + ComplianceIssue( + level="error", + code="protocol_member_wrong_return_type", + message=( + f"{member} returned {safe_type_name(value)!r}, not the " + "protocol-pinned return type; consumers negotiating " + "capabilities on truthiness would silently misread it " + "(e.g. a non-empty string reads as 'supported'). Return " + "a real value of the pinned type (for supports(): a bool)." + ), + model=model, + ) + ) + return True + + +def _check_batch_only_streaming_refusal( + instance: object, + name: str, + issues: list[ComplianceIssue], +) -> None: + """Verify a batch-only engine REFUSES ``start_transcription()`` correctly. + + The protocol's batch-only promise is behavioral, not just structural: + ``start_transcription`` is always present AND a batch-only engine raises + ``UnsupportedFeatureError`` from it (fail-closed: an undeclared capability + is not supported, spec Capabilities R1). Checking presence alone would + certify an engine that silently ACCEPTS a streaming session it never + declared -- the inverse capability lie -- or one that hands protocol-typed + callers a non-standard exception where the contract pins the type. + + Side-effect envelope: same as the streaming gating check -- the no-arg + call may CONSTRUCT a session on a non-compliant engine but never enters + it, so the standard layer opens no wire connection. A compliant engine + raises at the capability gate before any construction. + + Args: + instance: The instantiated engine, whose ``declared_capabilities`` + declare no streaming axis. + name: The model key (for issue attribution). + issues: The mutable list of issues to append to. + """ + method = getattr(instance, "start_transcription", None) + if not callable(method): + # Absence is already reported by the unconditional surface check; + # probing a non-callable would just crash with a redundant TypeError. + return + if inspect.iscoroutinefunction(method): + # Already reported by the surface modality check + # (protocol_member_not_synchronous); calling an `async def` here + # would only manufacture a never-awaited coroutine on top of it. + return + try: + session = method() + except UnsupportedFeatureError: + return + except Exception as exc: # noqa: BLE001 - reported, never re-raised + issues.append( + ComplianceIssue( + level="error", + code="batch_only_streaming_refusal_wrong_error", + message=( + f"start_transcription() on a batch-only engine raised " + f"{safe_exception_summary(exc)}; " + "the StandardASR protocol pins the refusal type: a batch-only " + "engine MUST raise UnsupportedFeatureError so protocol-typed " + "callers can rely on one standardized fail-closed rejection." ), model=name, ) ) + return + if _sync_member_violation(session, "start_transcription()", name, issues): + # A SYNC wrapper returning an awaitable (e.g. delegating to an + # internal `async def`) slips the iscoroutinefunction pre-checks; the + # shared guard closed the stray coroutine and reported the modality + # defect -- it must not additionally read as "returned a session". + return + issues.append( + ComplianceIssue( + level="error", + code="batch_only_streaming_not_refused", + message=( + "start_transcription() on a batch-only engine returned " + f"{safe_type_name(session)!r} instead of raising " + "UnsupportedFeatureError. Accepting a streaming session while " + "declaring no streaming axis is a capability lie in reverse " + "(undeclared-but-implemented); declare the axis or refuse the " + "call fail-closed." + ), + model=name, + ) + ) def prepare_requires_arguments(prepare: Callable[..., object]) -> bool: @@ -1047,7 +1649,7 @@ def _check_class_level_metadata(spec: ModelSpec, name: str, issues: list[Complia code="class_metadata_unreadable", message=( "declared_capabilities/properties are not readable without " - f"instantiation: {exc}" + f"instantiation: {safe_exception_summary(exc)}" ), model=name, ) @@ -1138,7 +1740,7 @@ def _check_class_level_metadata(spec: ModelSpec, name: str, issues: list[Complia code="provider_params_type_not_closed", message=( "provider_params_type must be a closed type (extra='forbid'); " - f"{params_type.__name__} is not." + f"{safe_class_name(params_type)} is not." ), model=name, ) @@ -1580,40 +2182,7 @@ def _safe_engine_id(engine: object) -> str | None: return None -def _synthesize_probe_audio_format(engine: EngineBase) -> AudioFormat: - """Build a *legal* wire :class:`AudioFormat` for a ``streaming_input`` probe. - - The streaming gating probe must hand the engine's - :meth:`~standard_asr.runtime.interface.EngineBase._start_transcription` hook a - valid session context: an engine that does not self-manage its wire format - (an incremental ElevenLabs-style adapter) legitimately fail-louds when opened - with ``audio_format=None`` (bare-PCM streaming locks the sample rate at - session establishment). Probing it with no ``audio_format`` - would make that *correct* fail-loud read as a compliance error. So the probe - uses the engine's own - :meth:`~standard_asr.runtime.interface.EngineBase.recommended_wire_format` -- the - single source of truth -- which yields a format the engine's own - :meth:`~standard_asr.runtime.interface.EngineBase.ensure_stream_format_supported` - accepts. - - Args: - engine: The engine under test (must declare ``streaming_input``). - - Returns: - A wire format that the engine's session-establishment guard accepts. - - Raises: - ValueError: When the engine recommends no usable wire format (declares no - usable sample rate), so no legal probe context can be built. The - caller maps this to a ``gating_probe_context_unbuildable`` issue. - """ - fmt = engine.recommended_wire_format() - if fmt is None: - raise ValueError("engine declares no usable wire sample rate to synthesize a probe format") - return fmt - - -def check_streaming_param_gating(engine: EngineBase) -> ComplianceReport: +def check_streaming_param_gating(engine: StandardASR) -> ComplianceReport: """Assert a streaming engine gates an unsupported standard parameter. Closes the streaming-gating bypass gap as a *compliance* failure rather @@ -1642,8 +2211,9 @@ def check_streaming_param_gating(engine: EngineBase) -> ComplianceReport: best_effort-diagnose contract. **Legal session context.** A ``streaming_input`` engine is probed with a - synthesized, *valid* wire :class:`AudioFormat` (see - :func:`_synthesize_probe_audio_format`), so an engine that legitimately + valid wire :class:`AudioFormat` taken from the engine's own + :meth:`~standard_asr.runtime.interface.StandardASR.recommended_wire_format` + (guarded like every other sync-member call), so an engine that legitimately fail-louds on a missing ``audio_format`` is not misjudged as non-compliant for obeying the standard. A ``streaming_output``-only engine is probed with a one-sample silent ``audio`` input, but **only under the strict @@ -1683,21 +2253,32 @@ def check_streaming_param_gating(engine: EngineBase) -> ComplianceReport: model = _safe_engine_id(engine) try: + # EVERY supports() result goes through the guard with + # expected_type=bool BEFORE its truthiness is consulted: an + # `async def` (or a conditional wrapper answering some paths with a + # coroutine) hands back a TRUTHY awaitable, and a truthy non-bool + # ("false", an object) reads as "supported" -- either way the probe + # would negotiate capabilities on a lie while leaking never-awaited + # coroutines per call. supports_input = engine.supports("streaming_input") + if _sync_member_violation(supports_input, "supports()", model, issues, expected_type=bool): + return ComplianceReport(registry=None, issues=issues) supports_output = engine.supports("streaming_output") + if _sync_member_violation(supports_output, "supports()", model, issues, expected_type=bool): + return ComplianceReport(registry=None, issues=issues) if not (supports_input or supports_output): # The engine does not declare streaming support; there is no # streaming gating contract to exercise. return ComplianceReport(registry=None, issues=issues) - probe = next( - ( - (p[0], p[1](), DIAG_UNSUPPORTED_PARAMETER_IGNORED) - for p in _GATING_PROBES - if not engine.supports(p[2]) - ), - None, - ) + probe: tuple[str, RuntimeParams, str] | None = None + for p in _GATING_PROBES: + supported = engine.supports(p[2]) + if _sync_member_violation(supported, "supports()", model, issues, expected_type=bool): + return ComplianceReport(registry=None, issues=issues) + if not supported: + probe = (p[0], p[1](), DIAG_UNSUPPORTED_PARAMETER_IGNORED) + break if probe is None: # Every probed parameter is supported at the feature level; fall # back to violating a declared sub-constraint of a supported @@ -1714,7 +2295,8 @@ def check_streaming_param_gating(engine: EngineBase) -> ComplianceReport: level="error", code="gating_probe_selection_raised", message=( - f"selecting a streaming gating probe raised {exc!r}; " + f"selecting a streaming gating probe raised " + f"{safe_exception_summary(exc)}; " "supports()/effective_capabilities must not raise while the " "compliance suite probes the engine's declarations." ), @@ -1736,8 +2318,14 @@ def check_streaming_param_gating(engine: EngineBase) -> ComplianceReport: # only under strict (a best_effort probe there would run real inference). open_kwargs: dict[str, object] = {"params": params} if supports_input: + # The probe must hand the engine's session hook a VALID wire format: + # an engine that does not self-manage its wire format legitimately + # fail-louds when opened with audio_format=None, and probing it bare + # would make that correct rejection read as a compliance error. The + # engine's own recommended_wire_format() is the single source (the + # compliance suite separately asserts its self-consistency). try: - open_kwargs["audio_format"] = _synthesize_probe_audio_format(engine) + fmt = engine.recommended_wire_format() except Exception as exc: # noqa: BLE001 issues.append( ComplianceIssue( @@ -1745,13 +2333,43 @@ def check_streaming_param_gating(engine: EngineBase) -> ComplianceReport: code="gating_probe_context_unbuildable", message=( f"could not synthesize a legal wire audio_format from the " - f"engine's Properties to probe gating ({exc!r}); declare a " + f"engine's Properties to probe gating " + f"({safe_exception_summary(exc)}); declare a " "reachable native_sample_rate / wire_encodings." ), model=model, ) ) return ComplianceReport(registry=None, issues=issues) + if _sync_member_violation( + fmt, + "recommended_wire_format()", + model, + issues, + expected_type=(AudioFormat, type(None)), + ): + # A coroutine is not None and not an AudioFormat: without this + # guard an `async def` recommendation leaked unawaited into the + # session open below and drew a context/crash verdict for a + # modality defect. The type pin also stops a duck-typed non-format + # object from being fed into session establishment. + return ComplianceReport(registry=None, issues=issues) + if fmt is None: + issues.append( + ComplianceIssue( + level="error", + code="gating_probe_context_unbuildable", + message=( + "could not synthesize a legal wire audio_format from the " + "engine's Properties to probe gating (no usable positive " + "sample rate is declared); declare a " + "reachable native_sample_rate / wire_encodings." + ), + model=model, + ) + ) + return ComplianceReport(registry=None, issues=issues) + open_kwargs["audio_format"] = fmt elif not strict: # streaming_output-only + best_effort: reaching gating requires an # ``audio`` input, which best_effort would decode and feed to the model @@ -1831,7 +2449,8 @@ def check_streaming_param_gating(engine: EngineBase) -> ComplianceReport: level="error", code="gating_probe_crashed", message=( - f"start_transcription raised {exc!r} while probing streaming " + f"start_transcription raised " + f"{safe_exception_summary(exc)} while probing streaming " f"parameter {field_name!r}; the only contractual exception for a " "gated parameter is UnsupportedFeatureError." ), @@ -1840,6 +2459,22 @@ def check_streaming_param_gating(engine: EngineBase) -> ComplianceReport: ) return ComplianceReport(registry=None, issues=issues) + if _sync_member_violation( + session, "start_transcription()", model, issues, expected_type=TranscriptionSession + ): + # A coroutine is TRUTHY: without this guard an `async def` + # start_transcription read as "strict engine accepted the parameter" + # (a wrong verdict for a different defect) while leaking a + # never-awaited coroutine into the run. The type pin mirrors the + # reference server's establishment boundary (require_sync_result + # pins TranscriptionSession): a duck-typed object exposing only + # diagnostics() satisfied the best_effort read below and PASSED the + # default compliance run, then failed every /v1/stream WebSocket + # with internal_error -- a defect the suite exists to catch before + # the plugin ships, reachable in the default run only here + # (check_sync_bridge pins it too but is opt-in/billable). + return ComplianceReport(registry=None, issues=issues) + # The session was created but NOT opened: the base start_transcription # template constructs the session without entering its context (no # __aenter__/_open), and the best_effort verdict below needs only @@ -1875,7 +2510,8 @@ def check_streaming_param_gating(engine: EngineBase) -> ComplianceReport: level="error", code="gating_diagnostics_raised", message=( - f"session.diagnostics() raised {exc!r} while checking for the " + f"session.diagnostics() raised " + f"{safe_exception_summary(exc)} while checking for the " f"expected {expected_code!r} diagnostic on best_effort streaming " f"parameter {field_name!r}; diagnostics() must not raise." ), @@ -1913,7 +2549,7 @@ class _ForeignProviderParams(ProviderParams): model_config = ConfigDict(extra="forbid") -def check_provider_params_swap_safety(engine: EngineBase) -> ComplianceReport: +def check_provider_params_swap_safety(engine: StandardASR) -> ComplianceReport: """Assert an engine always rejects another engine's ``provider_params``. The standard makes ``provider_params`` swap-safety an unconditional MUST: @@ -1949,16 +2585,18 @@ def check_provider_params_swap_safety(engine: EngineBase) -> ComplianceReport: silence = np.zeros(1, dtype=np.float32) try: - engine.transcribe(silence, params) + result = engine.transcribe(silence, params) except InvalidProviderParamError: # Correct: swapped provider_params rejected before any model work. return ComplianceReport(registry=None, issues=issues) - except ConfigError as exc: + except (ConfigError, EngineContractError) as exc: # The engine raised BEFORE the provider-params gate could run: the base - # template validates the language config (_validate_language_config) ahead - # of gate_params, and that method promises ConfigError (it even wraps a - # malformed-tag ValueError into ConfigError), so a broken language axis - # surfaces here as ConfigError. Swap-safety was therefore never + # template validates the language config (_validate_language_config) + # ahead of gate_params, and that method promises ConfigError for a + # bad configuration VALUE and EngineContractError for a DECLARATION + # defect (a malformed declared tag, a missing IC.6 default), so a + # broken language axis surfaces here as one of the two. Swap-safety + # was therefore never # exercised -- this is unverifiable, not a swap miss; attribute it to the # real defect rather than mislabel a language_config_invalid engine as # swap-unsafe. (A bare ValueError is NOT caught here: a swap rejection @@ -1969,7 +2607,8 @@ def check_provider_params_swap_safety(engine: EngineBase) -> ComplianceReport: level="error", code="provider_params_swap_unverifiable", message=( - f"transcribe raised {exc!r} before the provider_params gate, so " + f"transcribe raised " + f"{safe_exception_summary(exc)} before the provider_params gate, so " "swap-safety could not be exercised; resolve the " "engine's language_config_invalid defect first." ), @@ -1987,7 +2626,8 @@ def check_provider_params_swap_safety(engine: EngineBase) -> ComplianceReport: level="error", code="provider_params_swap_not_enforced", message=( - f"transcribe raised {exc!r} for a foreign provider_params type " + f"transcribe raised " + f"{safe_exception_summary(exc)} for a foreign provider_params type " "instead of InvalidProviderParamError; the standard requires " "provider_params swap-safety to raise InvalidProviderParamError " "ALWAYS (independent of strict/best_effort), validated before " @@ -1998,6 +2638,13 @@ def check_provider_params_swap_safety(engine: EngineBase) -> ComplianceReport: ) return ComplianceReport(registry=None, issues=issues) + if _sync_member_violation(result, "transcribe()", model, issues): + # An `async def` transcribe returns a coroutine WITHOUT raising: + # without this guard the probe read that as "silently accepted the + # foreign params" -- a wrong verdict for a different defect -- while + # leaking a never-awaited coroutine into the run. + return ComplianceReport(registry=None, issues=issues) + # No exception at all: the engine silently accepted another engine's params -- # exactly the swap bug this check exists to make loud. issues.append( @@ -2015,27 +2662,93 @@ def check_provider_params_swap_safety(engine: EngineBase) -> ComplianceReport: return ComplianceReport(registry=None, issues=issues) -def check_recommended_wire_format(engine: EngineBase) -> ComplianceReport: +class SupportsWireRecommendation(Protocol): + """The two-member surface :func:`check_recommended_wire_format` needs. + + A deliberately minimal protocol instead of ``EngineBase``: the check's + subjects include structural (non-``EngineBase``) engines -- the standard's + own promise -- and the previous ``EngineBase``-typed signature invited + calling ``EngineBase``-only members on them (the check once called + ``ensure_stream_format_supported``, not a ``StandardASR`` member, so a + fully-compliant structural engine failed with a false + ``recommended_wire_format_self_inconsistent`` verdict on an + ``AttributeError``). + """ + + properties: ClassVar[BaseProperties] + + def recommended_wire_format(self) -> AudioFormat | None: + """Return the engine's recommended minimal wire format. + + Returns: + The recommended format, or ``None`` when none is derivable. + """ + ... + + +def check_recommended_wire_format( + engine: SupportsWireRecommendation, *, model: str | None = None +) -> ComplianceReport: """Assert an engine's recommended wire format is one it would itself accept. :meth:`~standard_asr.runtime.interface.EngineBase.recommended_wire_format` is the single source of truth for the minimal wire :class:`AudioFormat` the standard layer opens a ``streaming_input`` session with when the application chose none -- the CLI sync-bridge runner and the streaming gating probe both rely on it. - A self-inconsistent engine, whose recommended format its own + A self-inconsistent engine, whose recommended format the standard + session-establishment rule rejects for its own declared Properties, would + make those paths fail-loud on a format the standard layer chose rather + than the application -- a silent-looking compliance trap. This closes that + loop: when a format is recommended it MUST pass + :func:`~standard_asr.runtime.interface.ensure_wire_format_supported` -- the + pure ``(Properties, AudioFormat)`` rule that :meth:`~standard_asr.runtime.interface.EngineBase.ensure_stream_format_supported` - rejects, would make those paths fail-loud on a format the standard layer - chose rather than the application -- a silent-looking compliance trap. This - closes that loop: when a format is recommended it MUST pass the engine's own - session-establishment guard. + itself implements. Validating via the pure rule (never the ``EngineBase`` + method) keeps the verdict correct for structural engines, which have no + such method. Args: - engine: The engine under test (declares ``streaming_input``). + engine: The engine under test. Deliberately NOT required to declare + ``streaming_input`` (or any capability): the recommendation is + Properties-pure and capability-blind (see + :meth:`~standard_asr.runtime.interface.EngineBase.recommended_wire_format`), + so the self-consistency round-trip holds for every engine — the + protocol member is unconditionally required (spec §3.1) and the + entrypoint-layer instance checks run this round-trip for EVERY + successfully constructed engine (batch-only included; an + output-only engine passes trivially). + model: The model key (``engine/model``) to attribute issues to, or + ``None`` for a single-engine run. In a multi-model run an + unattributed issue renders as ```` and the user cannot + tell which engine failed. Returns: A :class:`ComplianceReport`. ``passed`` is ``True`` when no format is recommended, or the recommended format is accepted by the engine. """ + return ComplianceReport( + registry=None, issues=_wire_format_round_trip_issues(engine, model=model) + ) + + +def _wire_format_round_trip_issues( + engine: SupportsWireRecommendation, *, model: str | None +) -> list[ComplianceIssue]: + """Run the wire-format self-consistency round-trip, returning its issues. + + The single body behind both entry points: the public + :func:`check_recommended_wire_format` (library API, wraps the issues in a + report) and the entrypoint-layer :func:`_check_instance_wire_format` + (appends them to the per-engine instance-check list). One body means the + two surfaces can never drift on what "self-consistent" means. + + Args: + engine: The engine under test. + model: The model key to attribute issues to, or ``None``. + + Returns: + The issues found (empty for a compliant engine). + """ issues: list[ComplianceIssue] = [] try: fmt = engine.recommended_wire_format() @@ -2044,34 +2757,129 @@ def check_recommended_wire_format(engine: EngineBase) -> ComplianceReport: ComplianceIssue( level="error", code="recommended_wire_format_raised", - message=f"EngineBase.recommended_wire_format() raised: {exc!r}.", - model=None, + message=( + f"EngineBase.recommended_wire_format() raised: {safe_exception_summary(exc)}." + ), + model=model, ) ) - return ComplianceReport(registry=None, issues=issues) + return issues + if _sync_member_violation( + fmt, + "recommended_wire_format()", + model, + issues, + expected_type=(AudioFormat, type(None)), + ): + # A coroutine is not None: without this guard an `async def` + # implementation fell into the round-trip below and was misreported + # as self-inconsistent while leaking a never-awaited coroutine. The + # type pin closes the same misreporting for any non-AudioFormat + # return: a duck-typed object with plausible attributes used to pass + # the round-trip silently, and one without them drew a + # "self-inconsistent" verdict for what is a wrong-return-type defect. + return issues if fmt is not None: try: - engine.ensure_stream_format_supported(fmt) + ensure_wire_format_supported(engine.properties, fmt) except Exception as exc: # noqa: BLE001 - reported as a compliance error issues.append( ComplianceIssue( level="error", code="recommended_wire_format_self_inconsistent", message=( - f"recommended_wire_format() returned {fmt!r}, but the engine's " - f"own ensure_stream_format_supported rejects it: {exc!r}. The " - "recommended format must be one the engine accepts." + f"recommended_wire_format() returned {fmt!r}, but the standard " + "session-establishment rule rejects it for the engine's own " + f"declared Properties: " + f"{safe_exception_summary(exc)}. The recommended format must be " + "one the engine accepts." ), - model=None, + model=model, ) ) - return ComplianceReport(registry=None, issues=issues) + return issues + + +class SupportsCapabilities(Protocol): + """The one-method surface :func:`check_sync_bridge` needs from an engine. + + A deliberately minimal protocol instead of ``StandardASR``: the check + consults nothing but ``supports()``, and demanding the full surface would + force every caller with a partial test double (or a wrapper) to fake + members the check never touches. (``StandardASR`` itself is now + strict-assignable from real plugins -- ``config`` is a read-only protocol + property -- so this narrowing is about least-surface, not a typing + workaround.) + """ + + def supports(self, dot_path: str) -> bool: + """Return whether the capability at ``dot_path`` is supported. + + Args: + dot_path: A capability dot-path. + + Returns: + ``True`` if supported. + """ + ... + + +#: Default per-phase timeout (seconds) for the sync-bridge check: session +#: establishment and the bridged drive each receive this budget. THE single +#: source of the value: the CLI's --bridge-timeout help and effective default +#: both read it, so a change here can never leave the CLI silently applying +#: (or --help advertising) a stale number. +DEFAULT_SYNC_BRIDGE_TIMEOUT = 5.0 + + +def validate_bridge_timeout(timeout: float) -> float: + """Validate a sync-bridge timeout: MUST be finite and strictly positive. + + The single owner of the rule, shared by :func:`check_sync_bridge` and the + CLI's ``--bridge-timeout`` parser (which wraps the ``ValueError`` into an + argparse usage error) so the two layers can never drift: ``<= 0`` yields + an instant false "did not terminate" verdict against a compliant engine, + ``inf``/``nan`` hangs the check on the very deadlock it diagnoses, and a + finite value above ``threading.TIMEOUT_MAX`` would blow up as an + ``OverflowError`` out of ``Thread.join`` / ``Future.result`` mid-check -- + a validated timeout MUST be one the bridge's waits can actually take. No + clamping: silently shortening a caller's timeout would be an implicit + rewrite of an explicit value. + + Args: + timeout: The candidate timeout in seconds. + + Returns: + ``timeout`` unchanged. + + Raises: + ValueError: If ``timeout`` is not finite, not strictly positive, or + exceeds this platform's ``threading.TIMEOUT_MAX``. + """ + if not math.isfinite(timeout) or timeout <= 0: + raise ValueError( + f"sync-bridge timeout must be a finite number of seconds > 0, " + f"got {timeout!r} (<= 0 yields an instant false 'did not terminate' " + "verdict; inf/nan hangs the check on the deadlock it diagnoses)." + ) + if timeout > threading.TIMEOUT_MAX: + raise ValueError( + f"sync-bridge timeout must be <= threading.TIMEOUT_MAX " + f"({threading.TIMEOUT_MAX!r} s on this platform), got {timeout!r}: " + "the bridge waits with Thread.join / Future.result, which raise " + "OverflowError beyond the platform's lock-wait cap. Pass a " + "smaller timeout (no clamping -- an over-cap budget is a caller " + "mistake, not a value to silently rewrite)." + ) + return timeout def check_sync_bridge( session_factory: Callable[[], TranscriptionSession], *, - timeout: float = 5.0, + timeout: float = DEFAULT_SYNC_BRIDGE_TIMEOUT, + model: str | None = None, + engine: SupportsCapabilities | None = None, ) -> ComplianceReport: """Drive an async adapter's :class:`SyncSession` from an external thread. @@ -2085,33 +2893,375 @@ def check_sync_bridge( Args: session_factory: A zero-argument callable returning a fresh async :class:`TranscriptionSession` (e.g. ``engine.start_transcription`` - bound with its arguments). - timeout: Seconds to allow the bridged session to drain and close. This + bound with its arguments). The return crosses the same sync-call + boundary as every protocol member: a factory handing back an + awaitable (an ``async def`` ``start_transcription`` behind the + CLI's canonical factory) or any non-``TranscriptionSession`` + object is reported as ``sync_bridge_invalid_session`` -- with a + stray coroutine closed -- instead of being driven into + :class:`SyncSession` and misreported as a bridge lifecycle fault. + timeout: Seconds granted to EACH phase of the check independently: + session establishment (``session_factory()`` plus, on an + unsupported refusal, the ``supports()`` classification probe) and + the bridged drive (open, end-of-audio, drain, close combined -- + also each bridged lifecycle call's ``submit_timeout``). Both + phases run under bounded daemon workers, so a hanging + ``start_transcription`` or ``supports()`` is reported instead of + hanging the check; worst case the check takes about twice this + value. Per-phase (not a shared total) so a slow-but-successful + establishment can never starve the drive join into a false + "did not terminate" verdict. MUST be finite and + strictly positive: ``<= 0`` would make the wait return immediately + (a false "did not terminate" verdict against a compliant engine) + and ``inf``/``nan`` would hang the check on the very deadlock it + exists to diagnose, so both are rejected loudly (the same rule the + CLI's ``--bridge-timeout`` enforces at parse time). It also caps + each bridged lifecycle call (forwarded as the ``SyncSession`` + ``submit_timeout``), so granting a larger budget genuinely extends + slow-but-compliant ``_open``/``_close`` phases. This MUST exceed the adapter's real ``_open`` + ``_close`` cost: a slow but compliant adapter (a cloud session doing a real network handshake) is *not* a deadlock, so when a run reports a timeout, re-run with a larger value to tell "slow" from "stuck". The driver thread is a daemon, so a false positive (or a real deadlock) never blocks interpreter exit -- the process is not held hostage by the fault this check diagnoses. + model: The model key (``engine/model``) to attribute issues to, or + ``None`` for a single-engine run (a multi-model run needs the + attribution to name the failing engine). + engine: The engine the factory drives, if available (anything with a + ``supports()`` method -- see :class:`SupportsCapabilities`; the + full ``StandardASR`` protocol is deliberately not required, so a + real plugin passes without casts). Used for exactly + one thing: classifying an ``UnsupportedFeatureError`` raised by + ``session_factory()`` itself (session establishment). Only an + engine that does NOT declare ``streaming_input`` earns the passing + ``sync_bridge_not_applicable`` verdict -- the bridge feeds bare + frames, which such an engine genuinely cannot accept. An engine + that DECLARES ``streaming_input`` yet refuses establishment is a + capability lie (a declared-but-unimplemented hook, or a + recommended wire format its own guard rejects) and FAILS. Without + ``engine`` the classification is fail-closed: an establishment + refusal is reported as a failure, with a hint to pass ``engine=`` + when the engine is genuinely output-only. Returns: A :class:`ComplianceReport`. ``passed`` is ``True`` when the bridge - terminated cleanly with no leaked background loop thread. + terminated cleanly with no leaked background loop thread, or when the + check is not applicable (session establishment refused as unsupported + by an engine KNOWN not to declare ``streaming_input``; reported as a + ``sync_bridge_not_applicable`` warning, never as an engine failure). + An ``UnsupportedFeatureError`` from anywhere PAST establishment (the + adapter's ``_open``, ``end_audio``, event drain, close) is always a + failing ``sync_bridge_raised`` -- the not-applicable carve-out is + scoped to the factory call alone. + + Raises: + ValueError: If ``timeout`` is not finite or not strictly positive (a + caller code bug, rejected independent of any policy). """ + validate_bridge_timeout(timeout) issues: list[ComplianceIssue] = [] + + # Establish the session BEFORE any bridging, in its own bounded daemon + # worker (NOT on the calling thread): a hanging start_transcription -- or + # a hanging engine.supports() during the classification probe below, + # plugin code is arbitrary -- is exactly the fault class this no-deadlock + # check exists to diagnose, so the check must never itself hang on either. + # Scoping establishment outside the DRIVE worker keeps the not-applicable + # carve-out surgical: an UnsupportedFeatureError from the adapter's own + # lifecycle (_open, end_audio, drain, close) can then NEVER be mistaken + # for "the check does not apply" -- it stays a failing sync_bridge_raised + # like any other mid-bridge exception. Each phase (establishment; bridged + # drive) is granted the FULL ``timeout``: carving one budget across both + # let a slow-but-successful establishment starve the drive join into an + # instant false "did not terminate" verdict. + established: dict[str, object] = {} + established_lock = threading.Lock() + abandoned = threading.Event() + + def _teardown_late_session(late_session: TranscriptionSession) -> None: + """Best-effort close of a session that arrived after the check gave up. + + Args: + late_session: The session ``start_transcription`` eventually built. + """ + try: + # Close-only drive: __exit__ without __enter__ is tolerated by the + # base session (a never-entered session just awaits _close), so + # the teardown NEVER opens the session -- driving _open here would + # initiate a fresh (for cloud adapters: billable) connection + # purely to destroy it, the very cost that makes the bridge + # opt-in. + SyncSession(late_session, submit_timeout=timeout).__exit__(None, None, None) + except BaseException: # noqa: BLE001, S110 - best-effort; the check already + # reported (this runs after the timeout verdict); BaseException so + # the establish worker's late cleanup dies as quietly as intended + # even under plugin SystemExit -- the same containment rule as the + # two verdict-bearing workers. + pass + + def _establish() -> None: + try: + session_local = session_factory() + except UnsupportedFeatureError as exc: + with established_lock: + established["exc"] = exc + # Classification probe runs HERE, inside the bounded worker: + # supports() is plugin code and may block; the caller's thread + # must stay hang-proof. ``classified`` is set only AFTER the probe + # completes: an exc without it means the probe is still hanging, + # and the main thread must report did-not-terminate rather than + # classify on incomplete state (a wrong "Pass engine=" hint for a + # caller who DID pass the engine). + if engine is not None: + try: + # cast to object, not the declared bool: the whole point + # of this guard is engines whose supports() violates its + # static type at runtime. + raw_declared = cast("object", engine.supports("streaming_input")) + except BaseException: # noqa: BLE001 - a broken supports() cannot earn a pass + # BaseException, like the two sibling workers: this probe + # runs INSIDE the `except UnsupportedFeatureError` block, + # so a BaseException raised here is NOT caught by that + # try's own BaseException arm (Python never routes an + # exception raised in an except block to a sibling + # clause). The worker would die with `classified` unset + # and the main thread would report + # sync_bridge_did_not_terminate -- a timeout verdict for + # what is really a broken supports(). + with established_lock: + established["supports_raised"] = True + else: + # The shared sync-call boundary: an awaitable (a TRUTHY + # coroutine bool() would coerce to a declared-streaming + # verdict, then leak unawaited) or a truthy non-bool + # ("false", an object) is a broken capability surface -- + # classify fail-closed, never fabricate a verdict. + supports_defect = sync_result_defect(raw_declared, expected_type=bool) + with established_lock: + if supports_defect is not None: + established["supports_invalid"] = supports_defect + else: + established["declared_streaming_input"] = raw_declared + with established_lock: + established["classified"] = True + except BaseException as exc: # noqa: BLE001 - classified below + # BaseException DELIBERATELY: this runs on a daemon worker thread, + # where an uncaught SystemExit/KeyboardInterrupt (or any + # BaseException an engine raises) would die silently and the main + # thread would misread the empty state dict as an establishment + # HANG -- a wrong verdict against the engine. Every escape is + # classified into the state dict instead; the main thread decides. + with established_lock: + established["exc"] = exc + established["classified"] = True + else: + # The factory's return value crosses the SAME sync-call boundary + # as every protocol member: the CLI's canonical factory wraps + # start_transcription, so an `async def` opener (or a sync + # wrapper delegating to one) hands back an awaitable here -- + # storing it as the session would misreport a modality defect as + # a bridge lifecycle fault deep inside SyncSession while the + # coroutine leaked unawaited, and an arbitrary non-session object + # would surface only as a confusing secondary AttributeError. + factory_defect = sync_result_defect(session_local, expected_type=TranscriptionSession) + if factory_defect is not None: + with established_lock: + established["factory_invalid"] = factory_defect + return + with established_lock: + if abandoned.is_set(): + late = session_local + else: + established["session"] = session_local + late = None + if late is not None: + # The check already reported an establishment timeout; do not + # leak the late session's resources (connections, state). + _teardown_late_session(late) + + establish_worker = threading.Thread( + target=_establish, name="compliance-sync-bridge-establish", daemon=True + ) + establish_worker.start() + establish_worker.join(timeout=timeout) + with established_lock: + # Success requires either a stored session or a FULLY classified + # exception (exc + classified): an exc whose supports() probe is still + # hanging must read as a timeout, not be classified on partial state. + # is_alive() is deliberately not consulted -- a worker momentarily + # alive while exiting after a completed store must not read as hung. + timed_out = ( + "session" not in established + and "factory_invalid" not in established + and not ("exc" in established and "classified" in established) + ) + if timed_out: + abandoned.set() + if timed_out: + issues.append( + ComplianceIssue( + level="error", + code="sync_bridge_did_not_terminate", + message=( + f"Session establishment did not complete within {timeout}s -- " + "start_transcription (or the supports() classification probe) " + "hung, or legitimately needs longer. Re-run with a larger " + "timeout to disambiguate (library: check_sync_bridge(..., " + "timeout=...); CLI: standard-asr compliance run " + "--include-bridge --bridge-timeout SECONDS). A session that " + "finishes establishing after this report is closed " + "best-effort, not leaked." + ), + model=model, + ) + ) + return ComplianceReport(registry=None, issues=issues) + factory_invalid = cast("str | None", established.get("factory_invalid")) + if factory_invalid is not None: + # The factory returned, but not a session: an awaitable (the CLI's + # canonical factory wraps start_transcription, so this is an + # `async def` opener -- the entry-point checks report the member as + # protocol_member_not_synchronous) or some other non-session object. + # Driving it into SyncSession would misreport the defect as a bridge + # lifecycle fault; report it at the boundary it violated instead. + issues.append( + ComplianceIssue( + level="error", + code="sync_bridge_invalid_session", + message=( + f"session_factory {factory_invalid}; check_sync_bridge " + "requires a factory that SYNCHRONOUSLY returns a " + "TranscriptionSession (start_transcription is a " + "synchronous protocol member -- async behavior lives " + "inside the returned session)." + ), + model=model, + ) + ) + return ComplianceReport(registry=None, issues=issues) + exc_or_none = established.get("exc") + + if isinstance(exc_or_none, UnsupportedFeatureError): + exc = exc_or_none + declared_streaming_input = cast("bool | None", established.get("declared_streaming_input")) + supports_raised = bool(established.get("supports_raised")) + supports_invalid = cast("str | None", established.get("supports_invalid")) + if supports_invalid is not None: + # supports() answered, but with the wrong SHAPE (an awaitable or + # a non-bool): the declaration is unverifiable through a broken + # capability surface, and fabricating a verdict from truthiness + # would be a capability decision built on a type error. + issues.append( + ComplianceIssue( + level="error", + code="sync_bridge_raised", + message=( + "Session establishment raised UnsupportedFeatureError " + f"({safe_exception_summary(exc)}). " + f"The engine's own supports() {supports_invalid} " + "while verifying streaming_input -- a broken capability " + "surface cannot earn a not-applicable pass; supports() " + "must synchronously return a bool (the entry-point " + "checks flag this too)." + ), + model=model, + ) + ) + return ComplianceReport(registry=None, issues=issues) + if declared_streaming_input is False: + # The one honest not-applicable shape: the engine itself says it + # cannot accept bare-frame input, so the bridge has nothing to test. + issues.append( + ComplianceIssue( + level="warning", + code="sync_bridge_not_applicable", + message=( + "Sync-bridge check not applicable: the engine does not " + "declare streaming_input and refused session " + f"establishment as unsupported ({safe_exception_summary(exc)}). " + "The bridge feeds " + "bare PCM frames; this is a property of the check, not " + "an engine failure." + ), + model=model, + ) + ) + return ComplianceReport(registry=None, issues=issues) + # Fail-closed: the engine declares streaming_input (a refusal is then a + # capability lie), or no engine was provided so the claim cannot be + # verified -- an unverifiable establishment refusal MUST NOT pass. + issues.append( + ComplianceIssue( + level="error", + code="sync_bridge_raised", + message=( + "Session establishment raised UnsupportedFeatureError " + f"({safe_exception_summary(exc)}). " + + ( + "The engine DECLARES streaming_input, so refusing a " + "bare-frame session is a capability lie (a declared-but-" + "unimplemented streaming hook, or a recommended wire " + "format the engine's own guard rejects)." + if declared_streaming_input + else ( + "The engine's own supports() raised while verifying " + "streaming_input -- a broken capability surface " + "cannot earn a not-applicable pass; fix supports() " + "first (the entry-point checks flag it too)." + if supports_raised + else "Pass engine=... so the check can verify " + "whether the engine declares streaming_input (a " + "genuinely output-only engine is then reported " + "not-applicable instead of failing)." + ) + ) + ), + model=model, + ) + ) + return ComplianceReport(registry=None, issues=issues) + if isinstance(exc_or_none, BaseException): + issues.append( + ComplianceIssue( + level="error", + code="sync_bridge_raised", + message=(f"Session establishment raised: {safe_exception_summary(exc_or_none)}."), + model=model, + ) + ) + return ComplianceReport(registry=None, issues=issues) + # _establish stored either "exc" (handled above) or the constructed + # session; the dict is object-typed only because it crosses the thread. + session = cast(TranscriptionSession, established["session"]) + outcome: dict[str, object] = {} worker_name = "compliance-sync-bridge" def _drive() -> None: sync: SyncSession | None = None try: - sync = SyncSession(session_factory()) + # The user's timeout budget applies to the bridged lifecycle calls + # too (submit_timeout), not only the outer join: otherwise a + # --bridge-timeout above SyncSession's internal 30 s default was + # silently inert for open/close and a slow-but-compliant adapter + # failed as "raised" no matter how much time the user granted. + sync = SyncSession(session, submit_timeout=timeout) with sync: sync.end_audio() events = list(sync) outcome["terminal"] = any(getattr(ev, "is_terminal", False) for ev in events) - except Exception as exc: # noqa: BLE001 - reported as a compliance error - outcome["error"] = repr(exc) + except BaseException as exc: # noqa: BLE001 - reported as a compliance error + # BaseException, matching _establish's containment: a SystemExit + # (or CancelledError) out of plugin code on this daemon worker + # would otherwise kill the thread WITHOUT writing "error", and + # the main thread mis-reads the silent death as + # sync_bridge_no_terminal -- a false verdict about the wrong + # defect. Store the exception OBJECT; the main thread renders it + # through the total safe renderer (freezing repr(exc) in-thread + # had the same silent-death failure mode under a hostile + # __repr__). + outcome["error"] = exc finally: # Record the bridge's OWN loop-thread liveness so the leak check below # asserts on this thread specifically. A compliant adapter may pull in a @@ -2138,11 +3288,13 @@ def _drive() -> None: f"SyncSession did not terminate within {timeout}s -- this may be a " "deadlock OR an adapter whose _open/_close legitimately takes " f"longer than {timeout}s. Re-run with a larger timeout to " - "disambiguate. If it is a deadlock, check the sync-bridge adapter " - "contract: bind loop resources in __aenter__, never touch the " - "ambient event loop." + "disambiguate (library: check_sync_bridge(..., timeout=...); " + "CLI: standard-asr compliance run --include-bridge " + "--bridge-timeout SECONDS). If it is a deadlock, check the " + "sync-bridge adapter contract: bind loop resources in " + "__aenter__, never touch the ambient event loop." ), - model=None, + model=model, ) ) return ComplianceReport(registry=None, issues=issues) @@ -2152,8 +3304,11 @@ def _drive() -> None: ComplianceIssue( level="error", code="sync_bridge_raised", - message=f"SyncSession raised while bridging: {outcome['error']}.", - model=None, + message=( + "SyncSession raised while bridging: " + f"{safe_exception_summary(cast('BaseException', outcome['error']))}." + ), + model=model, ) ) elif not outcome.get("terminal"): @@ -2166,7 +3321,7 @@ def _drive() -> None: level="error", code="sync_bridge_no_terminal", message="SyncSession ended without emitting a terminal event.", - model=None, + model=model, ) ) @@ -2181,7 +3336,7 @@ def _drive() -> None: level="error", code="sync_bridge_thread_leak", message="SyncSession did not tear down its owned background loop thread on close.", - model=None, + model=model, ) ) diff --git a/src/standard_asr/contract/capabilities.py b/src/standard_asr/contract/capabilities.py index ca2211c6..6b7b5413 100644 --- a/src/standard_asr/contract/capabilities.py +++ b/src/standard_asr/contract/capabilities.py @@ -27,12 +27,30 @@ from __future__ import annotations +from collections.abc import Mapping from typing import Any, Iterator, Literal, Sequence, cast -from pydantic import BaseModel, ConfigDict, Field, field_validator, model_validator +from pydantic import ( + BaseModel, + ConfigDict, + Field, + JsonValue, + TypeAdapter, + field_validator, + model_validator, +) + +from standard_asr.contract.results import require_json_string_keys WordTimestampGranularityName = Literal["word", "segment", "char"] +#: The closed set of mode-domain names (spec: the capability tree's top-level +#: partitions). Homed here -- the module that DEFINES the mode domains -- so +#: contract-layer signatures (e.g. ``effective_candidate_languages``'s +#: ``mode``) can use the precise type without importing upward from +#: ``runtime.gating`` (which re-exports it as ``Mode``). +ModeName = Literal["batch", "streaming"] + #: Mode values that count as "not supported" for enum/mode archetype nodes. _UNSUPPORTED_MODES = frozenset({"none", "unsupported"}) @@ -65,6 +83,53 @@ def _is_extension_key(key: object) -> bool: return isinstance(key, str) and key.startswith(_EXTENSION_PREFIX) +def _reject_separator_keys_on_node_surface(extras: Mapping[str, object]) -> None: + """Reject a queryable-surface key that embeds the dot-path separator. + + The dot-path grammar is the protocol's ONE query surface: + :meth:`DeclaredCapabilities.supports`, ``iter_queryable_paths`` / + ``iter_supported_paths``, ``covers`` and the compliance sweep all join + and split node names on ``"."``. A node key containing the separator + breaks that bijection both ways: the joined path can never resolve + (``supports`` splits it into segments the tree does not have, so the + sweep would fail a compliant plugin whose hand-written ``supports`` + honestly answers ``True`` for its own declared vendor capability), and + two DISTINCT trees (``{"a.b": node}`` vs ``{"a": {"b": node}}``) join + to the SAME path string, letting ``covers``'s set containment conflate + them. A dotted key is legal JSON, so it is rejected HERE, loudly, + naming the key -- never silently mis-resolved later. + + Only the node-traversal surface is constrained -- exactly the keys + :func:`_children` / :func:`_get_child` walk: ``x_*`` extension keys and + every key of a dict reachable by dict nesting from one. Dicts inside + lists are field internals (never nodes), and non-extension extras are + not queryable; their keys stay free, so value data keyed by e.g. + ``"v1.2"`` remains representable outside the path space. + + Args: + extras: A model's extra keys (values already key-vetted as exact + ``str`` at every depth by ``require_json_string_keys``). + + Raises: + ValueError: If a queryable-surface key contains ``"."``. + """ + stack: list[tuple[str, object]] = [ + (key, value) for key, value in extras.items() if _is_extension_key(key) + ] + while stack: + key, value = stack.pop() + if "." in key: + raise ValueError( + f"capability key {key!r} contains '.', the dot-path separator: " + "its path could never resolve via supports() and would collide " + "with a genuinely nested spelling. Rename the key (e.g. use " + "'_'), or move dotted-name data into a list or scalar value, " + "which is outside the queryable path space." + ) + if isinstance(value, dict): + stack.extend(cast("dict[str, object]", value).items()) + + def granularity_offers_all(granularities: Sequence[str]) -> bool: """Return whether a declared ``granularities`` list means "unbounded (all)". @@ -94,15 +159,102 @@ def granularity_offers_all(granularities: Sequence[str]) -> bool: return not granularities -class _CapNode(BaseModel): +#: The JSON value space every extra key is validated into. A capability +#: tree is a first-class wire-visible contract surface (G.5.2 -- the same +#: model on the Python and wire layers), so an extension value must be +#: expressible as a JSON document at CONSTRUCTION: otherwise the Python tree +#: accepts a state (an arbitrary object, NaN/Inf) whose wire projection can +#: only fail later, at the metadata endpoint. +#: +#: Why a validator + adapter rather than the ``__pydantic_extra__`` typed +#: annotation: the native mechanism cannot express the floor contract here +#: -- pydantic 2.5 builds typed extras only from an EAGER annotation, while +#: this module (like the whole project) annotates lazily via +#: ``from __future__ import annotations`` (the floor raises +#: ``PydanticSchemaGenerationError`` on the deferred form at class +#: creation). The adapter below was profiled to accept/reject identically +#: to the native mechanism on both the floor and current pydantic. +#: +#: ``allow_inf_nan=False`` is set on the ADAPTER explicitly: the enclosing +#: model's config does not propagate into a standalone ``TypeAdapter``. +_EXTRA_VALUE_ADAPTER = TypeAdapter(dict[str, JsonValue], config=ConfigDict(allow_inf_nan=False)) + + +class _JsonExtraModel(BaseModel): + """Base for capability-tree models: tolerant KEYS, closed VALUES. + + ``extra="allow"`` keeps parsing forward-compatible (a future standard + field or a typo is tolerated rather than fatal), while the value space + of every such key closes construction-side: anything a JSON document + cannot express is rejected loudly instead of surfacing as a projection + failure at the metadata endpoint. Non-finite floats are not JSON + (``allow_inf_nan=False``) and are rejected at the same boundary. + + "Tolerant keys" tolerates UNKNOWN keys, not un-JSON ones: every extra + key must be an exact ``str`` at every depth (the same + :func:`~standard_asr.contract.results.require_json_string_keys` rule + the results-layer wire slots enforce). The check runs BEFORE the value + adapter because the adapter's lax ``dict[str, ...]`` validation would + otherwise DECODE a bytes key into its str spelling and the merge would + re-home the laundered key -- ``{b"supported": True}`` silently + overriding a declared ``supported=False``, or ``b"x_vendor"`` minting a + canonical extension key the input never spelled. + """ + + model_config = ConfigDict(frozen=True, extra="allow", allow_inf_nan=False) + + @model_validator(mode="before") + @classmethod + def _extras_are_json_values(cls, data: Any) -> Any: + """Move every non-field key's value into the JSON value space. + + The canonicalized adapter output is stored, not just checked, so the + in-process tree and any later ``model_validate`` of the wire + document agree byte-for-byte (e.g. a str-subclass value settles to a + plain ``str`` here rather than at dump time). + + Args: + data: The raw constructor input. + + Returns: + The input, with extra values replaced by their validated + canonical form. + """ + if not isinstance(data, Mapping): + # An already-constructed instance gating through model_validate + # carries only values its own construction vetted. + return data + mapping = cast("Mapping[Any, Any]", data) + declared = cls.model_fields + extras: dict[Any, Any] = { + key: value for key, value in mapping.items() if key not in declared + } + if not extras: + return cast("Any", data) + # The KEY domain first (fail loudly): with every extra key proven an + # exact str, the adapter below canonicalizes only VALUES -- no key + # can change spelling, so no laundered collision with a declared + # field and no order-sensitive merge is possible. + require_json_string_keys(extras) + # And the PATH grammar: a queryable-surface key must not embed the + # dot-path separator, or the tree mints paths supports() can never + # resolve (see :func:`_reject_separator_keys_on_node_surface`). + _reject_separator_keys_on_node_surface(extras) + validated = _EXTRA_VALUE_ADAPTER.validate_python(extras) + merged = dict(mapping) + # Same keys, canonicalized values: updating in place keeps each key + # at its original position in the document. + merged.update(validated) + return merged + + +class _CapNode(_JsonExtraModel): """Base class for all capability leaf nodes. Subclasses MUST expose an ``is_supported`` boolean property derived from their archetype (flag/bounded -> ``supported``; enum/mode -> ``mode``). """ - model_config = ConfigDict(frozen=True, extra="allow") - @property def is_supported(self) -> bool: # pragma: no cover - overridden """Whether this capability is supported. @@ -143,18 +295,17 @@ def _mode_supported(mode: str) -> bool: # --------------------------------------------------------------------------- # # Constraint submodels (machine-checkable limits, live with their feature). # --------------------------------------------------------------------------- # -class CandidateLanguagesConstraints(BaseModel): +class CandidateLanguagesConstraints(_JsonExtraModel): """Constraints for the candidate-languages capability. Attributes: max: Maximum number of candidate languages accepted. """ - model_config = ConfigDict(frozen=True, extra="allow") max: int = Field(..., gt=0, description="Maximum number of candidate languages.") -class PromptConstraints(BaseModel): +class PromptConstraints(_JsonExtraModel): """Constraints for the prompt guidance channel. Attributes: @@ -172,7 +323,6 @@ class PromptConstraints(BaseModel): rather than at it; the standard will not exceed the declared value. """ - model_config = ConfigDict(frozen=True, extra="allow") max_tokens: int | None = Field( default=None, gt=0, @@ -184,7 +334,7 @@ class PromptConstraints(BaseModel): ) -class PhraseHintsConstraints(BaseModel): +class PhraseHintsConstraints(_JsonExtraModel): """Constraints for the phrase-hints guidance channel. Attributes: @@ -193,7 +343,6 @@ class PhraseHintsConstraints(BaseModel): max_words_per_term: Optional maximum words per term. """ - model_config = ConfigDict(frozen=True, extra="allow") max_terms: int | None = Field(default=None, gt=0, description="Maximum hint terms.") max_chars_per_term: int | None = Field( default=None, gt=0, description="Maximum characters per term." @@ -203,14 +352,13 @@ class PhraseHintsConstraints(BaseModel): ) -class DiarizationConstraints(BaseModel): +class DiarizationConstraints(_JsonExtraModel): """Constraints for the diarization capability. Attributes: max_speakers: Optional maximum number of speakers. """ - model_config = ConfigDict(frozen=True, extra="allow") max_speakers: int | None = Field(default=None, gt=0, description="Maximum speakers.") @@ -438,11 +586,9 @@ def is_supported(self) -> bool: # --------------------------------------------------------------------------- # # Container nodes (group leaves; not capabilities themselves). # --------------------------------------------------------------------------- # -class _Container(BaseModel): +class _Container(_JsonExtraModel): """Base for grouping containers; tolerant of unknown / ``x_*`` keys.""" - model_config = ConfigDict(frozen=True, extra="allow") - class LanguageCaps(_Container): """Language capabilities for one mode. @@ -712,6 +858,33 @@ def iter_supported_paths(self) -> Iterator[str]: """ yield from _iter_paths(self, prefix="") + def iter_queryable_paths(self) -> Iterator[str]: + """Yield the dot-path of every NODE in the tree -- supported or not. + + The node set is pinned by the two-layer isomorphism: exactly the + paths at which :meth:`canonical_json` renders a JSON object and at + which :meth:`supports` resolves a model/dict -- capability leaves, + containers, constraint submodels, and ``x_*`` extension subtrees + (typed or raw-dict; model extras pass the same ``x_*`` gate as every + other traversal, so a non-extension unknown key is not a node). + Scalar field values (a ``supported`` bool, a ``mode`` token, a + ``granularities`` list) are field internals, not nodes: neither + yielded nor descended. ``None`` children (an absent mode domain, a + ``constraints=None``) are skipped. + + Unlike :meth:`iter_supported_paths` (the supported-only view behind + ``effective ⊆ declared``), UNSUPPORTED nodes are yielded and + descended, so a consumer can verify the fail-closed ``False`` answers + too -- e.g. an unsupported feature's ``constraints`` submodel MUST + probe ``False``. The compliance suite sweeps this set to assert a + hand-written ``supports()`` agrees with the tree on every node. + + Yields: + Dot-paths of every capability node, container, submodel, and + extension subtree in the tree. + """ + yield from _iter_node_paths(self, prefix="") + def covers(self, other: DeclaredCapabilities) -> bool: """Return whether ``other`` is a valid narrowing of this tree. @@ -956,6 +1129,30 @@ def _iter_paths(node: object, prefix: str) -> Iterator[str]: yield from _iter_paths(cast("object", child), path) +def _iter_node_paths(node: object, prefix: str) -> Iterator[str]: + """Recursively yield every node path under ``node``, supported or not. + + A node is any :class:`~pydantic.BaseModel` or dict child reachable through + :func:`_children` (which applies the ``x_*`` gate to model extras) -- + the same object set :meth:`DeclaredCapabilities.canonical_json` renders + as JSON objects. Scalars and lists are field internals, ``None`` children + are absent domains; neither is a node. + + Args: + node: A pydantic model or dict to walk. + prefix: The accumulated dot-path prefix. + + Yields: + Every node dot-path, in traversal order. + """ + for name, child in _children(node): + if not isinstance(child, (BaseModel, dict)): + continue + path = f"{prefix}.{name}" if prefix else name + yield path + yield from _iter_node_paths(cast("object", child), path) + + def _children(node: object) -> list[tuple[str, object]]: """Return ``(name, child)`` pairs for a model or dict node. @@ -1133,6 +1330,7 @@ def _read_attr(node: object, name: str) -> object: "GuidanceCaps", "granularity_offers_all", "LanguageCaps", + "ModeName", "PhraseHintsCap", "PhraseHintsConstraints", "PromptCap", diff --git a/src/standard_asr/contract/exceptions.py b/src/standard_asr/contract/exceptions.py index 88595095..483b8245 100644 --- a/src/standard_asr/contract/exceptions.py +++ b/src/standard_asr/contract/exceptions.py @@ -59,16 +59,79 @@ def __init__( class ConfigError(StructuredError, ValueError): - """Raised when a configuration is invalid -- user-provided or engine-declared. - - Two fault domains share this type: a value the **caller** can fix (a bad - init-config field, a ``default_language`` not in ``selectable_languages``) - and a declaration mistake the **engine author** must fix (a malformed - ``selectable_languages`` / ``detectable_languages`` tag, surfaced by the - standard layer at first transcribe). The latter is a plugin bug, not a - request error -- if it reaches you through an installed engine, report it to - the plugin author rather than changing your own configuration. The server - maps this to HTTP 422 (``ValueError`` subclass). + """Raised when the CONFIGURATION -- supplied or ambient -- is invalid. + + The type asserts fault ownership: the configuration's SUPPLIER can fix + it (a bad init-config field, a ``default_language`` not in + ``selectable_languages``, a malformed ``--config`` / ``--set``). Who + that supplier is depends on the surface, and each surface maps the SAME + error accordingly: + + * **CLI**: the invoking user owns the config -- the flags AND the env + vars -- so every ``ConfigError`` is caller-actionable there: usage + exit 2, with the sanitized message naming the field to fix. + * **Reference server**: a wire client cannot supply engine config at + all (construction is zero-arg; options are the portable + ``WireRuntimeParams``), so a ``ConfigError`` reaching the server -- + at construction, transcription, or session establishment -- is a + deployment-side fault and maps to a scrubbed 500 (WS + ``internal_error``); the client-fixable rejections have their own + types (:class:`UnsupportedFeatureError` -> 422, request validation -> + 422). See :class:`ConfigurationRequiredError` for the absent-config + 503 state. + + Engine-DECLARATION defects (a malformed declared language tag, an + unsatisfiable ``prepare`` shape, an IC.6 violation) are NOT this type: + they raise :class:`EngineContractError`, because no configuration value + fixes them. An engine that raises ``ConfigError`` (or lets a + construction-time ``ValidationError``, which + ``ModelRegistry.create`` wraps into one, escape its factory) for a + fault that is NOT about the supplied configuration mis-asserts this + ownership contract -- the compliance suite's zero-arg construction + check fails such engines (``engine_construction_failed``); consumers do + not second-guess the type. The ``ValueError`` mixin serves IN-PROCESS + callers, who genuinely can pass a bad config value to a constructor. + + The one machine-distinguishable sub-state is ABSENT required + configuration: raise (or catch) :class:`ConfigurationRequiredError` for + that -- consumers such as the compliance suite treat "config missing from + this environment" (skip) differently from "config invalid" (fail). + """ + + pass + + +class ConfigurationRequiredError(ConfigError): + """Raised when required runtime configuration is ABSENT, not invalid. + + The narrow, machine-distinguishable subtype of :class:`ConfigError` for + the one state that is a fact about the ENVIRONMENT rather than about any + code or declaration: a required config field (typically a credential) was + neither passed explicitly nor found in the environment. Consumers use the + distinction to keep two very different verdicts apart: + + * the compliance suite SKIPS instantiation-level checks on this error (a + credentialed engine on a clean CI is behaving correctly; the verdict + must not depend on the runtime's credential state), while + * any other :class:`ConfigError` -- an invalid supplied value, an + internally inconsistent declaration, a factory contract bug -- stays a + compliance FAILURE (skipping those would let a broken plugin read as + green-with-warning). + + :meth:`~standard_asr.runtime.config.BaseConfig.from_env` raises this + automatically when construction failed solely because required fields are + missing, so engines following the documented ``explicit > env > raise`` + pattern get the classification for free. An engine building its config + another way should raise this type itself for the missing-credential + state. + + Transport mapping: the reference server maps this state to HTTP **503** + (REST) / a ``service_unavailable`` frame (WS) with a stable generic + detail -- whether it surfaces at zero-arg engine construction or lazily + at transcription/session establishment (an engine deferring its + credential check past ``__init__``). An operator-side availability + state, never the caller's 422, and never the absent field names (those + are deployment detail, safe-logged for the operator only). """ pass @@ -185,6 +248,79 @@ class InvalidProviderParamError(StructuredError, ValueError): pass +class EngineContractError(StandardASRError): + """Raised when a constructed engine breaks the protocol contract. + + The runtime counterpart of a compliance failure, in two shapes: + + * **Runtime behavior**: a SYNCHRONOUS ``StandardASR`` member + (``transcribe`` / ``start_transcription`` / ``supports`` / + ``recommended_wire_format`` / ``prepare``) returned an awaitable (an + ``async def`` implementation, or a sync wrapper delegating to one) or + a value outside its protocol-pinned return type. Raised by + :func:`standard_asr.runtime.protocol_boundary.require_sync_result` at + the consumer call sites (CLI, reference server) so the defect is loud + at the boundary instead of surfacing as a confusing secondary + ``AttributeError`` (or a silent misreading) deep inside another + subsystem. + * **Declaration shape**: the engine DECLARED something the contract + forbids -- a ``prepare`` that is a coroutine function, non-callable, + or parameter-requiring; a malformed ``selectable_languages`` / + ``detectable_languages`` tag; a language axis without the IC.6 + ``default_language`` obligation. No caller-side value can fix these, + which is what separates them from :class:`ConfigError` (an invalid + configuration VALUE, fixable by whoever supplies the config). + + An **engine/plugin fault, never a caller mistake** -- deliberately NOT a + :class:`ValueError`: transports and the CLI map the ``ValueError`` family + to caller-fixable surfaces (HTTP 422 / usage exit 2), while this must + land on the engine-fault surfaces (scrubbed HTTP 500 / ``internal_error`` + frame / CLI exit 1). If you hit it as an application developer, report it + to the engine's author. Messages carry type names only, never the + offending value. + """ + + pass + + +class SubtitleRenderingError(StandardASRError, ValueError): + """Raised by ``to_srt`` / ``to_vtt`` when segments cannot render as visible cues. + + A subtitle cue is an interval claim -- "this text occurs at this time" -- + and it must survive the output's millisecond grid to be seen at all. A + segment is therefore UNRENDERABLE in either of two ways: it lacks a + measured span (``Segment.timestamp_status`` is ``"start_only"`` or + ``"unavailable"``), or its measured span quantizes to zero milliseconds + on the output grid (``end`` and ``start`` format to the same timestamp + -- players silently drop such cues, so emitting one silently hides the + text while the render call reports success). Neither dropping the text + nor fabricating timing is the renderer's to choose silently: under the + default policy (``on_unrenderable="error"``) it raises this error, and + the caller picks the loss explicitly (``"omit"`` drops the unrenderable + segments' text from the timed cues; ``"collapse"`` renders one + whole-text cue with no real timeline). Mixes in :class:`ValueError`: + the caller can fix the call -- choose a policy, or supply renderable + segments. + + Args: + message: Human-readable description of the rejection. + unrenderable: How many segments cannot render as visible cues, if + known. + total: How many segments the result carries, if known. + """ + + def __init__( + self, + message: str = "", + *, + unrenderable: int | None = None, + total: int | None = None, + ) -> None: + self.unrenderable = unrenderable + self.total = total + super().__init__(message) + + class StreamClosedError(StandardASRError): """Raised when audio is delivered to a streaming session that is closed. @@ -217,8 +353,9 @@ class InvalidSessionUseError(StandardASRError, ValueError): application used it. Catching :class:`StreamClosedError` here would lead an application to wrongly conclude the session terminated and rebuild it. Mixes in :class:`ValueError` (like :class:`ConfigError` / - :class:`InvalidProviderParamError`): it is a bad-call programming error, and - the server maps it to HTTP 422. + :class:`InvalidProviderParamError`): it is a bad-call programming error. + (It has no HTTP mapping: it fires only against an in-process session object, + and the server drives its own sessions correctly by construction.) """ pass @@ -257,7 +394,9 @@ class FactoryLoadError(DiscoveryError, ImportError): __all__ = [ "AudioProcessingError", "ConfigError", + "ConfigurationRequiredError", "DiscoveryError", + "EngineContractError", "EntrypointValidationError", "FFmpegNotFoundError", "FFprobeNotFoundError", @@ -268,6 +407,7 @@ class FactoryLoadError(DiscoveryError, ImportError): "StandardASRError", "StreamClosedError", "StructuredError", + "SubtitleRenderingError", "TranscriptionError", "UnsupportedFeatureError", ] diff --git a/src/standard_asr/contract/identifiers.py b/src/standard_asr/contract/identifiers.py index 723eff0a..a4dca794 100644 --- a/src/standard_asr/contract/identifiers.py +++ b/src/standard_asr/contract/identifiers.py @@ -26,7 +26,7 @@ _MODEL_NAME_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._+%:-]*\Z") -def _validate_engine_id(engine_id: str) -> None: +def validate_engine_id(engine_id: str) -> None: """Validate the *declared* form of an engine identifier. This checks the surface syntax only. Canonicalisation to the PEP 503 @@ -43,7 +43,6 @@ def _validate_engine_id(engine_id: str) -> None: Raises: EntrypointValidationError: If the engine identifier is invalid. """ - if "/" in engine_id: raise EntrypointValidationError(f"engine_id must not contain '/' (got {engine_id!r})") if not _ENGINE_ID_RE.match(engine_id): @@ -53,7 +52,7 @@ def _validate_engine_id(engine_id: str) -> None: ) -def _validate_model_name(model_name: str) -> None: +def validate_model_name(model_name: str) -> None: """Validate and log guidance for a model name. Args: @@ -65,7 +64,6 @@ def _validate_model_name(model_name: str) -> None: Raises: EntrypointValidationError: If the model name is invalid. """ - if model_name == "": logger.warning( "model_name is empty for a standard_asr.models entry point. " @@ -79,33 +77,3 @@ def _validate_model_name(model_name: str) -> None: "model_name contains unsupported characters. Allowed characters: " "letters, digits, '.', '_', '+', '%', ':', '-'." ) - - -def validate_engine_id(engine_id: str) -> None: - """Validate an engine identifier. - - Args: - engine_id: Engine identifier string. - - Returns: - None. - - Raises: - EntrypointValidationError: If the engine identifier is invalid. - """ - _validate_engine_id(engine_id) - - -def validate_model_name(model_name: str) -> None: - """Validate a model name. - - Args: - model_name: Model name string (may be empty for defaults). - - Returns: - None. - - Raises: - EntrypointValidationError: If the model name is invalid. - """ - _validate_model_name(model_name) diff --git a/src/standard_asr/contract/language.py b/src/standard_asr/contract/language.py index 1b3b3dcf..58045888 100644 --- a/src/standard_asr/contract/language.py +++ b/src/standard_asr/contract/language.py @@ -8,7 +8,9 @@ import re from collections.abc import Collection -from standard_asr.contract.results import Diagnostic +from standard_asr.contract.capabilities import ModeName +from standard_asr.contract.exceptions import UnsupportedFeatureError +from standard_asr.contract.results import Diagnostic, to_json_value #: Reserved token meaning "let the engine auto-detect the language". #: This is NOT a BCP-47 tag; it is a Standard ASR reserved word. @@ -24,6 +26,15 @@ DIAG_CANDIDATE_LANGUAGES_IGNORED = "candidate_languages_ignored" DIAG_CANDIDATE_LANGUAGE_DROPPED = "candidate_language_dropped" DIAG_CANDIDATE_LANGUAGES_TRUNCATED = "candidate_languages_truncated" +#: The remaining language-family codes are emitted by the engine base's +#: language-axis resolution (``EngineBase._resolve_language_axis``), which has +#: the engine context (``default_language``, selectable set) this module never +#: sees -- but their single source of truth still lives HERE, with the rest of +#: the language-family codes, so a consumer imports every language diagnostic +#: code from one contract module. +DIAG_LANGUAGE_FELL_BACK = "language_fell_back" +DIAG_LANGUAGE_NOT_SELECTABLE = "language_not_selectable" +DIAG_LANGUAGE_REFINEMENT_ACCEPTED = "language_refinement_accepted" _BCP47_RE = re.compile(r"^[A-Za-z]{2,8}(?:-[A-Za-z0-9]{1,8})*$") _PRIVATE_USE_RE = re.compile(r"^(?:x|i)(?:-[A-Za-z0-9]{1,8})+$", re.IGNORECASE) @@ -213,6 +224,7 @@ def effective_candidate_languages( detectable_languages: Collection[str], max_count: int | None, strict: bool, + mode: ModeName | None = None, ) -> tuple[list[str] | None, list[Diagnostic]]: """Resolve the candidate languages in effect for a request. @@ -226,6 +238,12 @@ def effective_candidate_languages( engine base passes its pre-canonicalized, ConfigError-checked set). max_count: Maximum candidate count, if constrained. strict: Whether to raise (vs truncate/drop + diagnostic) on violations. + mode: The mode being resolved (``"batch"`` / ``"streaming"``; the + signature enforces the closed set), or ``None`` when unknown + (direct callers outside the engine pipeline); carried on the + strict-mode + :class:`UnsupportedFeatureError` so the rejection reads like every + other strict gate rejection. Returns: A ``(candidates, diagnostics)`` pair; ``candidates`` is ``None`` when not @@ -237,12 +255,25 @@ def effective_candidate_languages( diagnostic-free on ordinary requests. Raises: - ValueError: Unconditionally (independent of ``strict``) if a candidate is - a malformed BCP-47 tag or the reserved ``"auto"`` token, or if a + ValueError: Independent of ``strict``, if a candidate is a malformed + BCP-47 tag or the reserved ``"auto"`` token -- once per-item + validation is reached: per spec §LANG R3 the unsupported-capability + short-circuit (step 3) runs FIRST, so when the engine/mode does not + support candidate languages the provided list is ignored with a + diagnostic and its items are never validated here. Direct callers + relying on unconditional malformed-item rejection get it from + :class:`~standard_asr.contract.params.RuntimeParams` construction, which + validates every candidate before any resolution runs (the engine + pipeline is always covered by that). Also raised if a ``detectable_languages`` entry is empty/whitespace (engine paths pre-validate this into a ``ConfigError``; the bare error is the - direct-call contract); or, in strict mode, on a non-detectable or - over-limit candidate list. + direct-call contract). These are caller code bugs, never policy. + UnsupportedFeatureError: In strict mode, on a valid-but-unreachable + candidate list -- a candidate not in ``detectable_languages`` or a + list over ``max_count``. This is the standard strict-gate rejection + type (spec, Runtime Parameters R2), so every transport maps it to + the same client-error verdict as any other strict rejection (the + server's 422) instead of an internal-error 500. """ diagnostics: list[Diagnostic] = [] if effective_lang != AUTO: @@ -271,7 +302,7 @@ def effective_candidate_languages( "support candidate languages." ), param="candidate_languages", - provided=list(chosen), + provided=to_json_value(list(chosen)), effective=None, ) ) @@ -323,7 +354,20 @@ def effective_candidate_languages( for norm in deduped: if norm not in detectable: if strict: - raise ValueError(f"Candidate language {norm!r} is not detectable.") + # Valid-but-unreachable: the standard strict-gate rejection + # (spec §RT R2), NOT a bare ValueError -- a bare ValueError is + # reserved for caller code bugs (malformed / 'auto' above) and + # would surface as an internal-error 500 through the server. + raise UnsupportedFeatureError( + f"Candidate language {norm!r} is not detectable by this engine.", + param="candidate_languages", + mode=mode, + hint=( + "Request only detectable_languages members, or use " + "best_effort to drop non-detectable candidates with a " + "diagnostic." + ), + ) diagnostics.append( Diagnostic( level="warning", @@ -339,7 +383,15 @@ def effective_candidate_languages( if max_count is not None and len(result) > max_count: if strict: - raise ValueError(f"candidate_languages has {len(result)} entries; max is {max_count}.") + raise UnsupportedFeatureError( + f"candidate_languages has {len(result)} entries; max is {max_count}.", + param="candidate_languages", + mode=mode, + hint=( + f"Pass at most {max_count} candidates, or use best_effort " + "to truncate with a diagnostic." + ), + ) kept = result[:max_count] dropped = result[max_count:] diagnostics.append( @@ -350,8 +402,8 @@ def effective_candidate_languages( f"Truncated candidate languages to {max_count}: kept {kept}, dropped {dropped}." ), param="candidate_languages", - provided=result, - effective=kept, + provided=to_json_value(result), + effective=to_json_value(kept), ) ) result = kept @@ -364,6 +416,9 @@ def effective_candidate_languages( "DIAG_CANDIDATE_LANGUAGES_IGNORED", "DIAG_CANDIDATE_LANGUAGES_TRUNCATED", "DIAG_CANDIDATE_LANGUAGE_DROPPED", + "DIAG_LANGUAGE_FELL_BACK", + "DIAG_LANGUAGE_NOT_SELECTABLE", + "DIAG_LANGUAGE_REFINEMENT_ACCEPTED", "effective_candidate_languages", "effective_language", "is_valid_bcp47", diff --git a/src/standard_asr/contract/params.py b/src/standard_asr/contract/params.py index 9bc96463..44970701 100644 --- a/src/standard_asr/contract/params.py +++ b/src/standard_asr/contract/params.py @@ -24,6 +24,7 @@ from typing import Final, Literal, get_args from pydantic import BaseModel, ConfigDict, Field, field_validator +from pydantic_core import PydanticCustomError from standard_asr.contract.capabilities import WordTimestampGranularityName from standard_asr.contract.language import AUTO, is_valid_bcp47, normalize_bcp47 @@ -275,10 +276,11 @@ def _reject_bare_provider_params(cls, value: ProviderParams | None) -> ProviderP concrete subclass. """ if value is not None and type(value) is ProviderParams: - raise ValueError( + raise PydanticCustomError( + "standard_asr_provider_params_base", "provider_params must be the engine's concrete ProviderParams " "subclass, not the bare ProviderParams base (or a mapping coerced " - "into it). Pass an instance of the engine's published params type." + "into it). Pass an instance of the engine's published params type.", ) return value @@ -307,9 +309,10 @@ def _validate_language_tag(value: str | None) -> str | None: # unauthenticated 422 body, where validation errors never echo the # request input), and a mis-pasted secret sent as `language` # would otherwise be reflected back. - raise ValueError( + raise PydanticCustomError( + "standard_asr_language_tag", "language tag is not a well-formed BCP-47 language tag " - "(e.g. 'en', 'en-US', 'zh-Hans') or 'auto'." + "(e.g. 'en', 'en-US', 'zh-Hans') or 'auto'.", ) return value @@ -358,14 +361,16 @@ def _validate_candidate_language_list(value: list[str] | None) -> list[str] | No # The raw value MUST NOT be embedded in the message (same reasoning as # the scalar `language` validator: it is echoed verbatim by the # server's unauthenticated 422 body and logs). - raise ValueError( + raise PydanticCustomError( + "standard_asr_candidate_language_tag", "candidate_languages contains an entry that is not a well-formed " - "BCP-47 language tag (e.g. 'en', 'en-US', 'zh-Hans')." + "BCP-47 language tag (e.g. 'en', 'en-US', 'zh-Hans').", ) if normalize_bcp47(tag) == AUTO: - raise ValueError( + raise PydanticCustomError( + "standard_asr_candidate_language_auto", "candidate_languages MUST NOT contain 'auto' (it is a directive, " - "not a candidate language)." + "not a candidate language).", ) return value @@ -402,9 +407,10 @@ def _validate_phrase_hints_list(value: list[str] | None) -> list[str] | None: if any(not term.strip() for term in value): # The raw values are not echoed (a phrase hint could carry sensitive # text); the message names the rule, not the offending entry. - raise ValueError( + raise PydanticCustomError( + "standard_asr_phrase_hint_blank", "phrase_hints must not contain empty or whitespace-only terms " - "(use [] to request no hints)." + "(use [] to request no hints).", ) return value diff --git a/src/standard_asr/contract/properties.py b/src/standard_asr/contract/properties.py index 741d2852..ac362ac9 100644 --- a/src/standard_asr/contract/properties.py +++ b/src/standard_asr/contract/properties.py @@ -186,7 +186,8 @@ class BaseProperties(BaseModel): """Base class for ASR engine static properties. Attributes: - engine_id: Engine identifier (PEP 503 normalized). + engine_id: Engine identifier (surface syntax checked here; PEP 503 + canonicalization happens at discovery). model_name: Model preset name within the engine. protocol_version: Standard ASR protocol version supported by the engine. accepted_input: Audio shapes the engine accepts (MUST be non-empty). @@ -229,7 +230,14 @@ class BaseProperties(BaseModel): protected_namespaces=(), ) - engine_id: str = Field(..., description="Engine identifier (PEP 503 normalized).") + engine_id: str = Field( + ..., + description=( + "Engine identifier. Validated here for surface syntax (lowercase " + "alphanumerics plus '._-'); canonicalized to its PEP 503 routing " + "identity at discovery." + ), + ) model_name: str = Field(..., description="Model preset name within the engine.") protocol_version: str = Field( ..., description="Standard ASR protocol version supported by the engine." diff --git a/src/standard_asr/contract/results.py b/src/standard_asr/contract/results.py index 0b8f3dd9..6a629e87 100644 --- a/src/standard_asr/contract/results.py +++ b/src/standard_asr/contract/results.py @@ -18,9 +18,146 @@ from __future__ import annotations -from typing import Any, Literal, Sequence, cast +from collections.abc import Mapping +from typing import Annotated, Literal, Sequence, cast + +from pydantic import ( + BaseModel, + BeforeValidator, + ConfigDict, + Field, + JsonValue, + field_validator, + model_validator, +) +from pydantic_core import PydanticCustomError + +#: Diagnostic code attached to a :class:`TranscriptionResult` whose ``segments`` +#: include entries without a full measured span (``start`` and/or ``end`` is +#: ``None`` -- i.e. :attr:`Segment.timestamp_status` is not ``"measured"``). +#: The per-segment truth IS the nullable ``start``/``end`` values themselves; +#: this result-level diagnostic is the aggregate disclosure ("N of M segments +#: lack a usable span") consumers can surface without walking the list. It +#: lives here -- not in the emitting reducer module -- because it describes a +#: property of the RESULT; same family-home rationale as the language codes +#: living in :mod:`standard_asr.contract.language`. +DIAG_SEGMENT_TIMESTAMPS_UNAVAILABLE = "segment_timestamps_unavailable" + + +def to_json_value(value: object) -> JsonValue: + """Project a Python value into the wire value space. + + Every wire-visible slot -- ``Diagnostic.provided`` / ``effective``, every + ``extra`` mapping -- is declared :data:`~pydantic.JsonValue`, because the + Python objects and the JSON documents are meant to be the same protocol + seen twice (G5.2). Declaring them ``Any`` admitted values with no JSON + representation at all, which then failed during the wire projection -- + after an endpoint had already committed to a response. + + Two things stand between an ordinary value and that declaration, and this + helper is where both are handled: + + * a **structured** value (a pydantic submodel such as a + ``DiarizationRequest``) has a JSON form but is not itself JSON, so it + is dumped; + * a **typed container** (``list[str]``, ``dict[str, int]``) IS JSON data, + but a type checker will not accept it where ``list[JsonValue]`` is + expected, because ``list`` is invariant. That is a static-analysis + artifact, not a real mismatch, so it is absorbed here once instead of + forcing a ``cast`` at every call site. + + Runtime validation is unaffected: the model still validates what it is + given, so a value that is genuinely not JSON is rejected loudly at + construction, naming the field. -from pydantic import BaseModel, ConfigDict, Field, field_validator, model_validator + Args: + value: The value to hand to a wire-visible slot. + + Returns: + The value's JSON projection. + """ + if isinstance(value, BaseModel): + return cast("JsonValue", value.model_dump(mode="json")) + return cast("JsonValue", value) + + +def require_json_string_keys(value: object) -> object: + """Reject any non-string object key anywhere in a wire-visible JSON value. + + JSON object keys are strings, full stop. pydantic's lax ``dict[str, ...]`` + validation, however, COERCES a ``bytes`` key to ``str`` -- at every nesting + level -- so a Python caller could construct ``extra={b"x": 1}`` (a key no + JSON document can express) and have it silently become ``"x"``. Worse, + ``{"x": 1, b"x": 2}`` COLLAPSES to a single ``"x"`` (last wins): two + distinct Python keys silently become one -- the exact silent wrong result + a wire-visible slot must never produce, and a break of the Python/JSON + two-layer isomorphism (a Python-only key rewriting wire-visible content). + + Applied as a ``mode="before"`` validator on every wire-visible JSON slot + (the ``extra`` mappings, :attr:`Diagnostic.provided` / :attr:`effective`) + BEFORE that coercion, this walks the whole structure and rejects any key + that is not an EXACT ``str``. A ``str`` SUBCLASS is refused too: a hostile + one can define ``__eq__`` / ``__hash__`` so two subclass keys that both + serialize to ``"x"`` do NOT collide in the input mapping, reintroducing + the very wire collision the exact-type check exists to deny. Object keys + from an actual wire document are always strings, so this only ever fires + for a Python caller reaching past the key domain the wire defines + (fail-loud, never a silent rewrite). + + The walk is iterative (an explicit stack, no recursion limit to blow on a + deep structure) and cycle-safe (an identity memo over the containers it + descends), so a self-referential mapping terminates instead of hanging the + validator. + + Args: + value: The raw field input (any JSON-shaped Python value). + + Returns: + ``value`` unchanged when every object key is an exact string. + + Raises: + PydanticCustomError: On the first non-string object key, with a fixed, + input-echo-free message (the offending key -- possibly ``bytes`` + carrying credential-shaped data -- is never echoed). + """ + stack: list[object] = [value] + seen: set[int] = set() + while stack: + node = stack.pop() + if isinstance(node, Mapping): + mapping = cast("Mapping[object, object]", node) + ident = id(mapping) + if ident in seen: + continue + seen.add(ident) + for key, item in mapping.items(): + if type(key) is not str: + raise PydanticCustomError( + "standard_asr_json_object_key", + "JSON object keys must be strings.", + ) + stack.append(item) + elif isinstance(node, (list, tuple)): + sequence = cast("Sequence[object]", node) + ident = id(sequence) + if ident in seen: + continue + seen.add(ident) + stack.extend(sequence) + return value + + +#: A wire-visible ``extra`` mapping: ``dict[str, JsonValue]`` whose object keys +#: are enforced to be exact strings at every depth +#: (:func:`require_json_string_keys`), so the Python and JSON layers share one +#: key domain. Shared by every ``extra`` field across the result and streaming +#: models so the rule lives in exactly one place. +WireExtra = Annotated[dict[str, JsonValue], BeforeValidator(require_json_string_keys)] + +#: A wire-visible free JSON slot (:attr:`Diagnostic.provided` / ``effective``): +#: any :data:`~pydantic.JsonValue`, with the same exact-string-key rule applied +#: to any object it contains at any depth. +WireJsonValue = Annotated[JsonValue, BeforeValidator(require_json_string_keys)] class Diagnostic(BaseModel): @@ -38,14 +175,17 @@ class Diagnostic(BaseModel): effective: The value that took effect, if relevant. """ - model_config = ConfigDict(frozen=True, extra="forbid") + # allow_inf_nan=False for the same reason JsonValue is used above: NaN + # and Infinity are Python floats but not JSON, so a diagnostic carrying + # one would construct and then fail the wire projection. + model_config = ConfigDict(frozen=True, extra="forbid", allow_inf_nan=False) level: Literal["info", "warning"] = Field(default="info") code: str = Field(..., description="Stable machine-readable diagnostic code.") message: str = Field(..., description="Human-readable explanation.") param: str | None = Field(default=None, description="Parameter concerned, if any.") - provided: Any | None = Field(default=None, description="Value provided, if any.") - effective: Any | None = Field(default=None, description="Value applied, if any.") + provided: WireJsonValue = Field(default=None, description="Value provided, if any.") + effective: WireJsonValue = Field(default=None, description="Value applied, if any.") def validate_speaker_label(value: str | None) -> str | None: @@ -77,15 +217,17 @@ def validate_speaker_label(value: str | None) -> str | None: # personal name and this error surfaces verbatim through server 422 bodies # and logs (the same redaction stance as the language-tag validator). if not value.strip(): - raise ValueError( + raise PydanticCustomError( + "standard_asr_speaker_label_blank", "speaker label must not be empty or whitespace-only (use None for " - "'no speaker attribution')." + "'no speaker attribution').", ) if value != value.strip(): - raise ValueError( + raise PydanticCustomError( + "standard_asr_speaker_label_whitespace", "speaker label must not have leading or trailing whitespace (two " "labels differing only in edge whitespace would read as two " - "different speakers)." + "different speakers).", ) return value @@ -134,7 +276,7 @@ class Word(BaseModel): default=None, description="Optional speaker label (non-empty, no edge whitespace)." ) channel: int | None = Field(default=None, ge=0, description="Optional channel index (>= 0).") - extra: dict[str, Any] = Field(default_factory=dict, description="Engine-specific extra data.") + extra: WireExtra = Field(default_factory=dict, description="Engine-specific extra data.") @field_validator("speaker") @classmethod @@ -172,7 +314,9 @@ def _check_span(self) -> Word: ValueError: If ``end`` is earlier than ``start``. """ if self.end < self.start: - raise ValueError(f"Word end ({self.end}) must be >= start ({self.start}).") + raise PydanticCustomError( + "standard_asr_span_inverted", "Word end must be >= start (the span runs backwards)." + ) return self @@ -182,18 +326,30 @@ class Segment(BaseModel): Note: ``start`` / ``end`` follow the same time frame as :class:`Word`: non-negative finite float seconds with origin at the first submitted - sample (``t=0``), ``end >= start`` (zero-duration allowed), and NaN / Inf - rejected. Within one channel segments are time-ordered; the - top-level :class:`TranscriptionResult.segments` are sorted by - ``(start, channel, speaker)`` (cross-channel spans may overlap). - ``speaker`` is the final tie-break for equal-``(start, channel)`` - overlapping segments (the single-channel multi-speaker case); ``None`` - sorts before any real label. + sample (``t=0``), ``end >= start`` (zero-duration allowed), and NaN / + Inf rejected -- OR ``None`` when the engine measured no such time. + ``None`` is data, not absence-of-field: the values themselves are the + single source of timing truth (there is no side-channel marker), and + the legal shapes are pinned by :attr:`timestamp_status`. An ``end`` + without a ``start`` is unrepresentable (rejected at construction): + no engine measures where speech stopped without knowing it started. + + Ordering: within one channel, MEASURED segments are time-ordered, and + the top-level :class:`TranscriptionResult.segments` with a ``start`` + are sorted by ``(start, channel, speaker)`` (cross-channel spans may + overlap; ``speaker`` is the final tie-break for equal-``(start, + channel)`` overlapping segments, ``None`` sorting before any real + label). A ``start=None`` segment has no time position: the producer + keeps the list in READING order instead (list order is the reading + order, and ``TranscriptionResult.text`` joins segment texts in list + order), so a single unmeasured segment never scrambles -- or forces + fabricated positions into -- an otherwise real timeline. Attributes: start: Segment start time in seconds (origin = first submitted sample; - non-negative, finite). - end: Segment end time in seconds (non-negative, finite, ``>= start``). + non-negative, finite), or ``None`` when unmeasured. + end: Segment end time in seconds (non-negative, finite, ``>= start``), + or ``None`` when unmeasured. Requires ``start``. text: Segment transcript text. words: Optional word-level details for this segment. speaker: Optional speaker label (authoritative diarization shape). @@ -202,17 +358,29 @@ class Segment(BaseModel): no_speech_prob: Optional no-speech probability. temperature: Optional decoding temperature. compression_ratio: Optional compression-ratio metric. - extra: Engine-specific extra data. + extra: Engine-specific extra data (engine-owned; the standard reserves + no keys here). Raises: ValueError: If field validation fails (incl. NaN/Inf, a negative time, - or ``end < start``). + ``end < start``, or ``end`` without ``start``). """ model_config = ConfigDict(frozen=True, extra="forbid", allow_inf_nan=False) - start: float = Field(..., ge=0.0, description="Segment start time in seconds (>= 0).") - end: float = Field(..., ge=0.0, description="Segment end time in seconds (>= 0, >= start).") + start: float | None = Field( + ..., + ge=0.0, + description="Segment start time in seconds (>= 0), or null when unmeasured.", + ) + end: float | None = Field( + ..., + ge=0.0, + description=( + "Segment end time in seconds (>= 0, >= start), or null when " + "unmeasured; requires a non-null start." + ), + ) text: str = Field(..., description="Segment transcript text.") words: list[Word] | None = Field( default=None, description="Word-level details for this segment." @@ -230,7 +398,7 @@ class Segment(BaseModel): compression_ratio: float | None = Field( default=None, description="Optional compression-ratio metric." ) - extra: dict[str, Any] = Field(default_factory=dict, description="Engine-specific extra data.") + extra: WireExtra = Field(default_factory=dict, description="Engine-specific extra data.") @field_validator("speaker") @classmethod @@ -256,23 +424,61 @@ def _check_speaker(cls, value: str | None) -> str | None: @model_validator(mode="after") def _check_span(self) -> Segment: - """Reject an inverted span (``end < start``) at construction. + """Pin the legal timing shapes at construction. - ``ge=0`` and ``allow_inf_nan=False`` already constrain each bound to a - non-negative finite value; this enforces the remaining invariant - that a span never runs backwards. Equal bounds (zero duration) are - allowed. + ``ge=0`` and ``allow_inf_nan=False`` already constrain each non-null + bound to a non-negative finite value; this enforces the remaining + shape invariants (see :attr:`timestamp_status`): + + * ``(float, float)`` with ``end >= start`` -- a measured span (equal + bounds, i.e. zero duration, allowed); + * ``(float, None)`` -- a measured onset with no span (start-only); + * ``(None, None)`` -- timing unavailable; + * ``(None, float)`` -- REJECTED: an end without a start is not a + representable measurement, and admitting it would force every + consumer to define semantics for a shape no engine produces. Returns: The validated segment. Raises: - ValueError: If ``end`` is earlier than ``start``. + ValueError: If ``end`` is set without ``start``, or is earlier + than ``start``. """ - if self.end < self.start: - raise ValueError(f"Segment end ({self.end}) must be >= start ({self.start}).") + if self.end is not None: + if self.start is None: + raise PydanticCustomError( + "standard_asr_span_end_without_start", + "Segment end is set without a start; a measured end requires a " + "measured start (legal shapes: measured / start-only / " + "unavailable).", + ) + if self.end < self.start: + raise PydanticCustomError( + "standard_asr_span_inverted", + "Segment end must be >= start (the span runs backwards).", + ) return self + @property + def timestamp_status(self) -> Literal["measured", "start_only", "unavailable"]: + """The segment's timing shape, derived from ``start``/``end``. + + Derived, not stored: the nullable values are the single source of + truth, so the status can never disagree with them (the previous + design stored fabricated ``0.0`` spans guarded by a mutable + side-channel marker -- two truths that could, and did, diverge). + + Returns: + ``"measured"`` (full span), ``"start_only"`` (real onset, no + usable span), or ``"unavailable"`` (no timing). + """ + if self.start is None: + return "unavailable" + if self.end is None: + return "start_only" + return "measured" + def synthesize_segment_speaker(words: Sequence[Word] | None) -> str | None: """Derive a segment-level speaker label from its words (the pinned synthesis rule). @@ -355,20 +561,22 @@ class TranscriptionResult(BaseModel): ``auto`` mode; ``None`` when not applicable. language_confidence: Detection confidence in ``[0, 1]``. duration: Audio duration in seconds, if known (non-negative, finite). - segments: Segments across all channels, if available. They - SHOULD be sorted by ``(start, channel, speaker)`` (monotonic within - a channel; ``speaker`` is the final tie-break, ``None`` sorting - first); this ordering is an **engine obligation**, neither enforced - at construction nor checked by the compliance suite (the streaming - reducer legitimately keeps arrival order for timestamp-less engines). - The SRT/VTT renderers' defensive re-sort is the only standard-layer - safety net. + segments: Segments across all channels, if available. Segments WITH a + ``start`` SHOULD be sorted by ``(start, channel, speaker)`` + (monotonic within a channel; ``speaker`` is the final tie-break, + ``None`` sorting first); a ``start=None`` segment has no time + position, so the list stays in READING order instead (list order + is the reading order; ``text`` joins segment texts in list + order). The ordering is an **engine obligation**, neither + enforced at construction nor checked by the compliance suite + (the streaming reducer keeps arrival order whenever any retained + segment lacks a ``start``). The SRT/VTT renderers' defensive + re-sort of measured cues is the only standard-layer safety net. words: Flattened word-level details, if available. channels: Per-channel results when channel separation was performed. Each ``channel`` index MUST be unique (one entry per channel), enforced at construction. diagnostics: Conversion / best_effort / degradation diagnostics. - metadata: Standardized engine-agnostic metadata. extra: Engine-specific / experimental data (incl. provider formats). Raises: @@ -401,10 +609,13 @@ class TranscriptionResult(BaseModel): default_factory=lambda: cast("list[Diagnostic]", []), description="Non-fatal diagnostics.", ) - metadata: dict[str, Any] = Field( - default_factory=dict, description="Standardized engine-agnostic metadata." - ) - extra: dict[str, Any] = Field( + # No `metadata` pocket: the spec removed blanket metadata from Properties + # and Capabilities ("no known use case, invites unstructured data, breaks + # machine readability"), and a result-side "standardized metadata" dict with + # no standardized keys, no writer, and no reader was the same disease. + # Standardized result data gets a real field (additive-minor); everything + # engine-specific goes in `extra`. + extra: WireExtra = Field( default_factory=dict, description="Engine-specific / experimental data." ) @@ -458,12 +669,14 @@ def _check_top_level_derivable_from_channels(self) -> TranscriptionResult: constant top-level ``segments``) would silently lose all per-channel detail. That shape is an engine bug, so the model refuses it. - The complementary ordering invariant (top-level ``segments`` sorted - by ``(start, channel, speaker)``, monotonic within a channel) is + The complementary ordering invariant (top-level ``segments`` with a + ``start`` sorted by ``(start, channel, speaker)``, monotonic within a + channel; ``start=None`` segments keeping reading order) is intentionally *not* enforced here: the streaming reducer (:class:`~standard_asr.runtime.streaming.StreamReducer`) legitimately preserves - arrival order for timestamp-less engines and sorts only by ``start`` - (no channel/speaker tie-break), so a strict ``(start, channel, + arrival order whenever any retained segment lacks a ``start`` and + sorts only by ``start`` otherwise (no channel/speaker tie-break), so + a strict ``(start, channel, speaker)`` construct-time check would reject valid reduced results. For the same reason the compliance suite does not check ordering either; ordering is an engine obligation, and the renderers' defensive re-sort @@ -481,33 +694,44 @@ def _check_top_level_derivable_from_channels(self) -> TranscriptionResult: seen: set[int] = set() for entry in self.channels: if entry.channel in seen: - raise ValueError( - f"channels contains duplicate entries for channel index " - f"{entry.channel}; the standard defines channels as one ChannelResult " - f"per channel, so each channel index MUST be unique (a duplicate " - f"makes the top-level merge ambiguous and silently drops data for " - f"consumers keyed by channel)." + raise PydanticCustomError( + "standard_asr_channel_duplicate", + "channels contains duplicate entries for one channel index; " + "the standard defines channels as one ChannelResult per " + "channel, so each channel index MUST be unique (a duplicate " + "makes the top-level merge ambiguous and silently drops data " + "for consumers keyed by channel).", ) seen.add(entry.channel) for name in ("segments", "words"): if getattr(self, name) is None and any( getattr(entry, name) is not None for entry in self.channels ): - raise ValueError( - f"channels entries carry {name} but the top-level {name} is None; " - f"the standard requires the top level to be derivable from channels " - f"(ignoring channels must be lossless). Populate the top-level " - f"{name} with the time-merged union of all channels' {name}." + raise PydanticCustomError( + "standard_asr_channel_top_level_missing", + "channels entries carry " + + name + + " but the top-level " + + name + + " is None; the standard requires the top level to be " + "derivable from channels (ignoring channels must be " + "lossless). Populate the top-level " + + name + + " with the time-merged union of all channels' " + + name + + ".", ) return self __all__ = [ "ChannelResult", + "DIAG_SEGMENT_TIMESTAMPS_UNAVAILABLE", "Diagnostic", "Segment", "TranscriptionResult", "Word", "synthesize_segment_speaker", + "to_json_value", "validate_speaker_label", ] diff --git a/src/standard_asr/engine.py b/src/standard_asr/engine.py index 36664b99..f52bf8f4 100644 --- a/src/standard_asr/engine.py +++ b/src/standard_asr/engine.py @@ -8,7 +8,10 @@ engine, pass audio, read a result), ``standard_asr.engine`` aggregates the types an *engine* author implements and declares against: -- the base class and protocol (:class:`EngineBase`, :class:`StandardASR`); +- the base class and protocol (:class:`EngineBase`, :class:`StandardASR`), + plus the standard session-establishment wire-format rule + (:func:`ensure_wire_format_supported`) for structural engines that implement + their own establishment guard; - the typed config surface (:class:`BaseConfig`, the applicability mixins, :func:`secret_field`); - static metadata (:class:`BaseProperties`, :class:`SampleRateRange`, @@ -17,7 +20,9 @@ ``*Cap`` / ``*Constraints`` node); - language resolution and download-policy helpers (:func:`effective_language`, :data:`AUTO`, :func:`resolve_download_root`); -- the result and streaming types an engine constructs and emits. +- the result and streaming types an engine constructs and emits, plus the + wire projection helper (:func:`to_json_value`) for values headed into a + wire-visible slot. Exceptions an engine raises live in :mod:`standard_asr.contract.exceptions` (and are also re-exported at the package top level). Compliance helpers for testing your plugin @@ -83,6 +88,7 @@ Segment, TranscriptionResult, Word, + to_json_value, ) from standard_asr.runtime.config import ( BaseConfig, @@ -95,7 +101,7 @@ ) from standard_asr.runtime.downloads import allow_downloads, resolve_download_root from standard_asr.runtime.gating import Mode -from standard_asr.runtime.interface import EngineBase, StandardASR +from standard_asr.runtime.interface import EngineBase, StandardASR, ensure_wire_format_supported from standard_asr.runtime.streaming import TranscriptionEvent, TranscriptionSession __all__ = [ @@ -148,9 +154,11 @@ "allow_downloads", "effective_candidate_languages", "effective_language", + "ensure_wire_format_supported", "env_var_name", "granularity_offers_all", "normalize_bcp47", "resolve_download_root", "secret_field", + "to_json_value", ] diff --git a/src/standard_asr/plugins/discovery.py b/src/standard_asr/plugins/discovery.py index 00ff9934..6d928368 100644 --- a/src/standard_asr/plugins/discovery.py +++ b/src/standard_asr/plugins/discovery.py @@ -43,7 +43,10 @@ from standard_asr.contract.exceptions import EntrypointValidationError, FactoryLoadError from standard_asr.contract.identifiers import validate_engine_id, validate_model_name from standard_asr.runtime.config import BaseConfig -from standard_asr.runtime.redaction import config_error_from_validation +from standard_asr.runtime.redaction import ( + config_error_from_validation, + safe_exception_summary, +) if TYPE_CHECKING: # pragma: no cover from standard_asr.runtime.interface import StandardASR @@ -214,7 +217,14 @@ def load_factory(self) -> ASRFactory: try: target = self.entry_point.load() except Exception as exc: # noqa: BLE001 - message = f"Failed to load entry point target for {self.model_id!r}: {exc!r}" + # load() executes arbitrary plugin module code, so exc can be + # ANYTHING -- including a pydantic ValidationError whose repr + # echoes the offending input. safe_exception_summary (not {exc!r}) + # keeps the interpolated text input-echo-free. + message = ( + f"Failed to load entry point target for {self.model_id!r}: " + f"{safe_exception_summary(exc)}" + ) raise FactoryLoadError(message) from exc if not callable(target): raise FactoryLoadError( @@ -608,9 +618,22 @@ def create(self, name: str, /, *args: Any, **kwargs: Any) -> "StandardASR": A construction-time pydantic ``ValidationError`` -- whether from the bare constructor or an engine's own validator -- is wrapped into ``ConfigError`` with the offending input scrubbed, so a - caller can ``except ConfigError`` uniformly (and the HTTP server - still maps it to 422). Use :meth:`config_schema` to discover what - configuration a model requires. + caller can ``except ConfigError`` uniformly. The wrap ASSERTS + the type's ownership contract: a factory's construction-time + ``ValidationError`` means the supplied configuration was + rejected (the documented bare-constructor pattern). An engine + whose factory lets a NON-config internal ``ValidationError`` + escape construction mis-asserts that contract -- in-band the + two are indistinguishable, so the compliance suite's zero-arg + construction check (``engine_construction_failed``) polices + it, not consumer-side guessing. (The reference + server maps construction-time config faults by fault + ownership: absent required config -> 503, anything else -> + scrubbed 500 -- never a caller-blaming 422, since its + construction is zero-arg; the CLI maps them to usage exit 2, + since its invoker owns the config and the env.) Use + :meth:`config_schema` to + discover what configuration a model requires. Example: >>> asr = registry.create("faster-whisper/large-v3", device="cuda") @@ -839,8 +862,11 @@ def _dist_identity(ep: EntryPoint) -> str: if __name__ == "__main__": # pragma: no cover - logging.basicConfig(level=logging.INFO) - registry = discover_models() - print("Discovered models:") - for name in registry.names(): - print(f" - {name}") + # The old print-based demo was removed (AGENTS: no print in library code); + # exiting silently here would read as "no models discovered" to someone + # debugging plugin visibility, so point at the real tool loudly instead. + raise SystemExit( + "This module has no CLI; run `standard-asr list` to inspect the " + "discovered models (add --strict-discovery to fail on invalid entry " + "points)." + ) diff --git a/src/standard_asr/renderers.py b/src/standard_asr/renderers.py index 4dd7d552..aaacb729 100644 --- a/src/standard_asr/renderers.py +++ b/src/standard_asr/renderers.py @@ -11,14 +11,38 @@ ``result.extra["provider_formats"]``. Speaker labels are rendered only on explicit opt-in (``include_speakers=True``): SRT prefixes the cue text with ``[