Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
137 changes: 137 additions & 0 deletions conformance/domain_sdk_parity_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,137 @@
package conformance

// Drift guard: the domain rule the SDK ships IS the domain rule on the wire.
//
// The bare-domain shape now exists twice — as the protovalidate pattern on the
// contract's recipient-addressing fields (not on every field that happens to hold
// a domain; several deliberately carry no rule), and as the constant the three
// SDKs export so a client can refuse a bad value before sending it. The point of the
// second copy is that a value the SDK accepts is one the wire accepts, which
// holds only while the two are byte-identical. Nothing made them so; they were
// kept aligned by whoever remembered.
//
// That failed exactly once already, and quietly. When the shared constraint was
// first stamped on the fields, its port group was written as a real 1-65535
// range while the SDK's copy said "one to five digits". The SDK therefore
// accepted :0, :012, :0443, :00443, :65536 and :99999 — values the wire refuses —
// which inverts the property the client-side check exists to provide. Every
// suite in every language passed, because none of them compares the two. It was
// found by reading both regexes side by side.
//
// Division of labour with the neighbouring guard in domain_constraint_test.go:
// that one owns MEMBERSHIP — which fields carry the rule, that they all carry the
// SAME pattern, and a count that catches a field LOSING it. Note what that count
// does not do: a newly added field that never carried the rule leaves the total
// where it was, so neither guard notices. This one owns AGREEMENT with the SDK,
// and nothing else.
//
// The two halves of that agreement are not equally direct, and it is worth saying
// which is which. The max_len comparison is a live descriptor read: nothing else
// in the package pins that number, so a rule whose bound moved is caught here and
// only here. The pattern comparison is one step removed — findDomainFields selects
// fields BY equality to the neighbour's sharedDomainPattern literal, so what this
// file really compares is the SDK's copy against that literal. The chain still
// closes, because the neighbour pins the descriptor to the same literal and an
// edit to it that no field carries empties the set and trips the fatal below. But
// a failure here names the conformance package's literal, not a byte read from the
// descriptor in this function.
//
// The proto is authoritative. On failure the wire has not moved to meet the SDK;
// the SDK must be brought to the proto's bytes and its vectors regenerated.
//
// The expected values are READ from the SDK's committed vectors rather than
// restated here, for the reason ver_field_contract_test.go gives at length: this
// package cannot import sdk/go — nothing in conformance depends on sdk/, by
// design, since it is the guard tier BELOW the SDKs. A committed file generated
// from the real Go constants, and already replayed by the Python and TypeScript
// parity suites, is a data read exactly like the corpus and the doc scans.

import (
"encoding/json"
"errors"
"os"
"sync"
"testing"

"buf.build/gen/go/bufbuild/protovalidate/protocolbuffers/go/buf/validate"
)

// audienceVectors is the committed cross-language oracle for the bare-domain
// rule, emitted from the real Go constants by the sdk/go/helpers vector
// generator. Read as data (see the file header on why this is not an import).
const audienceVectors = "../sdk/go/helpers/testdata/audience-vectors.json"

// sdkDomainRule is the shape the SDK ships, as recorded in the vectors.
type sdkDomainRule struct {
Pattern string `json:"bare_domain_pattern"`
MaxLen uint64 `json:"bare_domain_max_len"`
}

// sdkRule reads the SDK's copy of the rule from the committed vectors. Errors are
// fatal rather than skipped: a missing file or a missing key means this guard has
// lost its anchor, and passing silently would be worse than failing.
var sdkRule = sync.OnceValues(func() (sdkDomainRule, error) {
b, err := os.ReadFile(audienceVectors)
if err != nil {
return sdkDomainRule{}, err
}
var doc sdkDomainRule
if err := json.Unmarshal(b, &doc); err != nil {
return sdkDomainRule{}, err
}
if doc.Pattern == "" || doc.MaxLen == 0 {
return sdkDomainRule{}, errNoSDKDomainRule
}
return doc, nil
})

var errNoSDKDomainRule = errors.New(
audienceVectors + " carries no bare_domain_pattern / bare_domain_max_len — this guard reads the SDK's copy of the domain rule from there")

// stringRules returns the field's string rules, reaching through the repeated
// wrapper when the field is a list. The singular/repeated split is the same one
// the membership guard makes; going through domainField.repeated keeps the two
// reading the descriptor the same way.
func stringRules(t *testing.T, df domainField) *validate.StringRules {
t.Helper()
rules, has := fieldRules(df.fd)
if !has {
t.Fatalf("%s.%s: protovalidate rules vanished between the two guards", df.msg.Name(), df.fd.Name())
}
if df.repeated {
return rules.GetRepeated().GetItems().GetString()
}
return rules.GetString()
}

// TestSDKBareDomainRuleMatchesTheWire is the gate the two copies of the rule were
// missing. A failure means a value one side accepts the other refuses.
func TestSDKBareDomainRuleMatchesTheWire(t *testing.T) {
want, err := sdkRule()
if err != nil {
t.Fatalf("read the SDK's domain rule: %v", err)
}
fields := findDomainFields(t)
// Guard the guard: an empty set would make every assertion below vacuous, and
// the failure would look like a pass.
if len(fields) == 0 {
t.Fatal("no field carries the shared domain constraint — this guard would assert nothing")
}
for _, df := range fields {
name := string(df.msg.Name()) + "." + string(df.fd.Name())
s := stringRules(t, df)
if got := s.GetPattern(); got != want.Pattern {
t.Errorf("%s: the wire and the SDK disagree about the domain pattern.\n"+
" contract (authoritative, via the shared literal): %s\n"+
" SDK (%s): %s\n"+
"Bring the SDK to the contract's bytes and regenerate its vectors.",
name, got, audienceVectors, want.Pattern)
}
if got := s.GetMaxLen(); got != want.MaxLen {
t.Errorf("%s: the wire and the SDK disagree about the domain length bound.\n"+
" wire (authoritative): max_len = %d\n"+
" SDK (%s): %d",
name, got, audienceVectors, want.MaxLen)
}
}
}
43 changes: 43 additions & 0 deletions docs/design-history.md
Original file line number Diff line number Diff line change
Expand Up @@ -217,6 +217,49 @@ top-level-versus-items mismatch to police. That last exemption is why
string: an empty value is unroutable, and the swap-protection its signature is
supposed to provide is vacuous when the signed bytes carry no recipient at all.

## The audience match is exact; the endpoint rule is not

Two host comparisons sit a few sections apart in this document and answer
deliberately different questions, so they compare differently and neither should
be relaxed into the other.

The endpoint rule admits a **set**: a manifest may advertise its service on the
host that served the document or on a subdomain of that host. The question there
is which addresses one Exchange can be reached at, and an operator fronting
`exchange.example` from `api.exchange.example` is the ordinary case.

The audience check admits **one** value. An Exchange has exactly one identity —
the domain it stamps into the offers it issues — and a subdomain of it is a
different party, not another address for the same one. `eu.exchange.example` does
not name `exchange.example`. Widening this to the endpoint rule's shape would let
anyone who controls a subdomain claim to be the parent, which is precisely what
the check exists to refuse. The configured value is the identity domain and never
the host the process listens on; when those differ, taking the listening host
refuses every correctly addressed request.

What survives normalization is only spelling. Case folds, and a port of 443
written out is the same as leaving it off, because a schemeless domain reads as
https throughout the SDK. Port 80 does not fold — it is not that scheme's
default — and a padded `:0443` is refused for its shape before any comparison,
since the wire rule's port group is a real 1-65535 range rather than a digit
count.

That rule is carried in two places on purpose — the protovalidate pattern on the
contract's recipient-addressing fields, and an exported constant in each SDK so a
client can refuse a bad value before sending it. It is *not* on every field that
happens to hold a domain; several carry no rule, and whether they should is a
separate question from this one. Exporting the constant breaks the precedent set by
the money pattern, which mirrors a wire rule and stays private in all three
languages. The difference is who needs it: money's is an internal detail of
formatting a decimal, while this one is a protocol constant an implementer writes
their own validator against — and in TypeScript the export is structural, since the
parity test imports the constant to assert it against the shared vectors. It is not
exported for the conformance guard's sake; that guard reads the vectors file, and
the emitter that writes it sits inside the same package as the constant either way.
Its length bound of 260 is the `max_len` the contract carries on every one of those
fields; it is not derived from DNS's 253, and trying to reconstruct it from that
plus a port will not land on the same number.

## CoMP as an extension; attestations instead of quality scores

Two earlier couplings were undone. The core protocol no longer imports IAB CoMP;
Expand Down
9 changes: 8 additions & 1 deletion docs/sdk-parity-matrix.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@

Go is the oracle (`sdk/go/{helpers,resolvers,core,connect,connectserver}`); Python and TS mirror it. This document is **generated** from the same two artifacts CI already enforces against the code, so it cannot drift from the real surface — a mismatch fails the API-surface gate or the corpus-completeness gate before it can reach this file.

**At a glance:** 77 symbols at cross-language parity · 14 documented divergences · 142 Go-idiomatic exclusions · 23 conformance corpora, each tri-replayed.
**At a glance:** 82 symbols at cross-language parity · 14 documented divergences · 143 Go-idiomatic exclusions · 24 conformance corpora, each tri-replayed.

Layering (L1 pure trust core vs L2 I/O resolvers), the SSRF transport-wiring invariant, and naming conventions are recorded in [`design-history.md`](./design-history.md).

Expand All @@ -28,10 +28,13 @@ Legend: a name = the public face in that language · `—` = intentionally none
| `AgentKeyHeader` | `AGENT_KEY_HEADER` | `AGENT_KEY_HEADER` |
| `AppendSignature` | `append_signature` | `appendSignature` |
| `ApplyScopes` | `apply_scopes` | `applyScopes` |
| `AudienceVerdict` | `AudienceVerdict` | `AudienceVerdict` |
| `BareDomainPattern` | `BARE_DOMAIN_PATTERN` | `bareDomainPattern` |
| `CanonicalAcceptanceBytes` | `jcs_acceptance_payload` | `acceptancePayload` |
| `CanonicalOfferBytes` | `canonical_offer_payload` | `canonicalOfferPayload` |
| `CanonicalizeMoney` | `canonicalize_money` | `canonicalizeMoney` |
| `CatalogRejectionDetail` | `catalog_rejection_detail` | `catalogRejectionDetail` |
| `CheckAudience` | `check_audience` | `checkAudience` |
| `ConnectProtocolVersion` | `ConnectProtocolVersion` | `ConnectProtocolVersion` |
| `ConnectProtocolVersionHeader` | `ConnectProtocolVersionHeader` | `ConnectProtocolVersionHeader` |
| `ContentDigest` | `content_digest` | `contentDigest` |
Expand All @@ -42,7 +45,9 @@ Legend: a name = the public face in that language · `—` = intentionally none
| `ErrUnknownKey` | `UnknownKeyError` | `UnknownKey` |
| `FormatMoney` | `format_money` | `formatMoney` |
| `HashURL` | `hash_url` | `hashUrl` |
| `IsBareDomain` | `is_bare_domain` | `isBareDomain` |
| `KeyResolver` | `KeyResolver` | `RequestKeyResolver` |
| `MaxBareDomainLen` | `MAX_BARE_DOMAIN_LEN` | `maxBareDomainLen` |
| `NewIdempotencyKey` | `generate_idempotency_key` | `generateIdempotencyKey` |
| `NormalizeScopes` | `normalize_scopes` | `normalizeScopes` |
| `OfferSignatureAlgorithm` | `OFFER_SIGNATURE_ALGORITHM` | `OFFER_SIGNATURE_ALGORITHM` |
Expand Down Expand Up @@ -227,6 +232,7 @@ Go constructs (functional-option builders, `errors.Is` sentinels, value types, c
| `helpers.ComponentParam` | Go value type for an RFC 9421 covered-component parameter; py/ts model components inline. |
| `helpers.CoveredComponent` | Go value type for an RFC 9421 covered component; py/ts model components inline. |
| `helpers.ErrAcceptanceSignatureInvalid` | Go errors.Is sentinel; py/ts express verification failures via typed failure unions / exception classes, not per-reason named sentinels. |
| `helpers.ErrAudienceIdentity` | Go errors.Is sentinel for an unusable configured Exchange identity; py/ts raise/throw instead of exporting sentinels. |
| `helpers.ErrBrokenSignatureChain` | Go errors.Is sentinel; py/ts express verification failures via typed failure unions / exception classes, not per-reason named sentinels. |
| `helpers.ErrDigestMismatch` | Go errors.Is sentinel; py/ts express verification failures via typed failure unions / exception classes, not per-reason named sentinels. |
| `helpers.ErrEmptyIdempotencyKey` | Go errors.Is sentinel; py/ts express verification failures via typed failure unions / exception classes, not per-reason named sentinels. |
Expand Down Expand Up @@ -306,6 +312,7 @@ Go emits each `*-vectors.json` oracle; Python and TS replay it. The completeness
| Corpus | go | python | ts |
|---|---|---|---|
| `helpers/testdata/acceptance-vectors.json` | ✅ | ✅ | ✅ |
| `helpers/testdata/audience-vectors.json` | ✅ | ✅ | ✅ |
| `helpers/testdata/error-detail-vectors.json` | ✅ | ✅ | ✅ |
| `helpers/testdata/hashurl-vectors.json` | ✅ | ✅ | ✅ |
| `helpers/testdata/idempotency-validate-vectors.json` | ✅ | ✅ | ✅ |
Expand Down
Binary file modified gen/descriptor.binpb
Binary file not shown.
4 changes: 2 additions & 2 deletions gen/go/ramp/v1/ramp.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion gen/python/wire/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -1822,7 +1822,7 @@ class Offer(WireModel):
max_length=260,
) = Field(
...,
description='REQUIRED. Bare host of the Exchange that issued this offer (e.g.\n "exchange.example" or "exchange.example:8081"), in the form "Request\n recipient" defines in the file header. This is the execute-routing target:\n the agent, or a relaying Broker, sends the ExecuteTransaction call for this\n offer to this Exchange, and a Broker relaying a mixed batch groups the items\n by this value. Because it is an ordinary Offer field it falls inside the\n signed bytes (see `signature` below — the signature covers every field\n except `signature` / `signature_algorithm`), so an intermediary cannot\n redirect the execute call to a different Exchange without invalidating the\n offer, and it is what retires the X-RAMP-Exchange-Endpoint transport header.\n It is also the audience statement of an ExecuteTransaction, which is why\n TransactionRequest carries no top-level `exchange`: on receipt, an Exchange\n MUST reject the request unless EVERY item\'s offer.exchange names one of its\n own domains. Presence is enforced because an empty value is unroutable — a\n relaying Broker has nothing to group or dial on, and the swap-protection\n above is vacuous when the signed bytes carry no recipient at all.',
description='REQUIRED. Bare host of the Exchange that issued this offer (e.g.\n "exchange.example" or "exchange.example:8081"), in the form "Request\n recipient" defines in the file header. This is the execute-routing target:\n the agent, or a relaying Broker, sends the ExecuteTransaction call for this\n offer to this Exchange, and a Broker relaying a mixed batch groups the items\n by this value. Because it is an ordinary Offer field it falls inside the\n signed bytes (see `signature` below — the signature covers every field\n except `signature` / `signature_algorithm`), so an intermediary cannot\n redirect the execute call to a different Exchange without invalidating the\n offer, and it is what retires the X-RAMP-Exchange-Endpoint transport header.\n It is also the audience statement of an ExecuteTransaction, which is why\n TransactionRequest carries no top-level `exchange`: on receipt, an Exchange\n MUST reject the request unless EVERY item\'s offer.exchange names its own\n domain. Presence is enforced because an empty value is unroutable — a\n relaying Broker has nothing to group or dial on, and the swap-protection\n above is vacuous when the signed bytes carry no recipient at all.',
)
expires_at: AwareDatetime | None = Field(
None, description='When this offer expires (ISO 8601).'
Expand Down
Loading
Loading