[HYPERSHELL-107] feat(web-console): distributed tracing for browser + BFF

[jsell-rh]

Summary

Adds OpenTelemetry distributed tracing to the HyperShell web console (browser + BFF), satisfying HYPERSHELL-27. Traces flow browser → BFF as a single W3C-propagated trace and export to a dev Jaeger instance in Kind.

Spans are derived from typed domain probes (Domain-Oriented Observability), not auto-instrumentation. OTel imports are confined to adapters/observability/** + composition/** per the architecture lint boundary.

What's included

• Browser trace sink (app/adapters/observability/gateway-trace-sink.ts): turns gateway workflow/dependency probes into OTel spans, adopts the probe's W3C trace id, exposes traceParentFor(correlationId). Flushes buffered spans on page hide (visibilitychange/pagehide) since BatchSpanProcessor does not auto-flush.
• W3C propagation: the fetch chokepoint (api.client.ts) injects traceparent/tracestate on /api/* calls via an injected provider (no OTel import in the API adapter).
• BFF tracing (bff/src/adapters/observability/otel-tracing.ts): emits a SERVER span per proxied request named by bounded route template, validates/replaces inbound trace context, relays OTLP/HTTP upstream. Config-driven — disabled when OTEL_EXPORTER_OTLP_ENDPOINT is unset.
• Dev Jaeger (deploy/kind/jaeger.yaml, scripts/kind/up.sh): opt-in via KIND_JAEGER=true make kind-up. Uses Jaeger v2 (jaegertracing/jaeger:2.20.0); OTLP receivers 4317 (gRPC, for the API server) and 4318 (HTTP, for browser/BFF) enabled by default.
• Redaction: span names and attributes come from bounded allowlists; secrets/high-cardinality values (correlation ids, tokens, query strings) never reach the collector. Covered by tests on both sides.
• Spec: specs/web-console/tracing.spec.md (WEB-TRACE-01..10) plus local-development spec updates.

Acceptance criteria

• OTel Web SDK initialized in the browser with a configurable collector endpoint
• BFF propagates incoming trace context headers upstream
• Gateway CRUD workflows produce browser spans
• End-to-end trace browser→BFF→API visible in Jaeger dev instance
• Fleet-navigation spans — out of scope here (no fleet probes exist yet)

The API-server hop is intentionally out of scope; it is tracked under HYPERSHELL-26 (PR #141).

Testing

• BFF: 56 tests pass; browser: 52 tests pass (Node 24, Vitest).
• Live E2E in Kind: drove a real browser through Keycloak login and the gateway list workflow with Playwright; confirmed cross-service traces land in Jaeger v2 — gateway.workflow.list (browser INTERNAL) → gateway.dependency.list (browser CLIENT) → /api/* (BFF SERVER, child of the browser client span). Verified genuine W3C propagation.

🤖 Generated with Claude Code

[jsell-rh]

Amber Analysis

The domain-probe-to-span architecture is well-factored and the current CI plus targeted browser/BFF tests are green. However, the main browser path ignores deployment sampling and several validation/reconciliation checks can report success while violating the new active tracing spec, so this is not ready to merge.

Major

1. components/web-console/app/composition/gateway-composition.ts:13 — Browser roots default to 100% sampling, making the BFF deployment ratio ineffective. Pass an allowlisted runtime ratio/enable flag into browser composition and test ratio zero. Confidence: High (99%).
2. components/web-console/app/adapters/observability/gateway-trace-sink.ts:285 — Asynchronous export/flush failures bypass the bounded delivery-health diagnostic. Route failures and overflow into that reporter and test rejecting/overflowing exporters. Confidence: High (97%).
3. components/web-console/bff/src/adapters/observability/otel-tracing.ts:38 — The hand-written W3C validators accept malformed tracestate and reject valid future-version traceparent. Use a conformant propagator/parser and broaden adversarial tests. Confidence: High (99%).
4. components/web-console/bff/src/adapters/observability/otel-tracing.ts:128 — OTLP validation checks only that resourceSpans is an array, and collector 4xx responses become 202 accepted. Validate the full bounded request and preserve rejection semantics. Confidence: High (100%).
5. deploy/kind/jaeger.yaml:5 — Jaeger is hard-coded to hypershell-system while the script addresses ${KIND_NAMESPACE}. Render all resources and references into the selected namespace. Confidence: High (100%).
6. scripts/kind/up.sh:261 — Turning KIND_JAEGER off does not remove Jaeger or unset the BFF endpoint on a reused cluster. Reconcile the disabled state and test the true-to-false transition. Confidence: High (99%).
7. components/web-console/e2e-live/tracing.live.spec.ts:54 — A dependency-only browser trace satisfies the workflow-root assertion. Require distinct workflow and dependency spans. Confidence: High (100%).

Minor

8. PR history — Nine Merge branch 'main'... commits are neither conventional nor atomic. Squash/rebase them before merge so the submitted history follows the project commit discipline. Confidence: High (100%).

Overall assessment: REQUEST_CHANGES

Findings Summary (ordered by severity, highest first):

1. [Major] Browser sampling ignores deployment configuration - Configuration (L13)
2. [Major] Export failures bypass bounded delivery health - Observability (L285)
3. [Major] W3C context validation is not conformant - Input Validation (L38)
4. [Major] Malformed nested OTLP is accepted - Input Validation (L128, L214)
5. [Major] Jaeger ignores the selected Kind namespace - Spec Consistency (L5, L52, L73)
6. [Major] Disabling KIND_JAEGER leaves tracing enabled - Reconciliation (L261)
7. [Major] E2E accepts a trace without a workflow root - Verification (L54)
8. [Minor] PR contains nine non-conventional merge commits - Commit Discipline (PR history)

Convention Checklist (omit conventions not applicable to the diff):

Convention	Result
No secrets in logs or responses	Pass
Input validated	Fail
SecurityContext on all pod specs	Pass
Configuration separate from code	Fail
Desired state reconciled	Fail
Observability failures bounded and surfaced	Fail
Conventional commit messages	Fail

[jsell-rh]

[Major] Apply deployment sampling to the browser trace root

sampleRatio is omitted here, so createGatewayTracing defaults every browser root to 1. The BFF setting does not repair that: its ParentBasedSampler always records a sampled remote parent, so OTEL_TRACES_SAMPLE_RATIO=0 still exports all browser spans and the BFF spans descended from them. This makes the WEB-TRACE-06 deployment control ineffective on the primary request path.

Fix: expose an allowlisted tracing-enabled/sample-ratio runtime value to the browser, construct a no-op sink when disabled, and pass the same validated ratio here. Add a test showing that ratio 0 emits an unsampled context and records neither browser nor BFF spans.

Confidence: High (99%).

[jsell-rh]

[Major] Surface exporter and flush failures through bounded delivery health

forceFlush() rejects when export fails or times out, but this lifecycle path discards that promise. Scheduled export failures and BatchSpanProcessor queue drops likewise stay inside the OTel diagnostic handler, so they never reach gatewayObservability.deliveryHealth() / recentDeliveryFailures(); a page-hide failure can also become an unhandled rejection. WEB-TRACE-09 and UI-OBS-05 explicitly require these failures to be best-effort and visible through the existing bounded diagnostic.

Fix: inject a bounded failure reporter into the tracing adapter/exporter, catch lifecycle flush rejections, expose queue-overflow/export failure health, and test a rejecting/overflowing exporter. If these custom hide handlers remain, disable the processor's built-in document-hide auto-flush to avoid duplicate flush paths.

Confidence: High (97%).

[jsell-rh]

[Major] Use a W3C-conformant context parser

This permissive expression accepts malformed tracestate values such as Vendor=1, foo==bar, duplicate keys, and trailing value whitespace, then forwards them across the trust boundary. The adjacent traceparent expression also accepts only version 00, so a valid future-version header with extension fields is unnecessarily discarded. That conflicts with WEB-TRACE-04's requirement to forward only valid state while continuing valid W3C context.

Fix: use the OpenTelemetry W3C propagator/TraceState parser (or implement the full W3C ABNF and versioning rules), attach the parsed state to the parent context, and add malformed-state plus future-version tests.

Confidence: High (99%).

[jsell-rh]

[Major] Reject malformed OTLP below the top-level array

This treats any array as valid OTLP, so payloads such as { "resourceSpans": [42] } or { "resourceSpans": ["secret"] } are forwarded. If the collector rejects one with a 4xx, line 214 maps that to unavailable, and the route still returns 202 accepted. The endpoint therefore does not satisfy WEB-TRACE-02 or the input-validation convention.

Fix: decode/validate an OTLP ExportTraceServiceRequest with bounded nested fields before forwarding, and distinguish collector 4xx rejection from 5xx/network unavailability. Add nested-malformation and collector-400 tests.

Confidence: High (100%).

[jsell-rh]

[Major] Deploy Jaeger into the selected Kind namespace

All three resources (and the HTTPRoute parent reference) are fixed to hypershell-system, while up.sh patches and waits in ${KIND_NAMESPACE} and points the BFF at jaeger.${KIND_NAMESPACE}.svc. With a non-default namespace, Jaeger lands in the default namespace, the configured service does not exist, and kind-up times out. This directly contradicts the new target-namespace scenario.

Fix: render/include this manifest through the namespace-aware Kind deployment path so resource namespaces and parent references use the selected namespace, then cover KIND_NAMESPACE != hypershell-system in a script test.

Confidence: High (100%).

[jsell-rh]

[Major] Reconcile the tracing-disabled state

There is only an enable branch. After one KIND_JAEGER=true make kind-up, rerunning without the flag leaves the standalone Jaeger workload/route and the kubectl set env value in place; the normal manifest apply does not remove that out-of-band environment entry. The new Tracing Disabled scenario therefore fails on reused clusters.

Fix: add an idempotent disabled branch that removes these exact optional resources and unsets OTEL_EXPORTER_OTLP_ENDPOINT (then waits for the web-console rollout), with a true-to-false transition test.

Confidence: High (99%).

[jsell-rh]

[Major] Require the browser workflow root in the e2e assertion

boundedSpanName matches both gateway.workflow.* and gateway.dependency.*, so a dependency span sets hasBrowserWorkflow = true. A trace whose workflow root was dropped or never exported but still contains the dependency and BFF spans passes this check—the exact decapitated trace WEB-TRACE-01/11 is meant to prevent.

Fix: use distinct workflow/dependency matchers and require at least the bounded workflow root (and the dependency span required by WEB-TRACE-11), then unit-test this predicate with a dependency-only trace.

Confidence: High (100%).

[jsell-rh]

Fifth re-review: availability finding addressed

The remaining Major finding is resolved in commit 541102d.

#	Sev	Finding	Fix
1	Major	BigInt(text) parsed the whole caller-controlled decimal before the range comparison (O(n^2)); a ~1M-digit timestamp under the 1 MiB body limit blocked the event loop ~125ms even though the value was rejected	withinBigIntRange now strips the sign and leading zeros, then rejects on significant-digit length (>20 digits is unconditionally out of range) before constructing a BigInt. The BigInt is built only from the bounded significant digits, so a body-sized leading-zero run never reaches it -- validation is now a linear scan. Added near-body-limit adversarial tests (all-9s and leading-zeros-then-out-of-range) asserting rejection without relay and completion under 100ms.
2	Minor	Nine Merge branch main commits remain	Unchanged: the project's squash-on-merge collapses the branch to a single conventional commit on main; interactive rebase is unavailable in this environment and force-pushing the shared PR branch is riskier than letting the configured merge strategy resolve it.

Verification

• BFF: 110 tests pass (otel-tracing 55/55, up from 53); the two new body-sized cases reject in a linear scan well under the 100ms ceiling (pre-fix was ~125ms)
• tsc (src + test) exit 0; eslint --max-warnings=0 clean; prettier clean
• No em dash / forbidden terms; lefthook pre-commit policy checks all pass

[jsell-rh]

The significant-digit guard fully resolves the availability finding: near-limit decimal inputs now reject in 2.9-5.5 ms while a million leading zeros followed by an in-range value still validates. Focused tests and typechecks are clean, and I found no new code findings; this is ready provided the PR uses squash merge as required by the project convention.

Minor note

1. PR history still contains nine Merge branch main commits, but no branch rewrite is needed if this PR is merged with GitHub squash merge. Keep squash selected so main receives one conventional commit. Confidence: High (100%).

Verification

• BFF tracing suite: 55/55 passed.
• BFF source and test typechecks passed.
• The two near-limit adversarial payloads reject in 2.9-5.5 ms, down from 125-129 ms.
• A body-sized leading-zero encoding of an in-range value still passes validation.
• git diff --check and bash -n scripts/kind/up.sh passed.
• The latest remote head is 541102d; completed CI checks are green, with remaining current-head checks still running at submission time.

Overall assessment: APPROVE (submitted as COMMENT because GitHub does not permit an approval review when this account owns the PR).

Findings Summary (ordered by severity, highest first):

1. [Minor] Use squash merge so the existing branch merge commits do not enter main - Commit Discipline (PR history)

Convention Checklist (omit conventions not applicable to the diff):

Convention	Result
No secrets in logs or responses	Pass
Input validation is bounded	Pass
Best-effort observability failures surfaced	Pass
Reconciliation errors propagated	Pass
Restricted SecurityContext on pod specs	Pass
Configuration separate from code	Pass
Image references consistent across manifests	Pass
Conventional commit on main	Pass (squash merge required)

[coderabbitai]

Important

• 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: Repository: openshift-online/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: bffa7c91-d639-4ef8-bdd5-0f1ddcf08d01

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}