Skip to content

Stop the browser tracing its own trace-export requests - #2598

Merged
rmunn merged 3 commits into
developfrom
claude/otel-trace-loop-lexbox-9c7178
Aug 25, 2026
Merged

Stop the browser tracing its own trace-export requests#2598
rmunn merged 3 commits into
developfrom
claude/otel-trace-loop-lexbox-9c7178

Conversation

@hahn-kev-bot

@hahn-kev-bot hahn-kev-bot commented Aug 25, 2026

Copy link
Copy Markdown
Collaborator

🤖 AI summary

The bug

When opening LexBox, the browser traced its own OTel trace-export requests. The exporter POSTs trace batches to /v1/traces; that request became a span, which got exported, producing another export request → a feedback loop that grew unbounded.

Root cause

The dependency bump in #2544 took @opentelemetry/otlp-exporter-base from 0.208 → 0.221, which removed the browser sendBeacon and XHR transports and now sends all exports via fetch. Previously the exporter used the untraced sendBeacon transport, so exports were a blind spot; now they go through fetch, which our tracing wraps.

The OTLP fetch-transport does try to avoid this loop — it sends via globalThis.fetch.__original when present, to bypass any fetch wrapper. But that escape hatch never survives in our setup: ZoneContextManager pulls in zone.js, which re-wraps window.fetch after app.html runs, and the zone wrapper (the fetch the exporter actually reads) carries no __original. So the export flows through our whole proxy chain and hits both span-creating layers: @opentelemetry/instrumentation-fetch and our own traceFetch in hooks.client.ts.

Fix

Zone's wrapper creates no span; only traceFetch and instrumentation-fetch do. So make both skip the collector URL, sharing one pattern (TRACE_EXPORT_URL_PATTERN). No export span is created regardless of how fetch is wrapped — no dependence on the __original convention or on wrap ordering.

  • frontend/src/lib/otel/otel.shared.ts — add TRACE_EXPORT_URL_PATTERN and an early return fetch() in traceFetch for that URL (no span).
  • frontend/src/lib/otel/otel.client.tsignoreUrls: [TRACE_EXPORT_URL_PATTERN] on instrumentation-fetch and instrumentation-xml-http-request.
  • otel/collector-config.yaml — a filter/drop-trace-export processor drops any span targeting /v1/traces as an ops-level safety net (also deployable on its own as an immediate hotfix, and covers older/cached clients).

Verified

Ran against a live build: a burst of app fetches drained into a couple of batch exports, then /v1/traces requests flatlined during idle (the old loop climbed ~1/sec indefinitely). Normal app requests are still traced; only the collector POSTs are skipped. No new console errors.

Follow-up (not in this PR)

The custom OTLPTraceExporterBrowserWithXhrRetry wrapper is now largely dead weight (its beacon-retry path is unreachable once all transports resolve to fetch). Handed off as a separate cleanup task.

Test plan

  • Burst of app fetches → /v1/traces requests flatline during idle (no runaway growth).
  • Normal app fetch/XHR requests are still traced.
  • Verify traces still arrive in Honeycomb.

The OTel dep bump (#2544) took @opentelemetry/otlp-exporter-base from
0.208 to 0.217+, which removed the browser sendBeacon and XHR transports
and routes all browser exports through fetch. The trace exporter's POST
to /v1/traces is now caught by instrumentation-fetch, producing a span
that gets exported, producing another span -- a feedback loop.

Tell the fetch and XHR web instrumentations to ignore the /v1/traces
endpoint so export requests are never turned into spans. Also drop any
such spans at the collector as defense-in-depth for older/cached clients.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@github-actions github-actions Bot added the 📦 Lexbox issues related to any server side code, fw-headless included label Aug 25, 2026
@coderabbitai

coderabbitai Bot commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: a18d6ea5-7117-402b-bb37-2e2fd419c140

📥 Commits

Reviewing files that changed from the base of the PR and between c8116c5 and e0c23d4.

📒 Files selected for processing (2)
  • frontend/src/app.html
  • otel/collector-config.yaml

Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review.


📝 Walkthrough

Walkthrough

The browser now preserves the native fetch reference for OpenTelemetry export requests. The collector drops spans for /v1/traces requests and runs this filter in the traces pipeline.

Changes

Trace loop prevention

Layer / File(s) Summary
Native fetch escape hatch
frontend/src/app.html
The inline script stores the native fetch reference in window.fetch.__original for OpenTelemetry OTLP export requests.
Collector export filtering
otel/collector-config.yaml
The collector drops export-request spans that match /v1/traces in http.url or url.full. The traces pipeline runs this filter before later processors.

Estimated code review effort: 2 (Simple) | ~10 minutes

Merge Risk: ⚪ Minimal · up to e0c23

This PR prevents trace-export requests from generating additional tracing spans and adds a collector safety filter; no actionable merge-blocking risk remains beyond normal checks and review.

Poem

A rabbit found a fetch by moonlit light

It kept the native path in sight
The traces hopped, then stopped their loop
The collector sorted every group
“Clean exports!” cheered the rabbit bright

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0…
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly and concisely describes the main change: preventing browser tracing of its own trace-export requests.
Description check ✅ Passed The description directly explains the runaway trace-export loop, its cause, the proposed fixes, and the test plan. It is related to the changeset.
Full details: Docstring Coverage

Explanation

No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0 files. (2 skipped: 2 unsupported.)

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch claude/otel-trace-loop-lexbox-9c7178

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

hahn-kev and others added 2 commits August 25, 2026 12:01
The ignoreUrls approach only silenced @opentelemetry/instrumentation-fetch,
but our own traceFetch proxy (layered on top of the instrumentation in
hooks.client.ts) still opened a span for every export POST to /v1/traces.

The OTLP browser fetch-transport already guards against this loop: it sends
via globalThis.fetch.__original when present, bypassing any fetch wrapper.
Our app.html proxy replaces the fetch reference the exporter reads and drops
that property, so the guard never fired. Re-expose the native fetch as
__original on the proxy arrow; the exporter then bypasses both traceFetch
and the instrumentation, so export requests are never traced.

Reverts the ignoreUrls config, which addressed the wrong layer.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The __original escape hatch the OTLP fetch-transport looks for never
survives in our setup: ZoneContextManager pulls in zone.js, which
re-wraps window.fetch after app.html runs, and the zone wrapper (the
fetch the exporter actually reads) carries no __original. So the export
request flows through our whole proxy chain and gets traced -> exported
-> traced.

Zone's wrapper creates no span; only traceFetch and instrumentation-fetch
do. Make both skip the collector URL: an early return in traceFetch and
ignoreUrls on the fetch/xhr instrumentations, sharing one pattern. No
export span is created regardless of how fetch is wrapped.

Reverts the app.html __original attempt.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

@rmunn rmunn left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@rmunn
rmunn merged commit 49f6fb9 into develop Aug 25, 2026
18 checks passed
@rmunn
rmunn deleted the claude/otel-trace-loop-lexbox-9c7178 branch August 25, 2026 05:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

📦 Lexbox issues related to any server side code, fw-headless included

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants