feat: Go context-engineering proxy/library (tree-sitter, TOON, Starlark extractor, config, metrics, real integrations)#1
Merged
Conversation
added 23 commits
June 24, 2026 18:16
Re-home the winnow context-engineering capability as a Kagenti platform component in Go: canonical request model + surfaces (Anthropic/OpenAI/Gemini), the reduce pipeline (relevance, signals, collapse/dedup/cmdfilter/format, markers + rewind store), cache_control injection, a containment-verified extract stage, the engine (Transform/Expand, fail-open stages), the lab-cx proxy binary, OTel GenAI observability, eval-containers wiring, and the AuthBridge integration guide. Replaces the ADK Python template. Signed-off-by: Osher-Elhadad <Osher.Elhadad@ibm.com>
Replace the chars/4 heuristic in internal/tokens with the pure-Go tiktoken-go o200k_base BPE tokenizer (lazy-initialized, offline, no CGO). Falls back to chars/4 only if the tokenizer fails to initialize (fail-open). Accurate token counts surfaced a latent bug: the deterministic extraction projector emitted indented JSON via MarshalIndent, whose whitespace inflated the BPE token count above the original and tripped the never-inflate gate, silently dropping the extraction (TestExtractionEndToEnd). resultToText now emits compact JSON so the projection is a genuine reduction. Signed-off-by: Osher-Elhadad <Osher.Elhadad@ibm.com>
…pen) Signed-off-by: Osher-Elhadad <Osher.Elhadad@ibm.com>
… comments) Signed-off-by: Osher-Elhadad <Osher.Elhadad@ibm.com>
… bodies) Signed-off-by: Osher Elhadad <Osher.Elhadad@ibm.com>
…on-go @7ca0e27c4e8c) Signed-off-by: Osher-Elhadad <Osher.Elhadad@ibm.com>
…boxed, containment-verified) Signed-off-by: Osher-Elhadad <Osher.Elhadad@ibm.com>
…void truncation Signed-off-by: Osher-Elhadad <Osher.Elhadad@ibm.com>
…ct-provider flag Signed-off-by: Osher-Elhadad <Osher.Elhadad@ibm.com>
…dy cap + timeout Signed-off-by: Osher-Elhadad <Osher.Elhadad@ibm.com>
Signed-off-by: Osher-Elhadad <Osher.Elhadad@ibm.com>
The proxyhttp.Config gained MaxBodyBytes and UpstreamTimeout fields, but runProxy never set them, so the request-body cap and timeout hardening were inert in the shipped binary. Add --max-body-bytes (default 32 MiB) and --upstream-timeout (default 0s = no timeout) flags and pass them into the Config literal. The timeout defaults off because a non-zero value caps the whole request including streamed SSE responses, which would truncate long LLM streams. Signed-off-by: Osher-Elhadad <Osher.Elhadad@ibm.com>
The Anthropic cheap-model client only sent x-api-key, but the IBM LiteLLM
Anthropic-compatible gateway used for testing authenticates with a bearer
token. Add an AuthScheme field (""/"x-api-key" keep current behavior,
"bearer" sends Authorization: Bearer while still sending anthropic-version),
a --extract-auth proxy flag, and an ANTHROPIC_AUTH_TOKEN env fallback.
Signed-off-by: Osher-Elhadad <Osher.Elhadad@ibm.com>
Signed-off-by: Osher-Elhadad <Osher.Elhadad@ibm.com>
Add config.Load/LoadWithTransport (gopkg.in/yaml.v3, strict) folding a YAML file onto a preset-derived Settings, plus named component-selection lists so reducers, format encoders, extract strategies, and engine stages are enabled, disabled, and ordered purely by name — register a component by name in its package, then list it in the config; no core edit. - Settings gains ExtractMode/Reducers/Encoders/ExtractStrategies/Stages (empty = all built-in defaults; additive only). - reduce.bestEncoding consults a name-keyed encoder table with a config-given priority; route filters reducers by Reducer.Name and threads the encoder set. - extract.strategyOrder intersects with Cfg.AllowedStrategies. - engine builds its stage pipeline from a name->stage registry. - proxy: --config PATH overrides --preset; --extract-* still override transport. - configs/lab-cx.yaml: commented example incl. the one-line extension path. Assisted-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Osher-Elhadad <Osher.Elhadad@ibm.com>
…/stats
Add an Aggregator Emitter that accumulates process-wide reduction stats
(requests, tokens before/after/saved, ratio, cache/extract counts, stage
errors, cost_saved_usd, added-latency p50/p95) and serve them at GET /stats.
- observability: Aggregator with bounded latency ring + sorted-slice
percentiles, configurable CostRates (per-model + default fallback),
Snapshot/WriteJSON/Summary; add Event.LatencyMillis and a Tee emitter.
- proxyhttp: time the reduce path into Event.LatencyMillis; add a pluggable
Config.Aggregator served at GET /stats.
- cmd/proxy: wire a default cost table + Tee{slog,aggregator}; add a
'lab-cx stats --addr URL' subcommand.
Additive only; stdlib-only.
Assisted-By: Claude Opus 4.8
Signed-off-by: Osher-Elhadad <Osher.Elhadad@ibm.com>
Add cmd/labcx-bench, an offline (no-model) harness that proves each deterministic reduction component reduces tokens/bytes on REAL committed fixtures, with reversibility verified via engine.FindMarkers + engine.Expand. Fixtures (testdata/fixtures/, provenance in README): rtk command outputs (pytest/cargo verbatim from rtk unit tests; cargo build reconstructed from rtk's real Cargo.lock), winnow structured tool outputs, rtk glab/oc JSON, and a real rtk Python source for the skeletonizer. Results in docs/RESULTS-offline.md (verbatim harness output, o200k_base tokenizer): cmdfilter -94% on verbose command output, format/TOON -35%/-29% on structured outputs, skeleton -78% on a code read, full deterministic pipeline -93% aggregate across 10 fixtures - all reversible. Tests assert reduction > 0 AND recoverable original on committed fixtures. No new Go deps; go 1.25.0. Assisted-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Osher-Elhadad <Osher.Elhadad@ibm.com>
Add --extract to labcx-bench: enables cheap-model extraction wired to a live Anthropic-compatible gateway (claude-haiku-4-5 via cheapmodel.Anthropic, bearer auth) and measures the real LLM extractor on the structured_json + search_results fixtures. Records tokens before/after, %saved, the strategy chosen (code/single/rlm/deterministic), containment (verified reversible via FindMarkers+Expand), and wall-clock latency including the model round-trip. Offline behavior is unchanged: no --extract still runs the deterministic offline harness. docs/RESULTS-extract.md records the real run (Floor=200, 2026-06-24): extractor shrank 4/6 fixtures (-57% to -80%), declined honestly on 2 (one below floor, one containment decline); all spliced results contained. Assisted-By: Claude (Opus 4.8) Signed-off-by: Osher-Elhadad <Osher.Elhadad@ibm.com>
…-harm result) Routed claude-haiku-4-5 through lab-cx via --settings; traffic traversed the proxy (requests=2, ~41ms p50 added latency) and the task succeeded. On a small task with self-caching Claude Code, lab-cx correctly defers (cache stage stands down, 0 tokens saved, 0 harm) — documents where real Claude Code savings come from instead. Assisted-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Osher-Elhadad <Osher.Elhadad@ibm.com>
Merged compose renders winnow before the gateway (validated, no pull); image builds. Full SWE-bench run not executed (image pulls + provider wiring + spend) and no numbers fabricated; runbook documents exactly how to run it. Assisted-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Osher-Elhadad <Osher.Elhadad@ibm.com>
Rewrite README.md as the front door (install with CGO note, run/flags/endpoints, components, config-extension path, integrations, real measured results table). Add docs/RUNNING.md (per-component how-to-run + how-to-measure + named-registry config-extension guide) and docs/RESULTS.md (consolidated index of the existing real-results docs). Every command verified against current proxy flags and harness targets. No numbers invented — all pulled from RESULTS-offline.md / RESULTS-extract.md and the two integration docs. Assisted-By: Claude Opus 4.8 (1M context) Signed-off-by: Osher Elhadad <Osher.Elhadad@ibm.com>
…before) Earlier the exposed ratio was after/before, reading 1.0 at zero savings. Now it is the fraction of input removed (0 = none, higher = more), with a clearer percentage in the summary line. Doc note updated. Assisted-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Osher-Elhadad <Osher.Elhadad@ibm.com>
Bob (OpenAI-compatible, non-self-caching): 5/5 verifiable tasks correct incl. a file edit; cache-injection lever fires on every request. Claude Code (self-caching): with reduce_cached_prefix, -50.2% tokens on a large-file read (34,954 saved) with the answer correct (42 funcs, exact) — real savings, no harm; do-no-harm on a trivial task. New docs/integration/bob.md + claude-code.md savings section + README/RESULTS entries. Assisted-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Osher-Elhadad <Osher.Elhadad@ibm.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
lab-context-engineering— a Go context-engineering component for the Kagenti platformthat sits between an LLM agent and the model API and reduces token cost without changing
the agent or hurting results. One Go core runs as a standalone proxy (
lab-cx), animportable library (
engine/surfaces/config), or inside eval-containers. Fail-openand fully reversible (markers + content-addressed rewind store).
This branch builds the component, deepens every naive piece with established libraries,
and validates it with real runs.
What's included
Core (port of the winnow prototype, native Go):
extraction + code skeletonization, format re-encoders incl. TOON.
only if structurally contained in the original) with single/rlm/deterministic
fallbacks.
o200k_base), reversibility.Transform/Expand, fail-open per stage) + standalone proxy binary.Libraries adopted (replacing from-scratch/stubs):
tree-sitter/go-tree-sitter+go-sitter-forest,toon-format/toon-go,tiktoken-go,go.starlark.net. CGO enabled(CI +
distroless/base).Operability:
selected by name; add a component → register by name → list it in config.
/statsmetrics (tokens before/after/saved,reduction_ratio= saved/before,cost, cache, p50/p95 added latency) +
lab-cx stats.cmd/labcx-bench) over real rtk/winnow fixtures.Real measured results (2026-06-24)
skeleton −78%, TOON/format −29–35%), all reversible.
claude-haiku-4-5): −56%…−80% on structured outputs, allcontainment-verified; honest declines shown.
reduce_cached_prefix, answer correct (42 funcs, exact); do-no-harm on a trivial task.edit; cache-injection lever fires on every request.
a documented runbook (not executed — no fabricated numbers).
Testing
make lint test buildgreen —go vetclean, 12 test packages pass (race), CGO build +binary verified. Per-component unit tests + an external public-API integration test.
Docs
README (install/run/components/config/integrations/results),
docs/RUNNING.md,docs/RESULTS.md, and per-integration guides (docs/integration/{claude-code,bob,swe-bench,authbridge}.md).