barber

Query-aware context trimming for LLM requests.

Your context could use a trim. 33% off the top.

The 30-second pitch

Retrieved context is mostly irrelevant to any single question. Your RAG stack fetches ten paragraphs, the question needs two, and you pay for all ten on every request.

barber embeds the chunks and the question, keeps what matters, and drops the rest. Nothing is rewritten or summarized: chunks survive verbatim or vanish. No model calls at trim time, no required dependencies, deterministic output.

In the published benchmark that locked barber's defaults, that meant 31.8 to 34.1% of context tokens gone with answer quality within noise of full context (numbers below).

Install

pip install barber-llm

The PyPI name is barber-llm; the import name is barber. Zero required dependencies. Extras when you want them:

Extra	Pulls in	Gives you
barber-llm[semantic]	sentence-transformers	semantic scoring, paraphrase-safe
barber-llm[tokens]	tiktoken	exact token counts in TrimResult
barber-llm[eval]	datasets, openai, tiktoken	the barber-eval benchmark harness
barber-llm[langchain]	langchain-core	a BaseDocumentCompressor for ContextualCompressionRetriever
barber-llm[llamaindex]	llama-index-core	a BaseNodePostprocessor for a query engine

JavaScript is a first-class citizen too. The npm package is a zero-dependency port of the same algorithm with the same defaults, kept decision-identical to this package by golden fixtures regenerated from the Python implementation and replayed in CI (js/):

npm install barber-llm

or one-shot from the shell:

npx barber-llm --keep 0.6 < messages.json > trimmed.json

The two packages version independently, so the numbers in the two badges above do not line up and are not meant to: PyPI runs ahead because sweep() and the eval harness are Python-only. trim() is the same algorithm with the same defaults and the same decisions on both, which is what the golden fixtures gate in CI.

Quickstart

Zero-dependency lexical mode, runnable as pasted:

from barber import trim

context = "\n\n".join(f"Passage {i}: facts about topic {i}." for i in range(40))  # your RAG block
messages = [{"role": "user", "content": context},
            {"role": "user", "content": "What do the passages say about topic 7?"}]

result = trim(messages)   # keep=0.6, lexical fallback, zero deps
print(result.tokens_saved, result.chunks_dropped, result.changed)
# result.messages is the same conversation, fewer tokens; send it to your LLM

tokens_saved is signed. Each dropped run costs about 20 tokens of marker, so on blocks with many small, scattered chunks the markers can cost more than the drops save. A negative number is barber telling you this shape is not worth trimming, so skip it or raise keep.

Semantic mode, the configuration the benchmark shipped with:

from barber import trim, embedders

embed = embedders.sentence_transformers(
    "llm-semantic-router/mmbert-embed-32k-2d-matryoshka",
    trust_remote_code=True,   # this checkpoint ships code; see below
)
result = trim(messages, keep=0.6, embedder=embed)

trust_remote_code runs Python from the HF repo, at load time, in your process. It defaults to False, so passing a model_name through from config cannot execute someone else's code without you writing that flag. mmbert needs it; BAAI/bge-small-en-v1.5 (which tied with it in the benchmark) does not.

The lexical fallback is deterministic and dependency-free, but it matches words, not meaning. Its tokenizer is Unicode-aware, so accented Latin, Cyrillic, Greek, Hebrew, Arabic and Hangul all score normally; unspaced CJK falls back to character unigrams, which is real signal but a weak one, and min_message_chars (800) is a Latin-sized gate you will want lower for Chinese or Japanese. What it never does in any language is match a paraphrase. For production traffic use the semantic embedder above (32K context window) or BAAI/bge-small-en-v1.5 as the lightweight alternative; the two tied on quality in the published runs. There is also embedders.endpoint(base_url, model, api_key) for any OpenAI compatible /v1/embeddings server (vLLM, TEI), which needs the openai package.

Multi-turn pipelines use the transform form. Pass a shared cache and a block is decided once, then replayed byte-identically on every later turn, so your provider prompt cache stays warm:

from barber import make_transform, Cache

cache = Cache()                          # bounded LRU, safe for a long-running process
name, fn = make_transform(embedder=embed, keep=0.6, cache=cache)
messages, changed = fn(messages)         # call this every turn

A plain dict works too, but it never evicts: one entry per distinct context block for the life of the process. Cache bounds it and is thread-safe, so a threaded server can share one. Two things to size against:

• An entry holds one whole trimmed block, so the bound is a count, not a byte budget. At maxsize=4096 with large RAG blocks that is hundreds of megabytes per process. Lower it if your blocks are big.
• Keep maxsize above your live-conversation count. Evicting a block means the next turn decides it again against the then-current question, which is the prefix churn the cache exists to prevent.

When barber does nothing

trim() returns changed=False and leaves your messages untouched unless some message meets every one of these:

Requirement	Why
Role is user, tool, or function	System and assistant messages are never touched.
Not the latest user message	That message is the question, and the question is never trimmed.
content is a string, or text/tool_result parts	Other parts (images, tool_use inputs) pass through untouched.
At least 800 characters	Anything shorter is not retrieved context.
At least 4 chunks	Below that there is nothing to choose between.

The common surprise is the single-message shape. This is a no-op, because the only message present is the question:

messages = [{"role": "user", "content": f"Context:\n{docs}\n\nQuestion: {q}"}]

Give the context its own message and barber has something to work with:

messages = [{"role": "user", "content": docs},
            {"role": "user", "content": q}]

Most APIs accept consecutive user messages. If yours does not, put the context in a tool or function message, which is where retrieved context usually arrives anyway.

LangChain and LlamaIndex

A retriever post-processor gets a query and N candidate documents and returns a subset. That is the same job the benchmark below measured — query plus candidate passages, keep what answers the question — so it is the one framework slot where those numbers describe the work being done rather than something adjacent to it.

pip install "barber-llm[langchain]"

from langchain_classic.retrievers import ContextualCompressionRetriever
from barber.integrations.langchain import BarberDocumentCompressor

retriever = ContextualCompressionRetriever(
    base_compressor=BarberDocumentCompressor(keep=0.6),
    base_retriever=vectorstore.as_retriever(search_kwargs={"k": 20}),
)
docs = retriever.invoke("What is the refund policy?")

BarberDocumentCompressor subclasses langchain_core.documents.compressor.BaseDocumentCompressor, so it drops into any slot that takes one, including DocumentCompressorPipeline. acompress_documents is inherited: the base class runs the sync path in an executor, which is what you want for CPU work with no I/O to await. ContextualCompressionRetriever lives in langchain_classic.retrievers on LangChain 1.x and langchain.retrievers on 0.x; the compressor itself only imports langchain_core, which both share (tested from langchain-core 0.3.0).

The incumbent in that slot, LLMChainExtractor, invokes its chain once per retrieved document, sequentially — 20 LLM calls for a k=20 retrieval, on every query. barber makes none: scoring is one embedding pass, or pure lexical math with no dependencies at all.

LlamaIndex is the same shape:

pip install "barber-llm[llamaindex]"

from barber.integrations.llama_index import BarberNodePostprocessor

engine = index.as_query_engine(
    similarity_top_k=20,
    node_postprocessors=[BarberNodePostprocessor(keep=0.6)],
)

BarberNodePostprocessor subclasses BaseNodePostprocessor and inherits apostprocess_nodes, which the base class runs on a thread. That method arrived in llama-index-core 0.13, which is where the extra floors; the sync path alone works from 0.11.

Both take keep, an embedder, and a SelectionConfig, and both retrieve wide-then-cut: k=3 leaves nothing to select between, and barber will say so by returning all three. Four things to know before wiring either one in:

• It filters, it does not extract. A kept document comes back as the object that went in — same text, same metadata, same order, no relevance score written. Nothing is rewritten, so citations and source links downstream still resolve. The flip side is that a long document that is half relevant comes back whole; trimming inside a document is trim() on a message list.
• barber's gates still apply. Under 4 documents, or under 800 characters across all of them, there is nothing worth cutting and the list comes back untouched (the full table).
• keep is a budget, not a quota. Pinning and lead/tail keep documents above it; the relevance floor cuts below it. On a candidate set where only one or two documents share vocabulary with the query — the normal case for the zero-dependency lexical fallback, which matches words and not meaning — the floor is what decides and keep barely shows up. Pass the same encoder you retrieve with, via embedders.sentence_transformers() or embedders.endpoint(), and the decision becomes semantic.
• The published numbers are not a benchmark of these adapters. The table below is HotpotQA passages, LLM-judged, run through trim(). The adapters map a document set onto that same call — one document, one chunk, the block shape the harness builds — but nobody has run the eval through ContextualCompressionRetriever on your corpus. barber-eval is committed if you want your own number.

Coding agents

A coding-agent transcript is not a RAG block, and two of its habits defeat plain trim():

• Its context lives in content parts. A tool result is a tool_result block, not a string, and that is where the big text is. trim() now walks into text and tool_result parts (tool_use inputs and images are passed through untouched).
• Its output has no blank lines. A file read, a grep, an ls, a diff — the paragraph splitter finds one chunk and selection declines the block. barber now falls back to line structure when, and only when, nothing else found any: a line-numbered read is chunked on the blank lines of the underlying file, a diff on its hunks, flat output on line windows. A JSON body is never line-chunked, because half a JSON object does not parse.

Selection still only guesses relevance. In an agent transcript, most of the dead weight does not need a guess at all — it is content the transcript itself proves is no longer true:

from barber import trim
from barber.agent import sweep

result = sweep(trim(messages).messages)

sweep() drops three things, each provable from the transcript, each marked with the file path so the agent can just read it again:

Dropped	Because
A file body written before the same file was fully rewritten	It is not the file any more. A later Edit does not count: the file still holds that body.
A Read result for a file modified since	It is not the file any more.
A byte-identical repeat of an earlier tool result	It says nothing new.

Blocks are emptied, never removed: a request rejects a tool_use with no matching tool_result, so the structure survives even when the content does not.

sweep() rewrites history, and trim() deliberately does not. The freeze-on-first-sight cache exists to keep the prefix byte-stable so the provider prompt cache stays warm; sweep() edits messages in the middle of the conversation, so everything after the earliest edit is re-primed once. That is a good trade at a compaction point with many turns left to amortize it, and a bad one on turn three — which is why it is a call you make, not something trim() does behind your back.

Tell it how many turns are left and it will do that arithmetic for you:

sweep(messages, remaining_turns=50)

An edit at position P costs re-writing every token after P at 1.25x instead of reading it back at 0.1x, and buys those removed tokens never being re-read again. With T tokens after the cut and S of them removed, it only pays once

S/T > 1.15 / (1.15 + 0.1 * remaining_turns)

— 53% of the tail at 10 turns, 19% at 50, 10% at 100. Editing later costs less and saves less, so sweep() scores every candidate cut point and keeps only the edits at or after the best one; everything before it is left alone and stays cached. When nothing clears the bar it changes nothing and says so. The 15.7% of tool tokens the sweep can find needs roughly 62 remaining turns to pay for itself if you claim all of it from the front of the transcript.

Left unset, remaining_turns edits everything it finds — right only when the cache is already cold.

Measured on six real Claude Code sessions (1.16M tokens of transcript), in quota units that charge cache reads at 0.1x and cache writes at 1.25x, and paying the sweep's re-prime cost in full:

	Tokens removed	Quota saved
trim() before this (content-parts guard)	0.0%	0.0%
trim()	11.5%	14.9%
trim() + sweep()	19.5%	23.7%

That is 1.31x the turns per unit of quota. Your mileage varies with what your session does: the spread across those six sessions was 1.18x to 1.68x, and the sessions that gain most are the ones that rewrite the same files repeatedly. sweep() is Python-only for now; the JS port has the trim() half.

Claude Code plugin

trim() and sweep() act on a conversation you already own. Inside Claude Code you do not own it, so the same work happens in a PostToolUse hook, and this repo installs as a plugin that registers it:

claude plugin marketplace add NadirRouter/barber
claude plugin install barber@barber

There is no pip install step and no path to edit. barber's hook import chain (barber.core plus the lexical embedder) touches nothing outside the standard library, and a plugin install is a clone of this repo with barber/ already in it, so the hook resolves the package next to itself. An installed barber-llm still wins if you have one.

The hook trims Read, Grep, Glob, Bash, BashOutput, NotebookRead, WebFetch and WebSearch results against the live question plus that call's own arguments — the grep pattern or file path is usually the sharper signal. It leaves Edit, Write, TodoWrite and Task alone, along with anything under 800 characters, any non-string result, and any body starting { or [, because half a JSON object does not parse. Measured on 12 real sessions (3,995 tool results, 958K tokens of tool output) the policy fires 464 times and removes 21.5% of tool-output tokens, versus 12.1% for token-optimizer's first-read structure map, and survivors stay byte-exact where a structure map discards function bodies irrecoverably.

Three env vars, no settings file involved:

BARBER_HOOK_KEEP=0.8	fraction of chunks kept
BARBER_HOOK_MIN_CHARS=800	leave anything smaller alone
BARBER_HOOK_DISABLE=1	off for this shell

This is experimental and the benchmark does not cover it. Those numbers were judged on RAG passages answering a question, not on tool output an agent is about to act on, and dropping the one grep hit the agent needed is a different and worse failure than dropping a passage a reader didn't need. The hook therefore defaults to keep=0.8 where the library defaults to 0.6.

That margin is close to free, which a second replay over 30 sessions from 30 different projects (3,970 tool results, 1.57M tokens of tool output) puts a number on:

keep	fires	of eligible tokens	of all tool output
0.6	476	18.3%	10.4%
0.8	467	18.1%	10.3%

Two tenths of a point, because keep is a budget cap and the relative floor is what does the cutting; the cap rarely binds. Note also the two denominators: only 56.6% of tool-output tokens clear the eligibility gates at all, so the same removal is "18.1% of eligible" or "10.3% of everything the tools emitted" depending on which you quote. Read the four caveats at the bottom of contrib/claude_code_hook.py before running it on real work. Off again with claude plugin uninstall barber@barber.

It is also the one thing here that is free: the rewrite happens before the output enters context, so nothing cached is disturbed (next section).

MCP server

The hook above only works in Claude Code, because it needs a point that rewrites tool output before the model sees it and Claude Code is currently the only agent that honours one. MCP is the portable shape. After pip install barber-llm:

{"mcpServers": {"barber": {"command": "barber-mcp"}}}

One tool, trim(text, query, keep?), which drops the parts of a block that are irrelevant to the question and returns the rest byte-for-byte. It is the benchmarked path rather than the experimental one: a question and some candidate passages is exactly the shape the benchmark judged.

The server speaks JSON-RPC over stdio directly instead of taking the mcp SDK, which would pull pydantic, anyio and httpx into a package whose whole premise is having no dependencies. pip install barber-llm still installs barber and nothing else.

Unlike the hook, this one is pull rather than push: the agent decides when to call it, so it trims what you point it at instead of everything that goes past.

The prompt cache

Providers bill a cached token at a fraction of a fresh one — Anthropic reads at 0.1x and writes at 1.25x — and the cache is a prefix match: change one byte at position N and everything after N is billed as new. That single fact decides what each part of barber is allowed to do.

	Disturbs the cached prefix	What it costs
trim() with a shared Cache	no	nothing; decisions are frozen and replayed byte-identically
trim() with no cache	yes, every turn	each block is re-decided against the current question, so the prefix churns
sweep()	yes, from the earliest edit	one re-prime of everything after it — priced when you pass remaining_turns
contrib/claude_code_hook.py	no	nothing; it rewrites tool output before it ever enters context

Three rules follow from the table:

1. Pass a shared Cache in any multi-turn loop. Without one, trim() still removes tokens but re-decides every block each turn, and a prefix that changes shape each turn is a prefix nobody caches. This is the single cheapest thing on this page.
2. Prefer trimming at admission over trimming in hindsight. An agent re-sends its whole conversation every turn, so a token removed before it enters context is saved once per remaining turn, while the same token removed afterwards costs a re-prime to collect. Both are worth doing; only one is free.
3. Treat sweep() as a spend, not a saving, until you have counted the turns. It is the only call here that rewrites history, which is why it is explicit and why remaining_turns exists.

One caveat worth stating plainly: a cached token is cheap, not free, and it still occupies the context window. Trimming the stable prefix of a long session is a context-quality decision, not a cost decision — do not expect the quota numbers above to follow.

The benchmark

barber's defaults were locked by a paired A/B on HotpotQA (distractor config): answer each question with full context and with trimmed context, then have a blind judge grade both answers against the gold reference.

	Medium (~6K tok)	Large (~14K tok)
Tokens saved	31.8%	34.1%
Answer-paragraph retention	100%	100%
Full-context accuracy	97.2%	94.8%
Trimmed-context accuracy	96.0%	95.9%

About 350 judged pairs, MiniMax M3 as the blind judge, answers graded against gold references, keep=0.6. On large contexts trimmed beat full: selection removes the distractors models trip over.

Two things to know before you rely on that table. The judge and the generator are the same model by default, which is the classic setup for self-preferencing; the protocol blunts it (blind, order-randomized, graded against a gold answer rather than on preference) but does not eliminate it, so re-run with GEN_MODEL pointed at a different model before treating these as model-independent. And these are our numbers on our run — the harness is committed and seeded, the per-pair records are not, so reproducing the table means spending your own credit. barber-eval --out results.jsonl writes the per-pair records if you want to keep or publish yours.

Full write-up: the benchmark post. Full protocol: docs/methodology.md. Reproduce it:

pip install "barber-llm[eval]" && barber-eval --n 200 --keep 0.6 --size large

How it works

1. Query. The latest user message is the question. It is never trimmed.

2. Candidate blocks. Large user, tool, and function messages (800+ chars, 4+ chunks). System and assistant messages are never candidates.

3. Chunk. Split each block on blank lines, --- rules, headings, and [n] citation markers, the boundaries RAG concatenations already have. Falls back to sentences for a single wall of prose.

4. Score. Embed every chunk plus the question, cosine each chunk against the question, keep the top keep fraction.

5. Guards. Pinned chunks bypass the budget entirely, the first and last chunk always survive, and a relevance floor drops pure noise even when the budget would keep it. Details below.

6. Marker. Each dropped run collapses into one line, so the model knows the cut happened and doesn't go looking for missing text:

   [… 4 passage(s) omitted as not relevant to this question — the remaining context is sufficient …]
   

   The assertive wording is deliberate and benchmark-locked: it won our marker ablation. The working theory is that a neutral "lower-relevance passages omitted" invites the model to hedge, while this one tells it to proceed. Re-run the ablation yourself with barber-eval --marker neutral.

Decisions are memoized on the block hash alone, never the query. The first turn to see a block decides it; every later turn replays the decision byte-identically. Your history prefix stays stable and your provider prompt cache keeps hitting.

What barber does not do

• No summarization. Chunks survive verbatim or vanish. A summary is a rewrite, and rewrites can silently invent or lose facts.
• No letter tricks. Dropping vowels, truncating words, gzip-then-base64: in the same benchmark, every character-level scheme cost MORE tokens, not fewer. Letter removal measured 1.34x to 1.69x the tokens; classic compression 3.2x to 3.7x. Tokenizers are already compressors; fighting them backfires. The numbers.
• No history compaction. Providers do that natively now, and re-writing old turns busts their prompt caches. barber targets fresh retrieved context only.
• No model calls at trim time. Scoring is an embedding pass (or pure lexical math). Nothing is sent anywhere unless you opt into the endpoint() embedder.

Guards, or why your answers survive

The failure mode of context pruning is silent: drop the one chunk that held the answer and nothing errors, the model just answers worse. barber ships with every guard on:

• Deontic and PII pinning. Chunks with constraint language ("must", "never", "do not", "don't", "shall not", "prohibited", "required", "only if") or sensitive-data markers (PII, HIPAA, PCI, SSN, password, secret, API key) are never dropped. These patterns are English. Nothing else in barber is: the tokenizer is Unicode-aware and scores any script. But a French block saying "ne doit jamais" is not pinned unless you say so:

  import re
  from barber import SelectionConfig, trim
  
  fr = SelectionConfig(pin_patterns=[
      re.compile(r"\b(doit|doivent|ne\s+doit\s+pas|interdit|obligatoire|jamais)\b", re.I),
      re.compile(r"\b(?:RGPD|données\s+personnelles|mot\s+de\s+passe|clé\s+API)\b", re.I),
  ])
  trim(messages, keep=0.6, cfg=fr)

  The JS port takes the same list as config.pinPatterns.

• Rare-query-entity pinning. A query term that appears in only one or two chunks of a block is a strong "this chunk answers the question" signal. Those chunks are never dropped, which is what protects multi-hop questions.

• Lead and tail keep. The first and last chunk of every block survive: headers, conclusions, and the lost-in-the-middle mitigation.

• Relevance floor (0.35). A chunk scoring far below the block's top chunk is noise even if the budget has room for it.

• Never touched at all: the latest user message, system messages, assistant messages, non-string content, any message under 800 chars, any block under 4 chunks. See when barber does nothing.

In the benchmark these guards held answer-paragraph retention at 100% in both published sizes (table above).

Evaluate it on your data

Published numbers are an existence proof, not a guarantee for your traffic. The harness that produced them ships in the package:

pip install "barber-llm[eval]"
export JUDGE_API_KEY=sk-...     # any OpenAI compatible judge; MiniMax by default

barber-eval --baseline --n 25   # sanity-check generator + judge first
barber-eval --n 200 --keep 0.6 --size large

The number to gate on is the regression rate: how often trimming turned a right answer wrong. Hold it under 1 to 2%, then take the most aggressive keep that stays under your bar. Tokens saved is the payoff, not the gate. The exact judge prompt ships in barber/eval/JUDGE_PROMPT.md, including a reference-free variant for chat data without gold answers. Judge and generator are swappable via JUDGE_* and GEN_* env vars (details).

Credits

• The MiniMax team: MiniMax M3 was the generator and the blind judge for the entire eval, about 2,000 model calls on well under $20 of credit.
• The vLLM Semantic Router team: their mmbert-embed-32k-2d-matryoshka is the recommended embedder, picked for its 32K window.

---

Want this tier-aware, quality-monitored in production, and billed only on verified savings? That's Nadir.