Sahil170595 · Sahil170595 · Jul 2, 2026 · Jun 25, 2026 · Jun 25, 2026 · Jun 25, 2026
diff --git a/.gitignore b/.gitignore
@@ -143,6 +143,7 @@ local.env
 .claude/
 experiments/TR118/models/
 PRODUCT_SPEC.md
+VISION.md
 experiments/TR118/results/**/*.jsonl
 experiments/TR116/all_metrics_extracted.json
 experiments/TR116/results/**/metrics.json

diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -7,6 +7,86 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [0.6.0] - 2026-06-25
+
+State-of-the-art serving model: the planner now reflects how LLM inference
+actually behaves (per the literature - PagedAttention, continuous batching,
+prefill/decode disaggregation, goodput/Pareto), not replicas-of-single-stream.
+
+### Added
+- **Continuous-batching throughput.** vLLM/TGI are modelled with per-GPU
+  continuous batching instead of single-stream replicas: aggregate decode
+  throughput rises with batch size up to the KV-cache cap, anchored to the
+  measured/roofline single-stream rate so it stays quant-correct. One GPU can now
+  replace several Ollama replicas (e.g. a 7B at 3 req/s on a 4090: Ollama 5 GPUs
+  vs vLLM 1 GPU at batch 8). `Candidate.effective_batch`.
+- **Prefill/decode split.** Separate **TTFT** (prefill, compute-bound, from GPU
+  FP16 TFLOPS) and **TPOT** (decode, bandwidth-bound); end-to-end p95 now includes
+  prefill. `GPUSpec.fp16_tflops` for all GPUs; `plan --prompt-tokens`.
+- **KV-cache-bound max concurrency** per GPU (`max_concurrent_seqs`), the real
+  concurrency limiter for batched backends.
+- **Pareto frontier** (`plan --pareto`): the non-dominated cost/latency/quality
+  trade-off menu (tags cheapest / fastest / best-quality), not a single pick.
+- **Variance-aware queueing** (`plan --workload steady|chatbot|bursty|agent`):
+  two-moment wait so high-variance/agent workloads inflate the tail and carry a
+  "validate with a load test" warning - analytical queueing otherwise silently
+  approves fleets that miss SLOs for heavy-tailed traffic.
+- Numerical accuracy tests pinning throughput, the roofline calibration anchor,
+  the VRAM formula, TTFT, and batching invariants to ground truth (falsifiability).
+
+### Changed
+- **Throughput scales linearly across GPU replicas** (replaced the Amdahl
+  serial-fraction model, which capped total throughput at ~1.8x regardless of
+  instance count and rejected models >=7B). Per-GPU batching is modelled
+  separately (above).
+- **`cost_per_1m_tok` no longer understated by the instance count** (uses N-GPU
+  cost with N-GPU throughput; $/token is invariant in replica count).
+- Broader quant support (legacy + i-quants: `Q4_0`, `Q5_1`, `IQ4_XS`, ...) with
+  effective bits-per-weight, so a model's native quant is costed correctly.
+- Docs realigned to the planner product (research guides moved to an archive
+  section); ASCII-only source.
+
+### Fixed
+- **Activation memory is now O(context), not O(context^2).** The quadratic term
+  diverged unphysically at long context (~130 GB at 32k for a 3B model), which
+  spuriously failed the VRAM gate and zeroed `max_concurrent_seqs` (killing
+  batching) at >=8k context. Flash/paged attention never materialises the
+  attention matrix, so it scales linearly; coefficient re-pinned to preserve the
+  calibrated 2k value. (Found by a blind code audit.)
+- **`--json` is now valid when piped** for `bench`, `refit`, `compare`, and
+  `report` (added `highlight=False, soft_wrap=True`, matching the other six
+  commands). Rich previously reflowed long string values at width 79 and produced
+  invalid JSON for `... --json | jq`.
+- **`refit --validate` is a real gate**: validation runs *before* the write, so
+  invalid coefficients are no longer persisted ahead of the failing exit.
+- **Quality tier is family-aware** for off-registry models (consistent with the
+  reported quality), so the "concerning drop" advisory can fire instead of the
+  tier silently collapsing to `unknown`.
+- **Per-key confidence weighting in `refit`**: each entry is blended by its own
+  successful-run count, not the global run total (which over-trusted
+  lightly-measured configs in a multi-config refit).
+- Measured-corpus staleness warning: `plan`/`suggest` now warn (instead of
+  silently shadowing) when the cached corpus predates the installed version.
+- Robust error handling: `measure` surfaces an unknown `--backend` cleanly;
+  backend `check_model` handles timeouts/HTTP errors (not just connect); the
+  resolver/discovery raise `ResolverError` (not a raw traceback) on a non-JSON
+  200 response; a degenerate `...0b` identifier no longer raises ZeroDivisionError.
+- `bench --context ... --quant Q` no longer drops the quant label; non-Ollama
+  context sweeps warn that the per-request context override was not applied.
+- Roofline throughput is bandwidth-correct above FP16 (e.g. FP32 ~ 0.5x FP16);
+  Ollama `F16`/`F32` native-quant strings are normalised to `FP16`/`FP32`.
+- `eval --fp16-baseline` exposes tier classification (previously always
+  `unknown` from the CLI). Build floor corrected to `setuptools>=77` (PEP 639).
+
+### Notes
+- Per-backend MFU/MBU *calibration* is deferred: the `measure` loop already
+  supersedes the roofline estimate with real measurements for any benchmarked
+  model, which is stronger than tuning a global constant.
+- Known minor limitations (low impact, deferred): VRAM mixes decimal-GB weight
+  with binary-GiB KV (~7.4%, conservative); ambiguous partial GPU names (e.g.
+  "RTX 4080") resolve to the first DB match by VRAM; a genuine 0.0 BERTScore is
+  treated as "unavailable".
+
 ## [0.5.0] - 2026-06-24
 
 ### Added

diff --git a/CLAUDE.md b/CLAUDE.md
@@ -4,7 +4,7 @@
 
 ChimeraForge is an LLM inference benchmarking and deployment planning platform, broken out from the Banterhearts program. It provides quantified, reproducible answers to LLM deployment decisions, backed by ~204,000 real measurements on consumer GPUs. Ships both research artifacts (32 technical reports, TR108-TR137 + TR142/TR146) and production CLI tools (`chimeraforge plan` and `chimeraforge bench`).
 
-**Version:** 0.5.0 | **License:** MIT | **Python:** >=3.10 | **Rust:** >=1.70
+**Version:** 0.6.0 | **License:** MIT | **Python:** >=3.10 | **Rust:** >=1.70
 
 ## Quick Reference
 
@@ -36,7 +36,7 @@ chimeraforge plan --model qwen3:14b --measure   # bench live first, then plan (p
 # Run benchmarks (requires live Ollama)
 chimeraforge bench --model llama3.2-3b --runs 5
 
-# Run tests (431 total; model-agnostic adds resolver/discovery/catalog/measure/diagnostics + specs)
+# Run tests (476 total; 0.6.0 adds KV-batch/prefill-decode/continuous-batching/variance/pareto/accuracy)
 pytest tests/ -v
 
 # Lint
@@ -53,7 +53,7 @@ cd src/rust/demo_multiagent && cargo build --release
 ```
 src/
   chimeraforge/                       # CLI tool + capacity planner (pip-installable)
-    __init__.py                       # Exports __version__ = "0.5.0"
+    __init__.py                       # Exports __version__ = "0.6.0"
     cli.py                            # Typer entry point, registers plan/suggest/safety/... (lazy imports)
     commands/                         # One module per CLI command (plan.py, suggest.py, safety.py, ...)
     planner/
@@ -154,6 +154,16 @@ resources/prompts/                    # Legacy banter_prompts.txt (not used in b
 
 ## Planner Architecture (src/chimeraforge/planner/)
 
+### Serving model (0.6.0)
+The planner models LLM serving as the literature describes it, not replicas-of-single-stream:
+- **Prefill vs decode:** TTFT = prefill (compute-bound, `2*params*prompt_tokens / (fp16_tflops*MFU)`); TPOT = decode (bandwidth-bound). End-to-end p95 = TTFT + decode + queueing. `GPUSpec.fp16_tflops` drives prefill; `--prompt-tokens` sets input length.
+- **Continuous batching:** vLLM/TGI serve B concurrent sequences per GPU; `ThroughputModel.batched_decode_tps()` = `B*bw*MBU / (weight_eff + B*kv_per_seq)`, `weight_eff = bw*MBU/n1_tps` (anchored to measured/roofline single-stream, quant-correct), capped by `max_concurrent_seqs` (KV-bound) and the decode compute ceiling. Ollama = B=1. The engine searches **(N replicas x B batch/GPU)** for the cheapest SLO-feasible config. `BACKEND_CONTINUOUS_BATCHING`, `Candidate.effective_batch`.
+- **Replicas scale linearly** (eta=1); the old Amdahl serial-fraction model was wrong for replica fan-out (capped throughput at ~1.8x; rejected >=7B) and is no longer applied.
+- **Variance-aware queueing:** two-moment wait `(1+Cs^2)/2 * M/M/1` (`Cs^2=0` reproduces M/D/1). `--workload {steady,chatbot,bursty,agent}` -> `WORKLOAD_CV2`; high variance inflates the tail + warns (analytical queueing silently approves broken fleets for agent traffic otherwise).
+- **Pareto output:** `plan --pareto` -> `pareto_frontier()` (non-dominated on cost/p95/quality), the trade-off menu instead of one cost-sorted pick.
+- **Cost:** `cost_per_1m_tok` uses N-GPU cost with N-GPU throughput (invariant in replica count).
+- Numerical accuracy gates in `tests/test_accuracy.py` pin predictions to ground truth.
+
 The `chimeraforge plan` CLI runs a 4-gate exhaustive search (plus an opt-in 5th safety gate) over (model x quant x backend x N_agents):
 
 **Gate 1 — VRAM:** `weight_gb + kv_cache_gb + activations_gb <= hw_vram`

diff --git a/README.md b/README.md
@@ -645,8 +645,8 @@ This research was conducted as part of the Banterhearts LLM Performance Research
 
 ---
 
-**Last Updated:** June 25, 2026 (v0.5.0)
+**Last Updated:** June 25, 2026 (v0.6.0)
 **Repository:** https://github.com/Sahil170595/Chimeraforge
 **PyPI:** https://pypi.org/project/chimeraforge/
-**Status:** Phase 1 + Phase 2 + Phase 3 Complete | v0.5.0
+**Status:** Phase 1 + Phase 2 + Phase 3 Complete | v0.6.0
 
diff --git a/docs/API.md b/docs/API.md
@@ -2,7 +2,7 @@
 
 ## Overview
 
-This document covers the public Python API for the `chimeraforge` package (v0.2.1).
+This document covers the public Python API for the `chimeraforge` package (v0.6.0).
 For CLI usage, see the [README](../README.md).
 
 Install: `pip install chimeraforge[all]`
@@ -164,7 +164,8 @@ scores = evaluate_quality(
     references=["The capital of France is Paris."],
 )
 print(f"Composite: {scores.composite:.3f}")
-print(f"Tier: {classify_tier(scores.composite)}")
+# Tier needs the FP16 baseline composite to measure the drop against:
+print(f"Tier: {classify_tier(scores.composite, fp16_composite=0.85)}")
 ```
 
 ### Metrics
@@ -174,7 +175,7 @@ print(f"Tier: {classify_tier(scores.composite)}")
 - **`compute_bert_score(preds, refs)`** — BERTScore F1 (requires `evaluate` + `bert-score`)
 - **`compute_coherence(preds, refs)`** — length-ratio heuristic
 - **`compute_composite(scores)`** — weighted: 0.2*EM + 0.3*ROUGE + 0.3*BERT + 0.2*coherence
-- **`classify_tier(score)`** — negligible (>=-3pp), acceptable (>=-10pp), concerning, unacceptable
+- **`classify_tier(composite, fp16_composite)`** — negligible (>=-3pp), acceptable (>=-10pp), concerning, unacceptable
 
 ### Built-in tasks
 

diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md
@@ -1,5 +1,11 @@
 # Architecture Documentation
 
+> **Scope:** this document covers the **research / agent-benchmarking subsystem**
+> (`src/python/banterhearts/`, `src/rust/`) that produced the TR-series data. For
+> the **planner CLI** (`src/chimeraforge/` -- the shipping product), see the
+> "Planner Architecture" section of the top-level `CLAUDE.md` and
+> [Using the Planner](planning.md).
+
 ## Overview
 
 Chimeraforge is a benchmarking and research repository focused on LLM performance optimization. The architecture is designed for reproducibility, extensibility, and comprehensive performance analysis.

diff --git a/docs/README.md b/docs/README.md
@@ -1,144 +1,71 @@
 # Documentation Index
 
-Comprehensive documentation for Chimeraforge, a benchmarking and research repository for LLM performance optimization.
-
-## Quick Start
-
-- **[Quick Start Guide](quick_start.md)** - Get up and running in minutes with your first benchmark
-- **[Installation Guide](installation.md)** - OS-specific setup instructions and prerequisites
-
-## Core Documentation
-
-### Setup & Configuration
-- **[Installation](installation.md)** - Complete installation guide for all platforms
-- **[Dual Ollama Setup](dual_ollama_setup.md)** - Setting up dual Ollama instances for true concurrency
-- **[Architecture](ARCHITECTURE.md)** - System architecture and design principles
-
-### Running Benchmarks
-- **[Benchmarking Guide](benchmarking.md)** - Comprehensive guide to running all benchmark types
-- **[Multi-Agent Guide](multi_agent.md)** - Multi-agent concurrent execution scenarios
-- **[Python Agents](python_agents.md)** - Python agent implementation details
-- **[Rust Agents](rust_agents.md)** - Rust agent implementation details
-
-### Optimization & Tuning
-- **[Chimera Optimization](chimera_optimization.md)** - Configuration optimization strategies
-- **[Performance Tuning](performance_tuning.md)** - Performance tuning techniques and best practices
-
-### Analysis & Reporting
-- **[Statistical Analysis](statistical_analysis.md)** - Statistical rigor, sample sizes, and analysis methods
-- **[Rust vs Python](rust_vs_python.md)** - Cross-language performance comparison
-- **[Technical Reports](technical_reports.md)** - Complete index of all technical reports (TR108+)
-- **[Methodology](methodology.md)** - Research methodology and experimental design
-
-### Reference
-- **[FAQ](faq.md)** - Frequently asked questions and troubleshooting
-- **[Benchmarks README](../benchmarks/README.md)** - Directory map for stored benchmark artifacts
-
-## Project Documentation
-
-### Contributing
-- **[Contributing Guide](../CONTRIBUTING.md)** - How to contribute to the project
-- **[Code of Conduct](../CODE_OF_CONDUCT.md)** - Community standards and expectations
-
-### Project Information
-- **[Changelog](../CHANGELOG.md)** - Version history and changes
-- **[Security Policy](../SECURITY.md)** - Security reporting and best practices
-- **[Architecture](ARCHITECTURE.md)** - System architecture documentation
-
-## Documentation Structure
-
-### By User Type
-
-**New Users**:
-1. Start with [Quick Start](quick_start.md)
-2. Follow [Installation](installation.md)
-3. Run your first benchmark
-4. Read [Benchmarking Guide](benchmarking.md) for details
-
-**Researchers**:
-1. Review [Methodology](methodology.md)
-2. Study [Technical Reports](technical_reports.md)
-3. Understand [Statistical Analysis](statistical_analysis.md)
-4. Review [Architecture](ARCHITECTURE.md)
-
-**Developers**:
-1. Read [Architecture](ARCHITECTURE.md)
-2. Review [Contributing Guide](../CONTRIBUTING.md)
-3. Study agent implementations ([Python](python_agents.md) / [Rust](rust_agents.md))
-4. Understand benchmarking framework
-
-**Operators**:
-1. Follow [Installation](installation.md)
-2. Set up [Dual Ollama](dual_ollama_setup.md)
-3. Run benchmarks using [Benchmarking Guide](benchmarking.md)
-4. Troubleshoot using [FAQ](faq.md)
-
-### By Topic
-
-**Performance Optimization**:
-- [Chimera Optimization](chimera_optimization.md)
-- [Performance Tuning](performance_tuning.md)
-- [Rust vs Python](rust_vs_python.md)
-
-**Multi-Agent Systems**:
-- [Multi-Agent Guide](multi_agent.md)
-- [Dual Ollama Setup](dual_ollama_setup.md)
-- Technical Reports: TR110, TR113, TR114
+ChimeraForge is a model-agnostic **LLM deployment / capacity-planning CLI**: given
+your hardware, workload, latency SLO, quality target, and budget, it recommends
+the best (model x quantization x backend x GPU-instance-count) configuration --
+backed by real benchmarks, with honest per-prediction provenance. It also bundles
+the research harness (agent benchmarking, Rust vs Python, the TR series) that the
+planner's data came from; those guides are archived below.
+
+## The Planner CLI (start here)
+
+- **[Installation](installation.md)** - install from PyPI, optional extras
+- **[Quick Start](quick_start.md)** - first run in minutes
+- **[Using the Planner](planning.md)** - `plan` / `suggest` / `measure` / `catalog`, end to end
+- **[API Reference](API.md)** - the public Python API (`chimeraforge.planner`, resolver, discovery, measure)
+
+```bash
+pip install chimeraforge
+chimeraforge plan --model Qwen/Qwen2.5-7B-Instruct --hardware "RTX 4090 24GB" --request-rate 2
+chimeraforge suggest --source ollama --hardware "RTX 4080 12GB" --budget 500
+chimeraforge measure --model qwen3:14b --ollama-url http://localhost:11434
+```
 
-**Language-Specific**:
-- [Python Agents](python_agents.md)
-- [Rust Agents](rust_agents.md)
-- [Rust vs Python](rust_vs_python.md)
+Ten commands: `plan`, `suggest`, `measure`, `catalog`, `safety`, `bench`, `eval`,
+`refit`, `compare`, `report`. Run `chimeraforge --help` or `chimeraforge <cmd> --help`.
 
-**Research & Analysis**:
-- [Methodology](methodology.md)
-- [Statistical Analysis](statistical_analysis.md)
-- [Technical Reports](technical_reports.md)
+## Project Information
 
-## Technical Reports
+- **[Changelog](../CHANGELOG.md)** - version history
+- **[Contributing](../CONTRIBUTING.md)** | **[Security](../SECURITY.md)** | **[Code of Conduct](../CODE_OF_CONDUCT.md)**
 
-The canonical technical report archive is `outputs/publish_ready/reports/`.
+---
 
-Current coverage:
+## Research Archive
 
-- Phase 1: `TR108-TR122`
-- Phase 2: `TR123-TR133`
-- Phase 3: `TR134-TR137`
-- Conclusive synthesis sets: `108-116`, `117-122`, `123-133`, `134-137`
+ChimeraForge began as the public breakout of the Banterhearts performance-research
+program (~204,000 measurements across TR108-TR137). These guides document that
+research -- the agent-benchmarking harness, the language/runtime studies, and the
+methodology behind the bundled data. They describe the code under
+`src/python/banterhearts/` and `src/rust/`, **not** the planner CLI.
 
-See [Technical Reports](technical_reports.md) for the current linked index.
+### Benchmarking & methodology
+- **[Benchmarking Guide](benchmarking.md)** - running the benchmark harness
+- **[Methodology](methodology.md)** - experimental design, isolation, cold starts
+- **[Statistical Analysis](statistical_analysis.md)** - sample sizes, CIs, rigor
+- **[Technical Reports](technical_reports.md)** - index of the TR series (TR108+)
 
-## Repository Structure
-```
-Chimeraforge/
-├── src/              # Source code (Python & Rust)
-├── experiments/      # Research experiments (TR series)
-├── data/             # Data files (baselines, CSV, research)
-├── outputs/          # Generated outputs (artifacts, reports, runs)
-├── benchmarks/       # Benchmark results
-├── scripts/          # Utility scripts
-├── docs/             # Documentation (this directory)
-└── logs/             # Log files
-```
+### Agent research (Rust vs Python, multi-agent)
+- **[Rust vs Python](rust_vs_python.md)** - cross-language comparison (TR111-TR116)
+- **[Rust Agents](rust_agents.md)** / **[Python Agents](python_agents.md)** - implementations
+- **[Multi-Agent Guide](multi_agent.md)** - concurrent execution scenarios
+- **[Dual Ollama Setup](dual_ollama_setup.md)** - required for the multi-agent experiments
 
-See [Architecture](ARCHITECTURE.md) for detailed structure documentation.
+### Optimization & structure
+- **[Chimera Optimization](chimera_optimization.md)** - config optimization (TR108)
+- **[Performance Tuning](performance_tuning.md)** - tuning techniques
+- **[Architecture](ARCHITECTURE.md)** - the agent/research subsystem architecture
+- **[Repository Structure](repo_structure.md)** - folder layout & data governance
+- **[FAQ](faq.md)** - common questions
 
-## Getting Help
+The canonical technical-report archive lives in `outputs/publish_ready/reports/`.
 
-- **FAQ**: Check [FAQ](faq.md) for common questions
-- **Issues**: Open an issue on GitHub
-- **Discussions**: Use GitHub Discussions
-- **Contributing**: See [Contributing Guide](../CONTRIBUTING.md)
+---
 
 ## Documentation Standards
 
-- All documentation is ASCII-only for universal compatibility
-- Commands assume execution from repository root unless noted
-- Code examples are tested and verified
-- Links are relative to documentation structure
-- Last updated dates are maintained
-
----
-
-**Last Updated**: January 2025
+- ASCII-only for universal compatibility
+- Commands assume execution from the repository root unless noted
+- Links are relative to this `docs/` directory
 
+**Last Updated:** June 2026 (v0.5.0)
diff --git a/docs/faq.md b/docs/faq.md
@@ -217,7 +217,7 @@ See individual technical reports for citation information.
 
 ### Can I contribute benchmarks?
 
-Yes! See [Contributing Guide](contributing.md) for details.
+Yes! See [Contributing Guide](../CONTRIBUTING.md) for details.
 
 ## Still Have Questions?