[Bug]: Analytics "ran Nx longer before /compact" metric is a byte ratio presented as time

[EvolveAegis]

Summary

The analytics report line that ends with your AI ran N× longer before /compact fired presents a byte ratio as a time / longevity multiplier. Nothing in the computation measures elapsed time, turns, or how close the session got to a compaction event — yet the wording claims the AI "ran N× longer" before /compact triggered.

This is a metrics-correctness bug in formatReport, not a platform-specific runtime issue, so the standard debug-script / exact-prompt fields below don't really apply. The repro is a unit test that drives the real formatReport.

The misleading line

src/session/analytics.ts (current main, v1.0.169):

const measuredAvoided  = realConv?.bytesAvoided   ?? 0;   // L2200
const measuredReturned = realConv?.bytesReturned  ?? 0;   // L2201
...
const convBytesWithout  = measuredAvoided + measuredReturned;            // L2203
const convBytesWith     = Math.max(1, measuredReturned);                 // L2204
const convTokensWithout = Math.max(1, Math.floor(convBytesWithout / 4)); // L2205
const convTokensWith    = Math.max(1, Math.floor(convBytesWith    / 4)); // L2206
...
const convMult   = Math.max(1, Math.round(convTokensWithout / convTokensWith)); // L2216
out.push(`... ${convPct.toFixed(1)}% kept out of context · your AI ran ${convMult}× longer before /compact fired`); // L2219

convMult is round((bytesAvoided + bytesReturned) / bytesReturned) — a redirect byte ratio. The two inputs (bytesAvoided, bytesReturned) are byte counters from the real-bytes stats. There is no timestamp, no turn count, no /compact trigger record, and no proximity-to-compact measurement anywhere in this branch. The ratio is then rendered with time language ("ran ... longer before /compact fired").

Why this is misleading

With a single redirect that avoids 100,000 B and returns 100 B, the multiplier is round(100100 / 100) = 1001, and the report tells the user their AI "ran 1001× longer before /compact fired." That 1001× figure is fully determined by the byte ratio of one redirect — nothing about the session's actual duration, number of turns, or compaction behavior contributes to it. A user reading "ran 1001× longer before /compact" reasonably infers a runtime/longevity claim that the code never measured.

This is independent of whether the underlying bytesAvoided/bytesReturned numbers themselves are accurate. Even with perfectly correct byte accounting, presenting a byte ratio as a time multiplier is a causation claim the data doesn't support.

Minimal repro (synthetic data)

Self-contained vitest test against the real formatReport. All inputs are synthetic.

git clone https://github.com/mksglu/context-mode.git
cd context-mode
git checkout 252e74b   # v1.0.169, current main HEAD
npm install
mkdir -p tests
cat > tests/cm4-repro.test.ts <<'EOF'
import { describe, it, expect } from "vitest";
import { formatReport } from "../src/session/analytics";

describe("byte ratio rendered as time multiplier", () => {
  it("synthetic 100000B avoided / 100B returned -> 'ran 1001x longer before /compact'", () => {
    // Synthetic inputs: one redirect that avoids 100000 B and returns 100 B.
    // No timing, turn-count, or compact-proximity data is supplied.
    const report = formatReport(
      {
        savings: {
          processed_kb: 100, entered_kb: 1, saved_kb: 99, pct: 99,
          savings_ratio: 99, by_tool: [], total_calls: 2,
          total_bytes_returned: 100, kept_out: 100_000, total_processed: 100_100,
        },
        session: { id: "s1", uptime_min: "5" },
        continuity: { total_events: 1, by_category: [], compact_count: 0, resume_ready: false },
      } as never,
      undefined,
      null,
      {
        conversation: { events: 1, snapshotBytes: 0, byCategory: [], byDay: [] } as never,
        realBytes: {
          conversation: {
            eventDataBytes: 0, bytesAvoided: 100_000, bytesReturned: 100,
            snapshotBytes: 0, contentBytes: 0, totalSavedTokens: 25_000,
          } as never,
        },
      } as never,
    );
    const text = Array.isArray(report) ? report.join("\n") : String(report);
    const m = text.match(/ran (\d+)× longer before \/compact fired/);
    console.log("rendered:", m?.[0] ?? "NOT FOUND");
    expect(m).not.toBeNull();
    expect(Number(m![1])).toBe(1001); // round(100100 / 100)
  });
});
EOF
npx vitest run tests/cm4-repro.test.ts --reporter=verbose

Actual output (just ran this on main @ 252e74b):

rendered: ran 1001× longer before /compact fired
 ✓ tests/cm4-repro.test.ts > byte ratio rendered as time multiplier > synthetic 100000B avoided / 100B returned -> 'ran 1001x longer before /compact' 15ms
   Tests  1 passed

Expected vs actual

• Expected: the "ran N× longer before /compact fired" claim should be backed by an actual measurement of how long the session ran (or how many turns it survived) relative to a baseline, or it shouldn't be made at all.
• Actual: with purely synthetic byte counters and no timing/compact input, formatReport renders ran 1001× longer before /compact fired. The 1001× comes entirely from round((100000 + 100) / 100).

What I checked

• formatReport is the function that emits the line (src/session/analytics.ts, Section 1 "Without / With context-mode" block).
• convMult at L2216 only depends on convTokensWithout / convTokensWith, both derived from bytesAvoided + bytesReturned. I grepped the function for any time/turn/compact-proximity input feeding this branch and found none.
• The empty-state guard (L2202) only handles the case where both byte counters are zero; once either is non-zero, the multiplier and the "ran ... longer before /compact fired" wording are emitted unconditionally.

Suggested fix

Either drop the causal wording (e.g. keep just ${convPct.toFixed(1)}% kept out of context), or only emit "ran N× longer before /compact fired" when there's an actual compaction-timing measurement behind N. The byte-ratio multiplier is fine to compute, but labeling it as runtime/longevity is the part that overstates what was measured.

I'd send a PR but wanted to flag the wording first since the fix is a one-line display change and you may have a preference on phrasing.

Environment

• context-mode: 1.0.169 (built from main @ 252e74b)
• Reproduced via vitest run (Node), independent of host agent / OS