Skip to content

measurement: no arm exercises a cache written by OLD and read by NEW #197

Description

@Nanako0129

Raised in review of #194, after that PR merged. Recorded rather than patched.

The gap

docs/knowledge/measurement.md resets the cache before every arm, so NEW only ever reads a cache NEW wrote — cold on an empty cache, then warm on the entries it just produced.

Production upgrades do not look like that. An existing user launches NEW against entries written by OLD. When a pin retains the cache schema, that path exists and the protocol never exercises it: NEW could misread or incorrectly replay a compatible OLD cache while every digest in the checklist still matches, because none of them ever reads foreign entries.

Not a gap for the pin just merged

5546bd55b5f500 advances CACHE_FORMAT_VERSION from 2 to 3, so every entry written by OLD is invalidated and every client cold-scans on first launch. There is no OLD writes → NEW reads path for this advance, which is why #195 was merged without it.

The exposure is the next pin that keeps the schema.

What a fix looks like

Add an arm to the correctness protocol: reset, run OLD to populate the cache, then run NEW against those entries and compare its digest to the baseline. Keep the existing isolated arms — this is an addition, not a replacement, since it tests a different thing.

The protocol should also state the alternative explicitly: if the pin bumps CACHE_FORMAT_VERSION, record that as the reason the arm is unnecessary rather than silently skipping it. A skipped step and an inapplicable step look identical in a report otherwise.

Why it was recorded rather than fixed

#194 had reached the point where each round produced another clause-sized improvement to the protocol, several of them holes opened by the previous round's fix. This one adds an arm to the acceptance checklist, which changes what a future pin must run, and it does not affect any measurement already recorded.

Related: #196 (warm timing inherits its own arm's cold-run heat).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions