Review: cleanupPeriodDays retention sweep — PAI dependencies and archival strategy

[virtualian]

Context

Claude Code's cleanupPeriodDays setting drives a background retention sweep. Per the changelog, the sweep now covers:

• ~/.claude/projects/ — conversation history / transcripts (long-standing behaviour)
• tool result files
• ~/.claude/tasks/ (added recently)
• ~/.claude/shell-snapshots/ (added recently)
• ~/.claude/backups/ (added recently)

Defaults and quirks worth noting:

• Default horizon is 30 days.
• cleanupPeriodDays: 0 is now rejected with a validation error (previously silently disabled transcript persistence).
• --setting-sources without user previously caused the sweep to ignore the configured value; fixed.

PAI does not currently set cleanupPeriodDays anywhere in ~/.pai/ or ~/.claude/settings*.json (checked 2026-04-22), so it inherits the default 30-day sweep.

Why review now

PAI's MEMORY/LEARNING/ pipeline (FAILURES/, REFLECTIONS/, SIGNALS/, mine-output.md, review.md) depends on session artefacts that originate under ~/.claude/projects/. If the retention sweep deletes a transcript before the learning pipeline consumes or copies it, signals and failure analyses are lost silently. The new dirs added to the sweep (tasks/, shell-snapshots/, backups/) may also contain data PAI hooks touch.

What to review

1. Dependency audit: enumerate every PAI hook, skill, or script that reads from ~/.claude/projects/, ~/.claude/tasks/, ~/.claude/shell-snapshots/, or ~/.claude/backups/. For each, determine whether it consumes within a predictable window or assumes indefinite retention.
2. Consumption horizon: what is the longest delay between a Claude Code session ending and a PAI pipeline consuming its artefacts? (E.g. monthly review.md synthesis may reach back further than the default 30 days.)
3. Archive vs sweep: should PAI's learning pipeline copy any required session data into ~/.pai/MEMORY/ at end-of-session (hook) so it is independent of Claude Code's sweep, rather than depending on live ~/.claude/projects/ files?
4. Recommended value: pick one of —
   • Leave default (30d), rely on end-of-session archival hooks for anything PAI needs beyond that horizon.
   • Ship PAI with an explicit higher value (e.g. 90d) in a PAI-managed settings.json fragment.
   • Make it user-configurable with a documented recommendation and guard-rails.
5. Interaction with --setting-sources: PAI loads rules at startup (loadAtStartup). Verify the current Claude Code CLI respects cleanupPeriodDays in whichever settings source PAI writes to; do not rely on the prior bug being the only issue.

Decision criteria

• If any PAI pipeline assumes retention beyond 30 days without making its own copy, the current default is a silent data-loss risk — raise the value and add an archival hook.
• If all PAI consumption happens within the default window, leaving the value alone is correct — do not over-configure.
• Archival hook is preferred over raising retention, because raised retention is a user-disk-space cost PAI shouldn't impose without cause.

Deliverable

• Audit output: a short table of PAI → ~/.claude/ directory dependencies with consumption windows.
• A decision: change default, add archival hook, or document current behaviour as sufficient.
• If config change: a proposed settings.json fragment with rationale.
• If archival hook: a hook script plus where it plugs into PAI (likely an end-of-session hook).

Out of scope

• Replacing Claude Code's sweep with a PAI-owned sweep.
• Broader review of ~/.claude/ directory layout (track separately if wanted).

[virtualian]

Finding from dependency audit (pre-work on this issue)

SessionHarvester.ts is hard-scoped to a single project shard — the one for sessions started with cwd = ~/.claude — not the current working directory or all shards.

// .claude/PAI/Tools/SessionHarvester.ts (v4.0.3+)
const CLAUDE_DIR = path.join(process.env.HOME!, ".claude");
const CWD_SLUG   = CLAUDE_DIR.replace(/[\/\.]/g, "-");            // → -Users-{username}--claude
const PROJECTS_DIR = path.join(CLAUDE_DIR, "projects", CWD_SLUG); // → ~/.claude/projects/-Users-{username}--claude/

CWD_SLUG is derived from CLAUDE_DIR (constant ~/.claude), not from process.cwd(). Harvester reads one shard, always. MEMORYSYSTEM.md documents the same single location across every release from v2.4 through v4.0.3+ — no wildcard, no fan-out.

Observed state on a live machine (2026-04-22)

Roughly 20 per-cwd shards under ~/.claude/projects/:

• -Users-{username}/ — sessions started from $HOME
• -Users-{username}--claude/ — sessions started from ~/.claude (the only shard the harvester reads)
• -Users-{username}-projects-pai/
• -Users-{username}-projects-marrbox/
• -Users-{username}-projects-ask-viki/
• -Users-{username}-projects-marrbox-marrbox-platform/
• …and ~14 more

Every session started from inside a repo (the common case for coding work) lands in one of those other shards.

Consequence for this issue

The coverage gap makes the retention problem strictly worse than the original issue description suggested. Per-repo session transcripts are:

1. Invisible to the learning pipeline — never contribute to REFLECTIONS/, SIGNALS/, FAILURES/, or mine-output.md.
2. Still subject to cleanupPeriodDays (default 30d) — swept without ever feeding the learning loop.

The retention question ("30d too short?") assumes the pipeline consumes the data. For ~19 of 20 shards, it doesn't. Fixing retention without fixing harvester coverage moves no needle.

Implication for the decision

Previously stated decision criteria in this issue said:

If any PAI pipeline assumes retention beyond 30 days without making its own copy, the current default is a silent data-loss risk

Sharper statement given the finding:

The current harvester consumes ≤ 1/20 of session data. Raising retention without broadening harvester coverage is cargo-cult — it preserves data nothing reads. Broadening harvester coverage without raising retention is fine if consumption happens within 30d.

Recommend splitting the work:

1. This issue (Review: cleanupPeriodDays retention sweep — PAI dependencies and archival strategy #152) — stays scoped to retention strategy, but blocks on (2).
2. New issue — "SessionHarvester only reads one project shard — per-repo sessions invisible". Distinct bug; fix is either (a) iterate all shards under ~/.claude/projects/ or (b) explicitly document the single-shard scope as intended.

Happy to open the new issue on request.