Skip to content

DeriveDenyHashes: dictish() strips "-ed" without restoring the dropped silent e, so ordinary participles become permanent write-blocking rules #2000

Description

@jlacour-git

Affected: main (verified today), LifeOS/install/LIFEOS/TOOLS/DeriveDenyHashes.ts
Component: deny-hash derivation → SystemFileGuard (via PreToolGuard)
Impact: medium-high — fails closed. It blocks legitimate writes to SYSTEM
files and reports them as a private-token leak, so the message points away from
the real cause.

Symptom

On an install whose private corpus contains the Title-Case phrase Last Updated
anywhere — a table header, a frontmatter label, a status line — every subsequent
Write/Edit into a SYSTEM file whose new content contains the words last and
updated adjacently is refused:

SystemFileGuard — write BLOCKED.
  Pattern:   derived:private-token (salted-hash)
  Matched:   last updated

The frontmatter key last_updated tokenizes to exactly that 2-gram, so this
covers the header of nearly every documentation file in the tree. On the install
where this surfaced: 1 distinct token fired across 141 scanned SYSTEM files,
and it fired in 33 of them.
The other five derived hashes never matched
ordinary system text, so this is not a general over-derivation — it is one
specific class of token.

Repro

import { extractTokens } from "./LIFEOS/TOOLS/DeriveDenyHashes.ts";

extractTokens("Last Updated");   // => Set { "last updated" }
extractTokens("Last Modified");  // => Set { "last modified" }
extractTokens("Never Updated");  // => Set { "never updated" }
extractTokens("Last Reviewed");  // => Set { "last reviewed" }

Each of those becomes a salted hash, and from then on a blocking rule.

Cause

dictish() strips suffixes to test a word against the system dictionary, but for
-ed it removes two characters without restoring the silent e that English
drops before the suffix:

// LifeOS/install/LIFEOS/TOOLS/DeriveDenyHashes.ts, current main
function dictish(lw: string): boolean {
  if (DICT.has(lw)) return true;
  const tries: string[] = [];
  if (lw.endsWith("ies")) tries.push(lw.slice(0, -3) + "y");
  for (const suf of ["s", "es", "ed", "ing", "'s", "’s", "ly", "er", "ers"]) {
    if (lw.endsWith(suf)) tries.push(lw.slice(0, -suf.length));
  }
  return tries.some((s) => s.length >= 3 && DICT.has(s));
}
  • updatedupdat — not a word. Correct stem is update.
  • modifiedmodifi — not a word. Correct stem is modify (iedy).
  • updatingupdat — same gap on the -ing branch.

The classic /usr/share/dict/words (web2) holds base forms only, so updated,
modified and reviewed are absent from it and, with the stem also missing,
dictish() returns false and the word counts as distinctive.

The pair test then only needs one distinctive half:

if (distinctive(m[1]) || distinctive(m[2])) out.add(`${m[1]} ${m[2]}`.toLowerCase());

So one ordinary participle is enough to turn a completely generic Title-Case
phrase into a permanent blocking rule.

Note that iesy is already handled one line above, which suggests the
silent-e and y restorations were simply overlooked rather than deliberately
omitted.

Why STOPWORDS is not the fix

The comment above STOPWORDS records the same failure class being patched by
hand on 2026-07-08:

modern-common words the classic /usr/share/dict/words list omits, so they read
as the "distinctive" half of Title-Case phrases and got hashed as false private
fingerprints (2026-07-08 fix; words ordered so no two form a currently-hashed pair).

Ten words were listed by hand, with an ordering constraint to avoid creating new
pairs. English -ed participles are an open class; they cannot be enumerated.
Fixing the stemmer removes the class instead of one more instance of it.

Suggested fix

Three candidate stems, alongside the existing ies rule:

if (lw.endsWith("ied")) tries.push(lw.slice(0, -3) + "y");   // modified -> modify
if (lw.endsWith("ed"))  tries.push(lw.slice(0, -1));         // updated  -> update
if (lw.endsWith("ing")) tries.push(lw.slice(0, -3) + "e");   // updating -> update

Verified locally after the change:

  • Last Updated, Last Modified, Never Updated, Last Reviewed,
    Auto Generated → no token derived.
  • Genuine Title-Case name pairs still derive normally, so the filter does not
    lose coverage.

Note for anyone already affected

The bad entry stays in a generated DENY_HASHES.json until it is re-derived.
Because the file holds only salted hashes, the offending entry can be identified
by hashing the suspected phrase with the install's own salt and matching it
against the stored list, then removing that one entry — no need to discard the
rest.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions