Skip to content

refactor(validate-skills): extract lint rules into a shared lib#379

Open
nucliweb wants to merge 2 commits into
addyosmani:mainfrom
nucliweb:test/skill-lint-extraction
Open

refactor(validate-skills): extract lint rules into a shared lib#379
nucliweb wants to merge 2 commits into
addyosmani:mainfrom
nucliweb:test/skill-lint-extraction

Conversation

@nucliweb

@nucliweb nucliweb commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator

What

Extracts the skill validation rules out of the validate-skills.js CLI into a shared scripts/lib/skill-lint.js: a pure lintSkillContent() (no I/O) plus a thin lintSkill() file wrapper. validate-skills.js becomes a thin CLI over the lib.

Why

The rules were only reachable by spawning the CLI, so they could not be unit-tested directly. Extracting a pure function makes them importable and testable, which unblocks a follow-up test battery (linter self-tests + per-skill checks). This PR is the small, behavior-preserving first half of that split, kept separate to avoid a large refactor of a hot file landing together with new test code.

No behavior change

node scripts/validate-skills.js produces byte-identical output and the same exit codes as before, verified against all 24 skills. One intentional string update: the exemption-guard error now points contributors to scripts/lib/skill-lint.js (where the SECTION_EXEMPT_SKILLS allowlist now lives); that path only fires on an exemption-bypass attempt, so no real skill output changes.

Follow-up

PR-B will add the node:test battery, linter self-tests, and length metrics on top of this lib (supersedes the now-closed #324).

…lint.js

Split the skill validation rules out of the validate-skills.js CLI into a
shared, importable scripts/lib/skill-lint.js: a pure lintSkillContent() with
no I/O plus a thin lintSkill() file wrapper. validate-skills.js becomes a thin
CLI over the lib. No behavior change to the validator's output or exit codes;
this only makes the rules unit-testable for a follow-up test battery.
@nucliweb

nucliweb commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator Author

Hi @addyosmani @federicobartoli, could you take a look when you have a moment?

@addyosmani

Copy link
Copy Markdown
Owner

I like where this is headed. A pure lintSkillContent() with no I/O is the right shape and makes the rules testable on their own. My one hesitation is timing: this touches the same file as a couple of other in-flight script PRs (the naming/description enforcement, the draft test battery), so I want to sequence it rather than rebase three ways. Can you hold while I clear those, or rebase once they land?

@nucliweb

Copy link
Copy Markdown
Collaborator Author

Thanks, and that sequencing instinct is exactly why I split this out, so let me give you the current state, because I think the collision you are guarding against is already gone:

So this is already the sequenced, on-top-of-merged-work small piece, not one of three racing edits. PR-B (the node:test battery + linter self-tests + metrics) will only come after this lands, so it stays one at a time.

Happy to hold if you would still rather clear the decks first, but there is nothing left for it to collide with that I can see. Shout if I am missing an in-flight PR you are tracking.

@federicobartoli

Copy link
Copy Markdown
Collaborator

Thanks for splitting this out, @nucliweb !! I took a deeper pass and I agree with the overall direction. The extraction is clean, the I/O boundary is much better, and the current CLI behavior is preserved.
There is one small API detail I think would be worth tightening in this PR: the policy collections (REQUIRED_SECTIONS, SECTION_EXEMPT_SKILLS, and the reference patterns) are exported by reference. A test or future consumer could accidentally mutate them and change subsequent lint results for the whole process. Keeping the canonical policy state private and exporting only the linting functions, or defensive snapshots, would preserve the intended purity and avoid order-dependent tests.
While exercising the new test seam, I also found a few useful cases for PR-B:

  • The current frontmatter helper implements a YAML-like subset: it can accept malformed YAML or non-string descriptions, reject valid folded/commented YAML, and miss the real length of multiline descriptions.
  • Section checks currently use substring matching, so headings inside fenced examples or ### headings can satisfy the rule. There is also an existing contract question to resolve: the docs describe these sections as recommended and allow equivalent headings, while the validator treats the exact headings as required.
  • The cross-reference patterns miss some reference styles already used in the corpus and can inspect references inside fenced examples.
  • A negative phrase such as “Do not use when…” currently satisfies the positive trigger check.
    These look like existing validator limitations rather than regressions introduced by this extraction, so I’m not suggesting we expand PR-A to solve them. I would just like to make sure they are captured in PR-B - or in a linked follow-up -before the new tests lock the current heuristics in as the contract.
    With the exported-policy state encapsulated, this looks good to me.

Per review: the policy collections (REQUIRED_SECTIONS, SECTION_EXEMPT_SKILLS,
SKILL_REF_PATTERNS, regexes) were exported by reference, so a consumer could
mutate shared state and change lint results process-wide. Export only the
linting functions; keep the policy collections private. No behavior change
(validator output byte-identical).
@nucliweb

Copy link
Copy Markdown
Collaborator Author

Thanks for the deep pass. Encapsulated the exported policy state in 495b5a8: the lib now exports only the functions (parseFrontmatter, extractSkillReferences, lintSkillContent, lintSkill), while REQUIRED_SECTIONS, SECTION_EXEMPT_SKILLS, SKILL_REF_PATTERNS, and the regexes stay private, so no shared state can be mutated by a test or future consumer. Validator output stays byte-identical.

On the heuristic limitations (YAML-subset frontmatter, substring section matching plus the recommended-vs-required contract question, missed or over-broad cross-ref patterns, and "Do not use when..." satisfying the positive trigger): agreed these are pre-existing validator behavior, not regressions from the extraction, and agreed they should be settled before PR-B's tests lock the current heuristics in as the contract. I'll capture them in a linked follow-up issue so PR-B references it and we decide bug-vs-intended per case first. Will link it here.

@nucliweb

Copy link
Copy Markdown
Collaborator Author

Captured as #387.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants