Skip to content

Generate valid prompt frontmatter and canonical filenames#30

Merged
mike1858 merged 6 commits into
mainfrom
fix/generated-prompt-frontmatter-and-filenames
Jul 20, 2026
Merged

Generate valid prompt frontmatter and canonical filenames#30
mike1858 merged 6 commits into
mainfrom
fix/generated-prompt-frontmatter-and-filenames

Conversation

@mike1858

@mike1858 mike1858 commented Jul 19, 2026

Copy link
Copy Markdown
Member

Why

The prompt release pipeline needs to reproduce the generated corpus deterministically. Before this change, the tracked generator could emit invalid YAML in HTML-comment frontmatter, derive filenames differently from the documented canonical algorithm, persist zero token counts after API failures, and rewrite source-escaped prompt punctuation literally during regeneration.

The 2.1.216 release already contains the corrected generated corpus. This PR codifies those rules in the tracked generator so the next release does not regress the metadata, paths, README structure, token counts, or prompt bodies.

What changed

  • Added tools/promptMarkdownUtils.mjs with deterministic YAML-safe string serialization, rejection of unsafe HTML comment terminators, the documented canonical filename algorithm, legacy/current generated-name parsing, source-escape decoding, and a pre-generation collision guard.
  • Updated tools/updatePrompts.js to use the shared helpers for every frontmatter string and list item, including variables and all agent metadata fields.
  • Preserved existing files and README token counts by prompt name or unique unchanged body during filename and metadata migrations, avoiding unnecessary token recounting and count churn.
  • Preserved Skill and Tool Parameter README categorization, corrected agent subsection heading levels, and normalized generated description punctuation.
  • Kept the current 2.1.216 prompt bodies byte-for-byte stable by decoding extractor-preserved JavaScript source escapes before writing Markdown.
  • Updated token counting to use an available dated Sonnet model and fail the release instead of persisting zero when counting fails.
  • Added focused Node coverage for canonical slugs, legacy/current YAML scalars, source-escape decoding, YAML-significant and multiline values, agent metadata, unsafe comment terminators, and filename collisions.

Literal example placeholders such as ${API_KEY} remain intentionally out of scope.

Validation

  • node --test tools/promptMarkdownUtils.test.mjs — 6 passed
  • node --check tools/updatePrompts.js
  • ANTHROPIC_API_KEY=not-used-for-unchanged-bodies node tools/updatePrompts.js /code/tweakcc/data/prompts/prompts-2.1.216.json — 598 prompts; 0 new, 0 changed, 0 deleted
  • python3 /tmp/validate-pr30-output.py — 598 Markdown files, 598 README entries, 0 errors; 228 pre-existing interpolation-reference warnings remain unchanged from main
  • git diff --check -- tools

Summary by CodeRabbit

  • New Features

    • Improved prompt Markdown generation with consistent filenames and structured frontmatter.
    • Added support for Skills categorization in generated documentation.
    • Improved prompt migration handling by matching existing content and metadata more accurately.
    • Enhanced token-count tracking and reuse for unchanged prompts.
  • Bug Fixes

    • Prevented duplicate canonical prompt filenames.
    • Improved handling of escaped characters, multiline values, and tool metadata.
    • Corrected README headings, formatting, and token-count parsing.

@coderabbitai

coderabbitai Bot commented Jul 19, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 899c626a-73d8-4a33-bf94-202445bc4f35

📥 Commits

Reviewing files that changed from the base of the PR and between 7175f3e and 44020cf.

📒 Files selected for processing (2)
  • tools/promptMarkdownUtils.mjs
  • tools/promptMarkdownUtils.test.mjs
🚧 Files skipped from review as they are similar to previous changes (2)
  • tools/promptMarkdownUtils.mjs
  • tools/promptMarkdownUtils.test.mjs

📝 Walkthrough

Walkthrough

Added shared utilities for canonical prompt Markdown filenames, YAML frontmatter, escape decoding, and collision checks. Updated prompt synchronization to preserve metadata and token counts across migrations, and added Skills categorization to README generation.

Changes

Prompt Markdown synchronization

Layer / File(s) Summary
Canonical Markdown contracts
tools/promptMarkdownUtils.mjs, tools/promptMarkdownUtils.test.mjs
Utilities define filename, YAML, escape-decoding, collision, and frontmatter behavior with tests for quoting, legacy parsing, metadata, and scalar tool fields.
Prompt reconstruction and Markdown parsing
tools/updatePrompts.js
Prompt reconstruction decodes source escapes, generated Markdown uses shared frontmatter, and existing metadata/body content uses the new parsing helpers.
Prompt migration and token accounting
tools/updatePrompts.js
Updates validate filename collisions, resolve existing prompts by path, name, or body, preserve reusable token counts, and throw on token-count failures.
README categorization and rendering
tools/updatePrompts.js
Skills and tool-parameter prompts receive updated categories, descriptions and token parsing are normalized, and README headings and conditional Skills output are revised.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant PromptJSON
  participant updatePrompts
  participant MarkdownFiles
  participant Anthropic
  PromptJSON->>updatePrompts: provide prompts
  updatePrompts->>MarkdownFiles: resolve existing files by path, name, or body
  updatePrompts->>Anthropic: count changed prompt bodies
  Anthropic-->>updatePrompts: return token counts
  updatePrompts->>MarkdownFiles: write Markdown and rebuild README
Loading

Poem

I’m a bunny with slugs in a neat little row,
Quoting frontmatter wherever I go.
Skills join the README, tokens stay bright,
Escapes hop back into prompts just right.
No duplicate filenames spoil the night!

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main changes: valid prompt frontmatter generation and canonical filename handling.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/generated-prompt-frontmatter-and-filenames

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

mike1858 added 5 commits July 20, 2026 15:53
Serialize every frontmatter string as deterministic YAML-safe output and
derive generated paths with the documented non-alphanumeric slug rules.

Add collision and serialization regression coverage, preserve README token
counts across metadata-only and filename-only rewrites, and regenerate the
complete 2.1.215 prompt corpus.
Fail before generation when a metadata scalar contains the HTML comment terminator, which YAML quoting cannot make safe inside the surrounding comment block. Cover the guard in the serialization regression test.
Regenerate the seven source-confirmed catalog corrections, replace all eleven zero token counts with real count-endpoint results, and make token counting fail closed instead of persisting zero on API errors. Preserve established counts across prompt renames by matching unchanged generated bodies.
@mike1858
mike1858 force-pushed the fix/generated-prompt-frontmatter-and-filenames branch from d33bd1a to 7175f3e Compare July 20, 2026 22:28

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (2)
tools/promptMarkdownUtils.mjs (1)

145-150: 🎯 Functional Correctness | 🔵 Trivial | 💤 Low value

disallowedTools scalar values are silently dropped, unlike tools.

tools has an else if (metadata.tools != null) fallback to render a non-array scalar (Line 139-141), but disallowedTools only handles the Array.isArray case — a non-array truthy disallowedTools value would be dropped from frontmatter entirely with no warning.

Proposed fix for parity with `tools`
     if (Array.isArray(metadata.disallowedTools)) {
       lines.push('  disallowedTools:');
       for (const tool of metadata.disallowedTools) {
         lines.push(`    - ${yamlString(tool)}`);
       }
+    } else if (metadata.disallowedTools != null) {
+      lines.push(`  disallowedTools: ${yamlString(metadata.disallowedTools)}`);
     }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tools/promptMarkdownUtils.mjs` around lines 145 - 150, Update the metadata
serialization logic adjacent to the disallowedTools array handling to add the
same non-array fallback used for tools. When disallowedTools is non-null and not
an array, render it as a scalar instead of dropping it, while preserving the
existing array serialization behavior.
tools/updatePrompts.js (1)

301-349: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Solid multi-source matching with defensive body-collision handling.

The path→name→body fallback chain, existingTokenCounts lookup chain, and setting existingFilesByBody to null on duplicate bodies to avoid ambiguous matches are all well-reasoned for preserving token counts across renames/metadata-only edits. Confirmed the deferred cleanup (no inline unlinkSync on rename) is safe: promptsByFilename is keyed only by canonical filenames, so a stale old-named file is correctly caught by the later "deleted files" pass (Line 371-372).

One minor nit: when a rename is detected via the name/body fallback (new canonical filepath differs from the old file), the console logs Changed: (Line 334) even though this is effectively a rename+possible-metadata-change, not an in-place edit — purely cosmetic, not worth a code change.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tools/updatePrompts.js` around lines 301 - 349, Review the existing-file
handling around existingFilesByPromptName, existingFilesByBody, and the
“Changed:” log: no implementation change is required. The review identifies only
a cosmetic distinction for rename detection via fallback matching, so preserve
the current behavior unless you choose to adjust the message separately.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@tools/promptMarkdownUtils.mjs`:
- Around line 145-150: Update the metadata serialization logic adjacent to the
disallowedTools array handling to add the same non-array fallback used for
tools. When disallowedTools is non-null and not an array, render it as a scalar
instead of dropping it, while preserving the existing array serialization
behavior.

In `@tools/updatePrompts.js`:
- Around line 301-349: Review the existing-file handling around
existingFilesByPromptName, existingFilesByBody, and the “Changed:” log: no
implementation change is required. The review identifies only a cosmetic
distinction for rename detection via fallback matching, so preserve the current
behavior unless you choose to adjust the message separately.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 8cd352e8-e43c-45cf-bec1-24728e3c7509

📥 Commits

Reviewing files that changed from the base of the PR and between dd33af3 and 7175f3e.

📒 Files selected for processing (3)
  • tools/promptMarkdownUtils.mjs
  • tools/promptMarkdownUtils.test.mjs
  • tools/updatePrompts.js

@mike1858
mike1858 merged commit c696225 into main Jul 20, 2026
1 check passed
@mike1858
mike1858 deleted the fix/generated-prompt-frontmatter-and-filenames branch July 20, 2026 23:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants