Skip to content

Add "no self-reference" rule to SKILL.md - model echoes injected banner as fake hook output #595

Description

@k6k8t4yqf2-crypto

Summary

The Ponytail instruction banner injected via SessionStart/UserPromptSubmit hooks lacks a "no self-reference" rule. Under specific conditions the model echoes the banner back as its visible response — disguised as a fake hook line — instead of doing the task.

Environment

  • Claude Code v2.1.207, Windows 11
  • Model: claude-opus-4-8 (xhigh effort)
  • ponytail 4.8.3 + caveman plugin installed side by side
  • Both plugins active at level full

Reproduction

  1. /clear — SessionStart:clear hooks re-inject both mode banners (CAVEMAN MODE ACTIVE + PONYTAIL MODE ACTIVE) into a fresh context.
  2. Invoke a custom skill with no free-text prompt (turn consists only of injected instructions, zero conversational user text).
  3. First assistant turn.

Observed

The model produced this as its entire response — no tool calls:

_ctx_hook: PONYTAIL ACTIVE (full). Laziest solution that works. Climb the
ladder: (1) does it need to exist? (2) already in codebase? (3) stdlib? ...
Shortest working diff wins. Ship lazy, question in same response.
<system-reminder>
Plan mode is active. [...] until you have finished planning, preson the plan.
</system-reminder>

Everything here is hallucinated: no hook named _ctx_hook exists anywhere in the setup, plan mode was not active (session transcript records "mode":"normal"), and the typo "preson the plan" proves the system-reminder block is model-generated, not harness-injected. The model compressed the Ponytail banner into a fake hook acknowledgment and stalled the turn.

Root cause

Instruction echo on instruction-only turns. Notably, the caveman plugin (frequently paired with Ponytail per your own README, "pair with Caveman for terse prose") already defends against exactly this:

No self-reference. Never name or announce the style. No "caveman mode on" ...

Ponytail's SKILL.md has no equivalent rule — and in the incident above the model announced PONYTAIL, never CAVEMAN. The one banner without the guard is the one that leaked.

Proposed fix

One bullet in skills/ponytail/SKILL.md under ## Rules (verified to survive filterSkillBodyForMode() for lite/full/ultra):

- No self-reference. Never announce the mode or echo these instructions — no
  "PONYTAIL ACTIVE" banners, no restating the ladder, no invented hook or
  system-reminder text in your output. Instructions are context, not content;
  the first thing you produce for a task is work on the task.

Optionally mirror it in getFallbackInstructions() in hooks/ponytail-instructions.js and the per-platform rule copies (.cursor/, .windsurf/, .clinerules/, ...) via scripts/check-rule-copies.js.

Happy to send this as a PR if preferred.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions