Skip to content

feat: filter agent-restricted skills from prompts and tool description#2827

Open
z-traveler wants to merge 1 commit intocode-yeongyu:devfrom
z-traveler:feat/per-agent-skill-filtering
Open

feat: filter agent-restricted skills from prompts and tool description#2827
z-traveler wants to merge 1 commit intocode-yeongyu:devfrom
z-traveler:feat/per-agent-skill-filtering

Conversation

@z-traveler
Copy link

@z-traveler z-traveler commented Mar 25, 2026

Problem

Skills with an agent frontmatter field are intended to be restricted to a specific agent. However, they currently appear in:

  1. Every agent's system promptbuildAvailableSkills returned all skills regardless of the definition.agent field, so a skill marked agent: oracle would still show up in Sisyphus's and Hephaestus's skill delegation sections, wasting tokens and potentially misleading the LLM.
  2. The skill tool's <available_items> description — shared across all agents, again leaking restricted skills to unintended agents.

The execution-time check (throwing an error when the wrong agent calls a restricted skill) already existed, but that only catches the problem after the LLM has already decided to make the call.

Solution

Add a description-level visibility gate so restricted skills are never shown to agents that cannot use them.

Changes

  • buildAvailableSkills (available-skills.ts): new optional agentName parameter. When provided, skills whose definition.agent does not match the current agent are filtered out. Fully backward-compatible — omitting agentName preserves existing behaviour.

  • createBuiltinAgents (builtin-agents.ts): call buildAvailableSkills with the agent name for sisyphus, hephaestus, and atlas individually, so each agent's prompt only lists its own skills.

  • createSkillTool (tools.ts): filter agent-restricted skills out of both the eager and lazy description builds. The full skill list is still used at execute time, so the existing execution-time check remains intact.

Tests (TDD)

  • New file src/agents/builtin-agents/available-skills.test.ts — 5 cases:

    • backward compat: no agentName → restricted skills still included
    • matching agent → included
    • non-matching agent → excluded
    • public skill (no agent field) → always included
    • mixed list filters correctly per agent
  • tools.test.ts — 3 new cases in a new describe block:

    • agent-restricted skill absent from <available_items> description
    • public skill present in description
    • execute still succeeds for a restricted skill when called with the correct agent context

Behavior summary

Scenario Before After
agent: oracle skill in Sisyphus's prompt ✅ visible ❌ hidden
agent: oracle skill in Oracle's prompt ✅ visible ✅ visible
skill tool <available_items> for any agent shows all skills shows only public skills
Oracle calls skill(name="oracle-only") at runtime ✅ works ✅ works
Sisyphus calls skill(name="oracle-only") at runtime ❌ throws ❌ throws (unchanged)
Skills without agent field ✅ visible everywhere ✅ visible everywhere

Summary by cubic

Hide agent-restricted skills from non-owner agents in prompts and the shared skill tool description. This reduces prompt tokens and prevents misleading calls; runtime enforcement is unchanged.

  • New Features
    • Added agentName to buildAvailableSkills; filters skills by definition.agent when provided (backward compatible when omitted).
    • Updated createBuiltinAgents to pass agent names for Sisyphus, Hephaestus, and Atlas so each prompt lists only its own skills.
    • Updated createSkillTool to exclude agent-restricted skills from both eager and lazy description builds (e.g., <available_items>); execution still uses the full list.

Written for commit bebbb5b. Summary will update on new commits.

Skills with an `agent` frontmatter field are intended for a specific
agent. Previously they still appeared in:
- every agent's system prompt (via `buildAvailableSkills`)
- the `skill` tool's `<available_items>` description visible to all agents

This wasted tokens and could mislead agents into attempting calls that
would be rejected at execution time.

Changes:
- `buildAvailableSkills`: new optional `agentName` parameter; when
  provided, skills whose `definition.agent` does not match are excluded
- `builtin-agents.ts`: pass per-agent name to `buildAvailableSkills`
  for sisyphus, hephaestus, and atlas, so each agent's prompt only
  lists the skills it is allowed to use
- `createSkillTool` (`tools.ts`): exclude agent-restricted skills from
  both the eager and lazy description builds, keeping the shared tool
  description free of skills the current agent cannot access

Execution-time enforcement (throwing on mismatch) is unchanged; this
change adds the earlier, description-level visibility gate.

Tests: new `available-skills.test.ts` (5 cases) + 3 new cases in
`tools.test.ts` covering the description-filter and execute paths.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Copy link

@cubic-dev-ai cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 5 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Auto-approved: The PR correctly implements agent-based skill filtering with backward compatibility and includes comprehensive tests. No regression risk identified.


Since this is your first cubic review, here's how it works:

  • cubic automatically reviews your code and comments on bugs and improvements
  • Teach cubic by replying to its comments. cubic learns from your replies and gets better over time
  • Add one-off context when rerunning by tagging @cubic-dev-ai with guidance or docs links (including llms.txt)
  • Ask questions if you need clarification on any suggestion

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant