Skip to content

feat(interview-me): add opt-in collaborative mode#360

Open
kevglynn wants to merge 3 commits into
addyosmani:mainfrom
kevglynn:contrib/251-interview-me-collaborative-mode
Open

feat(interview-me): add opt-in collaborative mode#360
kevglynn wants to merge 3 commits into
addyosmani:mainfrom
kevglynn:contrib/251-interview-me-collaborative-mode

Conversation

@kevglynn

@kevglynn kevglynn commented Jul 8, 2026

Copy link
Copy Markdown

Summary

Implements the collaborative mode proposed in #251 inside the existing interview-me skill. The skill's default remains extraction-first — one question at a time, guess attached, no option-listing. What's new is an opt-in mode for when the user wants the agent as an interlocutor rather than an interviewer: the agent may propose, with an explicit lean and risk attached, and the ~95% confidence threshold is reached through consensus instead of pure extraction.

Concretely, following the design in #251:

  • New "Collaborative Mode (opt-in)" section — three entry points (explicit invocation, offered-and-accepted after Step 3, recovery from "whatever you think" delegation), plus the anti-goals: never the default, never turn 1, never more than 2–3 options.
  • New Step 3.5 — the PROPOSAL / WHY / RISK / LEAN / ASKING format, the 2–3-option variant with a mandatory lean, one proposal per turn, and the expertise-deference rule.
  • Step 4 delta — in collaborative mode the restate is origin-tagged per line (user's answer vs. accepted proposal vs. confirmed push), so the agent can't launder its proposals as user intent.
  • Stop-condition sub-check — the anti-anchoring question ("can I distinguish what they wanted from what they accepted because I proposed it well?"), triggered early when more than one PROPOSAL is accepted without pushback.
  • Four new rationalizations, six new red flags, four new verification items — all collaborative-mode-scoped, appended without touching the existing entries.
  • Frontmatter description gains the collaborative trigger vocabulary ("give me your take", "propose something", "what would you do here?", "let's think together"). Description is 779 chars, under the 1024 limit.
  • Eval case (evals/cases/interview-me.json) gains two positive triggers and one behavioral eval asserting the mode's contract (no turn-1 proposal, lean+risk present, origin-tagged restate, anti-anchoring check). One trigger uses organic phrasing ("I keep flip-flopping on what I actually want here — weigh in with your own proposal and the risks you see"); the other exercises the explicit catchphrase invocations.

Two deliberate deviations from the issue text:

  1. It references a grill-me skill for the boundary discussion; this repo's equivalent is doubt-driven-development, so the boundary line was added there ("plan written → doubt-driven review; intent not fixed → collaborative interview").
  2. The design's entry point 1 included an explicit interview-me --collaborative flag invocation; this implementation keeps only the natural-language synonyms, since skills in this repo are invoked by intent matching, not flags.

Non-duplication check

  • No open PR implements Proposal: add a collaborative mode to interview-me #251. The only open PR touching skills/interview-me/SKILL.md is feat: replace plain-text confirmations with question tool in 10 skills #195 (replaces plain-text confirmations with the question tool across 10 skills) — different scope; worst case is a small merge conflict in Step 5.
  • Searched open PRs for "interview" and "collaborative": no other hits.
  • No existing skill covers this: idea-refine is downstream divergence, doubt-driven-development is post-plan review. The mode is added inside interview-me precisely to avoid a near-duplicate skill, per CONTRIBUTING's "prefer a focused edit over a new directory."

Test plan

  • node scripts/validate-skills.js → 24 skills checked — 0 errors, 0 warnings — PASSED
  • node scripts/run-evals.js → 122 checks passed — 0 errors, 0 warnings; trigger rank-1 rate 85% (63/74) — both new positive triggers rank interview-me first (rate unchanged from the 85% baseline, 61/72 before). The organic-phrasing trigger ranking first is the meaningful signal here; the catchphrase trigger confirms the explicit invocations route as intended. The behavioral eval is provisional (no fixtures yet), so it documents the mode's contract rather than evidencing it.
  • SKILL.md is 311 lines, under the 500-line cap; frontmatter valid.

Closes #251.

)

The skill was extraction-only; when the user wants the agent as an
interlocutor ("give me your take", "let's think together") there was no
sanctioned way to propose without breaking the skill's anti-anchoring
discipline. This implements the design from addyosmani#251 inside interview-me:
three explicit entry points, a Step 3.5 PROPOSAL format with mandatory
lean and risk, origin-tagged restates, and an anti-anchoring sub-check
on the 95% stop condition. Eval case gains two collaborative-phrasing
positive triggers and a behavioral eval for the new mode.

Co-authored-by: Cursor <cursoragent@cursor.com>

@nucliweb nucliweb left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for picking this up, @kevglynn, and for flagging it for me. I read the diff line by line against the #251 design; this is a faithful, near-verbatim implementation of the spec. Step 3.5, the origin-tagged restate, the anti-anchoring sub-check, the four rationalizations, six red flags, and four verification items all match the design as written. Both repo adaptations are correct calls.

On the open questions

  • Q2 (implicit promotion): keep entry point 2. It is the most useful part of the mode, and it is safely gated: the agent never enters unilaterally, and the "or just restate" fallback keeps extraction the default. No change wanted here.
  • Q3 (grill-me): confirmed. There is no grill-me in this repo (the one users may know is from a different skill set), so anchoring the boundary on doubt-driven-development is the right move.
  • Q1 (dilution): agreed. No reachable path skips extraction, and the anti-anchoring check sends the agent back to Step 3, so if anything the stop condition is stricter. That was the intent.

Two adjustments before merge

  1. Call out the dropped interview-me --collaborative invocation. The design listed the explicit flag as entry point 1; the PR drops it in favor of NL synonyms only. I think that is the right choice for this repo (skills here are not flag-invoked), but it is a real deviation from the spec and is not in your "one deliberate deviation" note. Worth adding it there so the record is clean.

  2. De-circularize one positive eval prompt. Both new positive prompts embed the exact catchphrases just added to the description ("give me your take" + "let's think together"; "what would you do here?" + "propose something"). The evals README warns against copying the description because it games the trigger check: the TF-IDF ranks interview-me first largely because those tokens were just added to its description, so "both new prompts rank first" is partly tautological rather than independent evidence. Suggest rephrasing at least one to express the collaborative intent without the catchphrases (e.g. "I keep flip-flopping on this, weigh in with what you would pick and why") so it exercises organic routing.

Note, not a change request

The behavioral eval is provisional with no fixtures, so CI only checks lexical routing, not the mode's actual contract (no turn-1 proposal, lean+risk, origin tags, anti-anchoring). That is inherent to tier 3 until fixtures land (#352), just worth not citing that eval as evidence the mode behaves correctly yet.

Net: technically mergeable, faithful to the design. The two adjustments above are small. Nice work.

Review on addyosmani#360 flagged that both new positive prompts embedded the
exact catchphrases just added to the description, making their rank-1
result partly tautological. Reword the first to organic phrasing; the
second still exercises the explicit catchphrase invocations, which
users genuinely say. Rank-1 rate holds at 85% (63/74).

Co-authored-by: Cursor <cursoragent@cursor.com>
@kevglynn

kevglynn commented Jul 8, 2026

Copy link
Copy Markdown
Author

Both adjustments are in.

  1. Dropped flag documented. The PR description now lists two deliberate deviations — the doubt-driven-development boundary and the dropped interview-me --collaborative flag (kept NL synonyms only, since skills here route by intent matching, not flags). You're right that it belonged in the record; it was a silent call at implementation time.

  2. First trigger de-circularized (4e12016). Now reads: "I keep flip-flopping on what I actually want here — weigh in with your own proposal and the risks you see before we settle on anything." One data point from getting it to route: a fully organic phrasing without any description vocabulary ("I keep flip-flopping on this design decision — weigh in with what you'd pick…") ranked 5th, behind api-and-interface-design. The version that landed shares only "what I actually want" (pre-existing description vocabulary, not the new catchphrases) plus the naturally-overlapping "proposal"/"risks", and ranks first. That gap is arguably a milder instance of the Description vocabulary gaps: frontend-ui-engineering, performance-optimization #351 pattern — collaborative asks phrased without any interview vocabulary don't route here — but widening the description further felt like the wrong trade against collision risk, so I left it. Flagging in case you read it differently.

Second trigger intentionally keeps the catchphrases: it now serves as the check that the explicit invocations route as intended, while the first provides the organic evidence. Rank-1 rate holds at 85% (63/74).

On the behavioral eval: agreed — it's provisional with no fixtures, so it documents the mode's contract rather than evidencing it. The PR description now says exactly that. Happy to write fixtures for it once the #352 prerequisites land.

@addyosmani

Copy link
Copy Markdown
Owner

Really like this. Collaborative mode fills the gap where the user wants an interlocutor rather than an interrogator, and keeping it opt-in with explicit entry phrases means it can't hijack the default extraction behavior. The origin-tagged restate and the anti-anchoring check are the thoughtful touches. This delivers something I've had on the roadmap, so I want to read the new section end to end and then land it.

@addyosmani

Copy link
Copy Markdown
Owner

Did the end-to-end read I promised, and it holds up well. The three entry paths (explicit, promote-from-flow, recover-from-delegation), the differential-confidence rule, and the don't-propose-in-the-user's-domain guardrail are the thoughtful bits, and keeping it strictly opt-in with proposals never on turn 1 is what stops it wrecking the default extraction behavior. Structurally clean, CI's green.

One coordination thing, not a flaw in this PR: your eval2 is provisional with no fixture, which is fine today but collides with #381 (the eval-graduation PR) the moment it lands, since that flips provisional evals to a hard CI error. interview-me is genuinely awkward to fixture since it's a dialogue, so let's either add a light fixture here or settle the conversation-skill exemption in #381 first, then merge these in the right order. Content-wise I'm ready to take it.

@addyosmani

Copy link
Copy Markdown
Owner

Quick unblock for the eval collision I flagged: I've proposed a kind: "dialogue" exemption on #381 (tested, suite stays green) so conversation skills don't need a code fixture. Once that lands, your eval2 just becomes kind: "dialogue" and drops the provisional flag, nothing to fixture. So content's ready and this is good to go pending that merge ordering, nothing more needed from you here.

addyosmani flagged on addyosmani#360 that the new behavioral eval's
provisional/no-fixture state collides with addyosmani#381, which
promotes provisional evals to a hard CI error. addyosmani#381 now ships a
kind: "dialogue" exemption for conversation-shaped skills, so adopt
it here: eval 2 becomes kind: "dialogue" and drops trust_level.

The current runner ignores the kind field, so this is green under
both regimes and the two PRs can merge in either order. Eval 1 is
left untouched because addyosmani#381 converts it on main; editing it here
would conflict. 122 checks pass, rank-1 rate unchanged at 85%.

Co-authored-by: Cursor <cursoragent@cursor.com>
@kevglynn

Copy link
Copy Markdown
Author

Adopted the kind: "dialogue" exemption ahead of time in fe390b8 — eval 2 is now kind: "dialogue" with the provisional flag dropped. The current runner ignores the kind field (full suite still green locally), so this PR and #381 can land in either order now; no merge sequencing needed. I left eval 1 alone since #381 already converts it on main, and touching it here would just create a conflict.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Proposal: add a collaborative mode to interview-me

3 participants