feat(interview-me): add opt-in collaborative mode#360
Conversation
) The skill was extraction-only; when the user wants the agent as an interlocutor ("give me your take", "let's think together") there was no sanctioned way to propose without breaking the skill's anti-anchoring discipline. This implements the design from addyosmani#251 inside interview-me: three explicit entry points, a Step 3.5 PROPOSAL format with mandatory lean and risk, origin-tagged restates, and an anti-anchoring sub-check on the 95% stop condition. Eval case gains two collaborative-phrasing positive triggers and a behavioral eval for the new mode. Co-authored-by: Cursor <cursoragent@cursor.com>
nucliweb
left a comment
There was a problem hiding this comment.
Thanks for picking this up, @kevglynn, and for flagging it for me. I read the diff line by line against the #251 design; this is a faithful, near-verbatim implementation of the spec. Step 3.5, the origin-tagged restate, the anti-anchoring sub-check, the four rationalizations, six red flags, and four verification items all match the design as written. Both repo adaptations are correct calls.
On the open questions
- Q2 (implicit promotion): keep entry point 2. It is the most useful part of the mode, and it is safely gated: the agent never enters unilaterally, and the "or just restate" fallback keeps extraction the default. No change wanted here.
- Q3 (grill-me): confirmed. There is no
grill-mein this repo (the one users may know is from a different skill set), so anchoring the boundary ondoubt-driven-developmentis the right move. - Q1 (dilution): agreed. No reachable path skips extraction, and the anti-anchoring check sends the agent back to Step 3, so if anything the stop condition is stricter. That was the intent.
Two adjustments before merge
-
Call out the dropped
interview-me --collaborativeinvocation. The design listed the explicit flag as entry point 1; the PR drops it in favor of NL synonyms only. I think that is the right choice for this repo (skills here are not flag-invoked), but it is a real deviation from the spec and is not in your "one deliberate deviation" note. Worth adding it there so the record is clean. -
De-circularize one positive eval prompt. Both new positive prompts embed the exact catchphrases just added to the description ("give me your take" + "let's think together"; "what would you do here?" + "propose something"). The evals README warns against copying the description because it games the trigger check: the TF-IDF ranks
interview-mefirst largely because those tokens were just added to its description, so "both new prompts rank first" is partly tautological rather than independent evidence. Suggest rephrasing at least one to express the collaborative intent without the catchphrases (e.g. "I keep flip-flopping on this, weigh in with what you would pick and why") so it exercises organic routing.
Note, not a change request
The behavioral eval is provisional with no fixtures, so CI only checks lexical routing, not the mode's actual contract (no turn-1 proposal, lean+risk, origin tags, anti-anchoring). That is inherent to tier 3 until fixtures land (#352), just worth not citing that eval as evidence the mode behaves correctly yet.
Net: technically mergeable, faithful to the design. The two adjustments above are small. Nice work.
Review on addyosmani#360 flagged that both new positive prompts embedded the exact catchphrases just added to the description, making their rank-1 result partly tautological. Reword the first to organic phrasing; the second still exercises the explicit catchphrase invocations, which users genuinely say. Rank-1 rate holds at 85% (63/74). Co-authored-by: Cursor <cursoragent@cursor.com>
|
Both adjustments are in.
Second trigger intentionally keeps the catchphrases: it now serves as the check that the explicit invocations route as intended, while the first provides the organic evidence. Rank-1 rate holds at 85% (63/74). On the behavioral eval: agreed — it's |
|
Really like this. Collaborative mode fills the gap where the user wants an interlocutor rather than an interrogator, and keeping it opt-in with explicit entry phrases means it can't hijack the default extraction behavior. The origin-tagged restate and the anti-anchoring check are the thoughtful touches. This delivers something I've had on the roadmap, so I want to read the new section end to end and then land it. |
|
Did the end-to-end read I promised, and it holds up well. The three entry paths (explicit, promote-from-flow, recover-from-delegation), the differential-confidence rule, and the don't-propose-in-the-user's-domain guardrail are the thoughtful bits, and keeping it strictly opt-in with proposals never on turn 1 is what stops it wrecking the default extraction behavior. Structurally clean, CI's green. One coordination thing, not a flaw in this PR: your eval2 is provisional with no fixture, which is fine today but collides with #381 (the eval-graduation PR) the moment it lands, since that flips provisional evals to a hard CI error. interview-me is genuinely awkward to fixture since it's a dialogue, so let's either add a light fixture here or settle the conversation-skill exemption in #381 first, then merge these in the right order. Content-wise I'm ready to take it. |
|
Quick unblock for the eval collision I flagged: I've proposed a |
addyosmani flagged on addyosmani#360 that the new behavioral eval's provisional/no-fixture state collides with addyosmani#381, which promotes provisional evals to a hard CI error. addyosmani#381 now ships a kind: "dialogue" exemption for conversation-shaped skills, so adopt it here: eval 2 becomes kind: "dialogue" and drops trust_level. The current runner ignores the kind field, so this is green under both regimes and the two PRs can merge in either order. Eval 1 is left untouched because addyosmani#381 converts it on main; editing it here would conflict. 122 checks pass, rank-1 rate unchanged at 85%. Co-authored-by: Cursor <cursoragent@cursor.com>
|
Adopted the |
Summary
Implements the collaborative mode proposed in #251 inside the existing
interview-meskill. The skill's default remains extraction-first — one question at a time, guess attached, no option-listing. What's new is an opt-in mode for when the user wants the agent as an interlocutor rather than an interviewer: the agent may propose, with an explicit lean and risk attached, and the ~95% confidence threshold is reached through consensus instead of pure extraction.Concretely, following the design in #251:
PROPOSAL / WHY / RISK / LEAN / ASKINGformat, the 2–3-option variant with a mandatory lean, one proposal per turn, and the expertise-deference rule.evals/cases/interview-me.json) gains two positive triggers and one behavioral eval asserting the mode's contract (no turn-1 proposal, lean+risk present, origin-tagged restate, anti-anchoring check). One trigger uses organic phrasing ("I keep flip-flopping on what I actually want here — weigh in with your own proposal and the risks you see"); the other exercises the explicit catchphrase invocations.Two deliberate deviations from the issue text:
grill-meskill for the boundary discussion; this repo's equivalent isdoubt-driven-development, so the boundary line was added there ("plan written → doubt-driven review; intent not fixed → collaborative interview").interview-me --collaborativeflag invocation; this implementation keeps only the natural-language synonyms, since skills in this repo are invoked by intent matching, not flags.Non-duplication check
interview-me#251. The only open PR touchingskills/interview-me/SKILL.mdis feat: replace plain-text confirmations with question tool in 10 skills #195 (replaces plain-text confirmations with the question tool across 10 skills) — different scope; worst case is a small merge conflict in Step 5.idea-refineis downstream divergence,doubt-driven-developmentis post-plan review. The mode is added insideinterview-meprecisely to avoid a near-duplicate skill, per CONTRIBUTING's "prefer a focused edit over a new directory."Test plan
node scripts/validate-skills.js→ 24 skills checked — 0 errors, 0 warnings — PASSEDnode scripts/run-evals.js→ 122 checks passed — 0 errors, 0 warnings; trigger rank-1 rate 85% (63/74) — both new positive triggers rankinterview-mefirst (rate unchanged from the 85% baseline, 61/72 before). The organic-phrasing trigger ranking first is the meaningful signal here; the catchphrase trigger confirms the explicit invocations route as intended. The behavioral eval isprovisional(no fixtures yet), so it documents the mode's contract rather than evidencing it.SKILL.mdis 311 lines, under the 500-line cap; frontmatter valid.Closes #251.