feat(commands): improve learn-eval with checklist-based holistic verdict#385
feat(commands): improve learn-eval with checklist-based holistic verdict#385
Conversation
Replace the 5-dimension numeric scoring rubric with a checklist + holistic verdict system (Save / Improve then Save / Absorb into [X] / Drop). Key improvements: - Explicit pre-save checklist: grep skills/ for duplicates, check MEMORY.md, consider appending to existing skills, confirm reusability - 4-way verdict instead of binary save/don't-save: adds "Absorb into [X]" to prevent skill file proliferation, and "Improve then Save" for iterative refinement - Verdict-specific confirmation flows tailored to each outcome - Design rationale explaining why holistic judgment outperforms numeric scoring with modern frontier models
Change h4 (####) to h3 (###) for sub-steps 5a and 5b to comply with heading increment rule (headings must increment by one level at a time).
|
Caution Review failedPull request was closed or merged during review 📝 WalkthroughWalkthroughTwo CI workflow files add npm dependency installation steps for validation jobs, documentation is updated to replace numeric evaluation rubrics with a holistic verdict checklist system, and the ajv dev dependency is updated from v8.17.0 to v8.18.0. Changes
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Superseded by #360, which is now merged on main. Closing this duplicate follow-up to keep review surface clean. |
Summary
/learn-evalholistic-verdict improvements from#360Improve then Saveconfirmation branchmainand include the CI validator dependency fix so the branch validates on current runnersSupersedes
Test plan
npm ci --ignore-scriptsnode scripts/ci/validate-agents.jsnode scripts/ci/validate-hooks.jsnode scripts/ci/validate-commands.jsnode scripts/ci/validate-skills.jsnode scripts/ci/validate-rules.jsSummary by cubic
Upgrades
/learn-evalto a checklist-based, holistic verdict system that reduces duplicate skills and guides where and how to save knowledge. Improves reliability of CI validators on current runners.New Features
MEMORY.md, and consider appending to existing skills.Dependencies
npm ci --ignore-scripts) before running validators.ajvto^8.18.0.Written for commit c6e7552. Summary will update on new commits.
Summary by CodeRabbit
Release Notes
Documentation
Chores