feat(reward-memory): attribute post-outcome utility - #3280
Open
yuefengw wants to merge 2 commits into
Open
Conversation
Signed-off-by: yuefengw <60574042+yuefengw@users.noreply.github.com>
Signed-off-by: yuefengw <60574042+yuefengw@users.noreply.github.com>
huangruiteng
approved these changes
Aug 17, 2026
huangruiteng
left a comment
Owner
There was a problem hiding this comment.
Approval conclusion (community PR; exact head reviewed)
详细中文评审:post-outcome 记忆效用归因 Stage 1(PR #3280)
评审目标 head:5fee3c676f3072ccd35c45b6706a0b47dbd45498(base: main 0deff62a0,10 文件,+2074/-122;Refs #3214,实现 Stage 1 契约)
改动范围与结构
新增 loopx/capabilities/reward_memory/memory_utility.py(memory_utility_observation_v0 契约、校验与构建),并把 dogfood 输出从 v0 升级到 v1:application_disposition(applied/not_applied/refuted)与 utility_evaluation(accepted/rejected/not_requested)分离,applied + success 在归因证据不足时保持 unknown,不再暗示 helpful。
application_receipt_id绑定一次应用/结果结算,evaluator 缺席、拒绝、重试或新证据不会重复 disposition 与成本指标。observation_id是稳定身份字段(scope、application_receipt_id、memory digests、retrieval/policy snapshot、outcome_ref、attribution level/basis、evidence refs、evaluator ref/version)的规范哈希;evaluator 版本或证据变化会生成新的 observation 身份。- 边界强制为 False:
grants_new_action_authority、provider_write_performed、external_writes_performed、raw_content_captured;evaluator 默认不开启、fail-open、只能产生只读 observation。 - 拒绝本地路径、provider 精确引用、畸形 envelope、重复 application settlement、非规范 memory digest;fixture 里包含
/Users/example/...、/private/tmp/...的负例(应被拒绝)。
安全与语义边界
- 观察记录与 trusted facts(application receipt / verified outcome / attribution context)做绑定校验,归因对象必须落在 application 的 memory digests 内。
attribution_level(item/set/none)、evidence_basis(owner_correction / controlled_replay / deterministic_effect / evaluator_inference / insufficient)、confidence与reason_codes都是 typed 字段,弱推断不会自动升级为强结论。- 明确不在本 PR:Stage 2 reducer/只读投影、检索排序、scheduler/quota/todo/memory 生命周期改动、OpenViking/provider rank-prior 写回、新能力或执行权威。
验证
- exact head 实跑:
uv run --extra test python -m pytest -q tests/capabilities/test_reward_memory_agent_scoped_recall.py tests/capabilities/test_reward_memory_experiment.py tests/capabilities/test_reward_memory_ingestion.py tests/capabilities/test_reward_memory_utility_attribution.py→ 98 passed(含 54 个 utility attribution 用例,覆盖applied+success→unknown、harmful 保留、观察身份稳定性、重复 settlement 拒绝等)。 python examples/reward-memory-dogfood-smoke.py→ ok。git diff --check无空白错误;变更文件未发现真实本地路径/凭据(负例 fixture 除外)。- GitHub 状态检查:5 项成功、0 失败、build/pytest 仍 pending;建议合入前确认两项跑绿。
非阻塞问题
memory_utility.py约 740 行、dogfood.py增加 458 行,属同一边界内的合理增长;若后续 Stage 2 继续膨胀,可把 schema/字段集合抽成独立契约模块。- dogfood v1 是预览输出 schema 的显式版本升级(非静默重定义),方向正确;任何外部消费方需跟随 v1 字段名。
我的整体评价
实现与 #3214 的 Stage 1 最小切片一致:disposition 与 utility 分离、稳定身份、fail-open、无权威/写回,且用 typed 字段与负例 fixture 把“applied 不等于 helpful”钉死。结论:APPROVE。合入按仓库策略执行;Stage 2(reducer + 只读投影)应复用同一 observation 契约。
English Verdict (PR #3280)
- Exact head reviewed:
5fee3c676f3072ccd35c45b6706a0b47dbd45498 - Verdict: APPROVE (Stage 1 of #3214; provider-neutral memory_utility_observation_v0; v1 dogfood receipts separate application disposition from utility evaluation)
- Key findings: application settlement identity is stable and independent of evaluator delivery state; observation identity is a canonical hash so evaluator/evidence changes create new observations; authority/provider-write/raw-content flags are forced False;
applied + success -> unknownwhen attribution evidence is insufficient. Focused validation on the exact head: 98 reward-memory tests passed plus the dogfood smoke;git diff --checkclean. Non-blocking: GitHub build/pytest were still pending at review time; consider extracting the schema/field sets into a contract module if Stage 2 grows further.
Owner
|
@yuefengw 你好,比较建议实际跑一下用用,在你的某些生产任务上体验一下 reward memory 的能力,也许有助于打磨这个 PR 的质量并合入 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
memory_utility_observation_v0Stage 1 contract under the existing Reward Memory capabilityapplied + success -> unknownwhen attribution evidence is insufficientapplied/not_applied/refutedno longer implyhelpful/harmfulutilityCompatibility and identity
This intentionally changes the preview dogfood output schema from
reward_memory_dogfood_receipt_v0/reward_memory_dogfood_batch_v0to v1 instead of silently redefininghit,miss, andrefute.The dogfood
receipt_ididentifies one application/outcome settlement and deliberately excludes evaluator delivery state, so evaluator absence, rejection, retry, or new evidence cannot duplicate disposition or cost metrics. Utility delivery has its own stableobservation_id; a changed evaluator version or new evidence creates a new observation identity.Validation
python -m pytest -q: 3228 passed, 2 skipped on a clean, non-concurrent rerunloopx canary premerge --from-git-diff --no-progress: passed; 17/17 selected canaries/boundary checks passed, 0 warnings, 0 manual holdsok=True; its only two warnings were the expected absence of the worktree-local.loopx/registry.jsonThe first full-suite run was executed while other validations were active and produced one unrelated timing failure in
tests/extensions/test_extension_runtime.py::test_extension_run_terminates_provider_on_timeout(execution_failedvstimeout). That test passed immediately in isolation, and the subsequent non-concurrent full-suite run passed as reported above.Out of scope
Refs #3214