[Task]: GH-C16 public-safe non-SkillsBench trajectory summary

[liubf21]

Task ID or area

GH-C16 / benchmark evidence

Intent

I want to claim an existing task.

Summary

Add one public-safe public_trajectory_summary_v0 path for a non-SkillsBench benchmark adapter so case analysis can compare compact lifecycle counters without retaining raw task text, logs, verifier output, or trajectory bodies.

Proposed scope

In scope:

• target exactly one existing non-SkillsBench adapter on main;
• derive and emit the existing typed summary schema from compact adapter lifecycle facts;
• add deterministic fake-fixture coverage for expected and malformed/missing lifecycle input;
• document the resulting public-safe contract where the selected adapter already documents its result projection.

Out of scope:

• live benchmark runs, task launches, scoring, uploads, or submissions;
• raw trajectory/task/log/verifier retention;
• synthetic aggregate scoring or a new benchmark authority;
• changes to existing adapter task semantics or runner behavior.

Relevant files or commands

• loopx/benchmark_trajectory.py
• selected adapter under loopx/benchmark_adapters/
• examples/benchmark-case-analysis-smoke.py
• examples/benchmark-core-adapter-contract-smoke.py

Validation plan

• focused unit or deterministic fake-fixture smoke for the selected adapter;
• python3 examples/benchmark-case-analysis-smoke.py;
• python3 examples/benchmark-core-adapter-contract-smoke.py when the lifecycle seam is touched;
• loopx check over touched benchmark and documentation paths.

Public/private boundary

• This issue contains no private benchmark traces, verifier output, credentials, internal document links, raw agent sessions, or local runtime state.
• I will not run or duplicate maintainer-owned benchmark cases unless a maintainer explicitly splits out a public task.

[liubf21]

Baseline check found that GH-C16's stated implementation seam is no longer active on current main:

• commit 76fe5bfe / PR refactor: reset benchmark research around native runners #3267 moved loopx/benchmark_trajectory.py and the non-SkillsBench adapter stack to deprecate/benchmark-legacy/;
• current benchmark/ now contains the native-runner path, and no active public_trajectory_summary_v0 consumer remains outside the archived stack;
• the contributor task board still lists GH-C16.

I am pausing implementation rather than adding new behavior to archived code. Could a maintainer confirm one of the following?

1. close GH-C16 as superseded; or
2. define its smallest active-native-runner successor (including the public-safe observation schema, selected adapter, and validation surface).

No benchmark run, scoring change, raw trajectory access, or private artifact was performed.

[huangruiteng]

我这两天会把跑bench的一些基础toolkit上传上去，到时候可以综合看这方面能力的增量价值。我和一位zju phd开发者都在跑bench，感兴趣可拉个小群交流

[huangruiteng]

老的benchmark代码是gpt5.4写的，太乱废弃了，参考最新的benchmark capability，我们从头建设

[liubf21]

我这两天会把跑bench的一些基础toolkit上传上去，到时候可以综合看这方面能力的增量价值。我和一位zju phd开发者都在跑bench，感兴趣可拉个小群交流

可以的，目前跑了哪些 bench 呀

[yuefengw]

老的benchmark代码是gpt5.4写的，太乱废弃了，参考最新的benchmark capability，我们从头建设

带我一个请求加入🙋

[huangruiteng]

你们联系方式是什么呀，可以加我vx huangrt00，或者飞书 ***@***.***。飞书和vx都有群，vx有一些学术界benchmark交流 | | huangrt01 | | ***@***.*** | ---- 回复的原邮件 ---- | 发件人 | ***@***.***> | | 发送日期 | 2026年08月17日 21:02 | | 收件人 | ***@***.***> | | 抄送人 | huangruiteng ***@***.***>, Comment ***@***.***> | | 主题 | Re: [huangruiteng/loopx] [Task]: GH-C16 public-safe non-SkillsBench trajectory summary (Issue #3282) | yuefengw left a comment (huangruiteng/loopx#3282) 老的benchmark代码是gpt5.4写的，太乱废弃了，参考最新的benchmark capability，我们从头建设 带我一个请求加入🙋 — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: ***@***.***>

[ktwu01]

开源给大家随意使用哈哈哈：一键导出全网4861 条 benchmark/eval/dataset 数据:
我从 GitHub、Hugging Face等 11 个来源，持续抓了25天，持续更新 https://github.com/ktwu01/benchmark-radar
家人们如果需要寻找 related work 或找到适合eval自己的agent 的 bench 或者关注最新的 eval 进展，可以看这里 哈哈哈
也新增了 rss 订阅方式，用rss来看每天的benchmark日报