Skip to content

feat(data): preserve additive append-only recipe variants - #743

Merged
cquil11 merged 2 commits into
masterfrom
agent/append-only-ingest
Aug 14, 2026
Merged

feat(data): preserve additive append-only recipe variants#743
cquil11 merged 2 commits into
masterfrom
agent/append-only-ingest

Conversation

@cquil11

@cquil11 cquil11 commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

Summary

  • assemble a logical curve from consecutive same-image append runs plus the nearest full snapshot
  • preserve each point's producer date, run, URL, and image while exposing separate logical curve identity fields
  • add nullable recipe_fingerprint storage and identity so two additive recipes at the same topology and concurrency cannot overwrite each other
  • propagate recipe identity through ingestion, latest/exact/history queries, APIs, overlays, charts, replay, and audited purge handling
  • apply logical snapshots consistently to chart, Overview, calculator, exact-run, as-of, and historical consumers
  • preserve legacy rows with a null fingerprint and keep fingerprint out of visual-series grouping

Why

Append-only producer runs contain only their new rows. Without logical snapshot assembly, the client treats that partial run as a new curve and drops inherited points. Without recipe identity, additive variants that share the App's prior coordinates can collide in ingestion or later consumers.

This PR separates:

  • producer identity: where an individual point came from
  • curve identity: which logical snapshot the point belongs to
  • recipe identity: which generated benchmark recipe produced the point

Companion PRs

Live preview

Vercel team sign-in may be required because preview protection remains enabled.

The preview uses disposable Neon branch append-only-poc-20260814 (br-tiny-queen-aib16lin), which automatically expires at 2026-08-15 21:19:54 UTC.

End-to-end verification

  • migrations 011 and 012 executed successfully against the disposable Neon branch
  • GitHub sweep 31842466187 scheduled only B200 SGLang AgentX Qwen 3.5 TP4 concurrency 6
  • measured profile duration: 303.07 seconds
  • ingestion wrote one benchmark row and one append-only changelog entry
  • the deployed exact-run API returns the concurrency-6 point plus inherited points from both producer runs
  • the filtered curve page and point-detail page both return successfully
  • a transactional seeded test also verified a new TP8 recipe and a same-config, same-concurrency fingerprint variant without collapsing either point

Test verification

@vercel

vercel Bot commented Aug 14, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
inferencemax-app Ready Ready Preview Aug 14, 2026 11:09pm

Request Review

@blacksmith-sh

blacksmith-sh Bot commented Aug 14, 2026

Copy link
Copy Markdown

Found 32 test failures on Blacksmith runners:

Failures

Test View Logs
Inference CSV export with an unofficial-run overlay/
"before all" hook for "exports second-based latency headers and only currently visible
overlay rows"
View Logs
Inference CSV export with an unofficial-run overlay/
"before all" hook for "exports second-based latency headers and only currently visible
overlay rows"
View Logs
Official legend X works while an unofficial overlay is loaded/
"before all" hook for "shows official points and an official legend entry initially"
View Logs
Official legend X works while an unofficial overlay is loaded/
"before all" hook for "shows official points and an official legend entry initially"
View Logs
Overlay points follow canonical Optimal Only policy (agentic interactivity)/
"before all" hook for "hides trace-less overlay points in the default Optimal Only view
"
View Logs
Overlay points follow canonical Optimal Only policy (agentic interactivity)/
"before all" hook for "hides trace-less overlay points in the default Optimal Only view
"
View Logs
X-Axis Mode Toggle (inference chart)/
follows the percentile selector in the Interactivity axis label
View Logs
X-Axis Mode Toggle (inference chart)/
follows the percentile selector in the Interactivity axis label
View Logs
X-Axis Mode Toggle (inference chart)/
keeps the Advanced selection when the remaining tab is focused
View Logs
X-Axis Mode Toggle (inference chart)/
keeps the Advanced selection when the remaining tab is focused
View Logs

...and 22 more test failures. View all on Blacksmith

Fix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 9f82414. Configure here.

Comment thread packages/db/migrations/011_append_only_curves.sql
@cquil11 cquil11 changed the title feat(data): merge append-only benchmark runs feat(data): preserve additive append-only recipe variants Aug 14, 2026
@cquil11
cquil11 merged commit b367003 into master Aug 14, 2026
17 of 24 checks passed
@cquil11
cquil11 deleted the agent/append-only-ingest branch August 14, 2026 23:32
Oseltamivir added a commit that referenced this pull request Aug 15, 2026
#736 removed E2E normalized-interactivity gating on purpose — "compute each
Pareto frontier from its selected axes without E2E normalized-interactivity
gating" — and shipped with "full checks intentionally skipped for this focused
change". Master went red: unit tests and e2e suites still asserted the gating.
This updates the tests to the behaviour that was chosen. No src behaviour
changes.

Unit:

- overview-data: an agentic point dominated on total tokens and slower on E2E is
  no longer dropped, so the tier-50 read lands on it (8100) instead of coming
  back null.
- useThroughputData: buildGpuGroups keeps an e2e-dominated point, because
  eligibility is now decided downstream from the selected axes.

The api-route-catalog digest for overview-data.ts was part of this fix until the
rebase onto #743, which changed that file again and re-reviewed the entry. It is
no longer touched here.

E2E:

- The "Advanced" x-axis popover is gone and all four modes are flat tabs, so the
  four specs that opened it now click the tab. Agentic also defaults to
  Interactivity rather than E2E Normalized Interactivity.
- The derived-metrics stub moves to beforeEach. Cypress clears intercepts
  between tests, and these specs relied on the old default fetching during
  `before` while the stub was alive; React Query then held the result for the
  spec. Nothing fetches on load now, so a later switch into that mode issued a
  live request and rendered no chart.
- The URL-restored-mode regression test now restores TTFT. It asked for
  Interactivity, which is the new default, so it could no longer fail.
- overlay-optimal-only asserted that overlay points are hidden because
  trace-less rows cannot join the canonical frontier. They now stand or fall on
  the selected axes like any other point, and all five fixture configs are
  non-dominated there — throughput falls monotonically as interactivity rises —
  so the suite would have asserted five visible either way. It gets a sixth,
  deliberately dominated overlay config, so Optimal Only has something to drop;
  making that point non-dominated fails the test.

Verified on b367003: 3372 unit tests, 600/602 e2e locally, and the four repaired
specs 30/30 at retries=0. The two local e2e failures are inference-replay, which
asserts a scrubber advances after a wall-clock cy.wait(800); it imports nothing
touched here and passes in CI.

中文:#736 有意移除了端到端归一化交互性的前沿门控,并"刻意跳过完整检查",导致 master
转红:单测与 e2e 套件仍在断言已被移除的门控。本提交只把测试对齐到既定行为,不改动任何
源码行为。单测:概览档位读数不再因总 token 被支配而丢弃该点(改为 8100);
buildGpuGroups 保留被 e2e 支配的点。overview-data.ts 的 API 文档摘要原本也在本次修复
范围内,但在 rebase 到 #743 后,该 PR 已再次修改此文件并重新复核了该条目,故此处不再
改动。E2E:Advanced 弹出菜单已移除,四个模式改为平铺标签页;派生指标 stub 移至
beforeEach(Cypress 会在测试间清除拦截);URL 恢复模式的回归测试改用 TTFT(原用
Interactivity 已与新默认值相同而无法失败);overlay-optimal-only 新增一个刻意被支配的
配置,否则该断言无论过滤器如何都会通过。已在 b367003 上验证:单测 3372 项通过,本地
e2e 600/602,四个修复套件在 retries=0 下 30/30。
Oseltamivir added a commit that referenced this pull request Aug 15, 2026
#736 removed E2E normalized-interactivity gating on purpose — "compute each
Pareto frontier from its selected axes without E2E normalized-interactivity
gating" — and shipped with "full checks intentionally skipped for this focused
change". Master went red: unit tests and e2e suites still asserted the gating.
This updates the tests to the behaviour that was chosen. No src behaviour
changes.

Unit:

- overview-data: an agentic point dominated on total tokens and slower on E2E is
  no longer dropped, so the tier-50 read lands on it (8100) instead of coming
  back null.
- useThroughputData: buildGpuGroups keeps an e2e-dominated point, because
  eligibility is now decided downstream from the selected axes.

The api-route-catalog digest for overview-data.ts was part of this fix until the
rebase onto #743, which changed that file again and re-reviewed the entry. It is
no longer touched here.

E2E:

- The "Advanced" x-axis popover is gone and all four modes are flat tabs, so the
  four specs that opened it now click the tab. Agentic also defaults to
  Interactivity rather than E2E Normalized Interactivity.
- The derived-metrics stub moves to beforeEach. Cypress clears intercepts
  between tests, and these specs relied on the old default fetching during
  `before` while the stub was alive; React Query then held the result for the
  spec. Nothing fetches on load now, so a later switch into that mode issued a
  live request and rendered no chart.
- The URL-restored-mode regression test now restores TTFT. It asked for
  Interactivity, which is the new default, so it could no longer fail.
- overlay-optimal-only asserted that overlay points are hidden because
  trace-less rows cannot join the canonical frontier. They now stand or fall on
  the selected axes like any other point, and all five fixture configs are
  non-dominated there — throughput falls monotonically as interactivity rises —
  so the suite would have asserted five visible either way. It gets a sixth,
  deliberately dominated overlay config, so Optimal Only has something to drop;
  making that point non-dominated fails the test.

Verified on b367003: 3372 unit tests, 600/602 e2e locally, and the four repaired
specs 30/30 at retries=0. The two local e2e failures are inference-replay, which
asserts a scrubber advances after a wall-clock cy.wait(800); it imports nothing
touched here and passes in CI.

中文:#736 有意移除了端到端归一化交互性的前沿门控,并"刻意跳过完整检查",导致 master
转红:单测与 e2e 套件仍在断言已被移除的门控。本提交只把测试对齐到既定行为,不改动任何
源码行为。单测:概览档位读数不再因总 token 被支配而丢弃该点(改为 8100);
buildGpuGroups 保留被 e2e 支配的点。overview-data.ts 的 API 文档摘要原本也在本次修复
范围内,但在 rebase 到 #743 后,该 PR 已再次修改此文件并重新复核了该条目,故此处不再
改动。E2E:Advanced 弹出菜单已移除,四个模式改为平铺标签页;派生指标 stub 移至
beforeEach(Cypress 会在测试间清除拦截);URL 恢复模式的回归测试改用 TTFT(原用
Interactivity 已与新默认值相同而无法失败);overlay-optimal-only 新增一个刻意被支配的
配置,否则该断言无论过滤器如何都会通过。已在 b367003 上验证:单测 3372 项通过,本地
e2e 600/602,四个修复套件在 retries=0 下 30/30。
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant