test: realign the AgentX frontier tests with #736 / 将 AgentX 前沿相关测试对齐 #736 - #742
Merged
Merged
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This was referenced Aug 14, 2026
Oseltamivir
force-pushed
the
fix/agentx-frontier-test-drift
branch
from
August 15, 2026 06:00
0b03fd9 to
76f6f4f
Compare
#736 removed E2E normalized-interactivity gating on purpose — "compute each Pareto frontier from its selected axes without E2E normalized-interactivity gating" — and shipped with "full checks intentionally skipped for this focused change". Master went red: unit tests and e2e suites still asserted the gating. This updates the tests to the behaviour that was chosen. No src behaviour changes. Unit: - overview-data: an agentic point dominated on total tokens and slower on E2E is no longer dropped, so the tier-50 read lands on it (8100) instead of coming back null. - useThroughputData: buildGpuGroups keeps an e2e-dominated point, because eligibility is now decided downstream from the selected axes. The api-route-catalog digest for overview-data.ts was part of this fix until the rebase onto #743, which changed that file again and re-reviewed the entry. It is no longer touched here. E2E: - The "Advanced" x-axis popover is gone and all four modes are flat tabs, so the four specs that opened it now click the tab. Agentic also defaults to Interactivity rather than E2E Normalized Interactivity. - The derived-metrics stub moves to beforeEach. Cypress clears intercepts between tests, and these specs relied on the old default fetching during `before` while the stub was alive; React Query then held the result for the spec. Nothing fetches on load now, so a later switch into that mode issued a live request and rendered no chart. - The URL-restored-mode regression test now restores TTFT. It asked for Interactivity, which is the new default, so it could no longer fail. - overlay-optimal-only asserted that overlay points are hidden because trace-less rows cannot join the canonical frontier. They now stand or fall on the selected axes like any other point, and all five fixture configs are non-dominated there — throughput falls monotonically as interactivity rises — so the suite would have asserted five visible either way. It gets a sixth, deliberately dominated overlay config, so Optimal Only has something to drop; making that point non-dominated fails the test. Verified on b367003: 3372 unit tests, 600/602 e2e locally, and the four repaired specs 30/30 at retries=0. The two local e2e failures are inference-replay, which asserts a scrubber advances after a wall-clock cy.wait(800); it imports nothing touched here and passes in CI. 中文:#736 有意移除了端到端归一化交互性的前沿门控,并"刻意跳过完整检查",导致 master 转红:单测与 e2e 套件仍在断言已被移除的门控。本提交只把测试对齐到既定行为,不改动任何 源码行为。单测:概览档位读数不再因总 token 被支配而丢弃该点(改为 8100); buildGpuGroups 保留被 e2e 支配的点。overview-data.ts 的 API 文档摘要原本也在本次修复 范围内,但在 rebase 到 #743 后,该 PR 已再次修改此文件并重新复核了该条目,故此处不再 改动。E2E:Advanced 弹出菜单已移除,四个模式改为平铺标签页;派生指标 stub 移至 beforeEach(Cypress 会在测试间清除拦截);URL 恢复模式的回归测试改用 TTFT(原用 Interactivity 已与新默认值相同而无法失败);overlay-optimal-only 新增一个刻意被支配的 配置,否则该断言无论过滤器如何都会通过。已在 b367003 上验证:单测 3372 项通过,本地 e2e 600/602,四个修复套件在 retries=0 下 30/30。
Oseltamivir
force-pushed
the
fix/agentx-frontier-test-drift
branch
from
August 15, 2026 06:01
76f6f4f to
6094873
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
masterwent red at #736. That PR removed E2E normalized-interactivity frontier gating on purpose — its own summary says "compute each Pareto frontier from its selected axes without E2E normalized-interactivity gating" — and merged with "full checks intentionally skipped for this focused change". The tests asserting the old gating were never updated.Bisected:
08e1f7dgreen,7fded35(#736) red. This PR changes nosrcbehaviour — it moves the tests to the behaviour that was chosen. Two later PRs (#737, #743) hit the same failures and merged past them.Unit
overview-data8100) instead ofnulluseThroughputDatabuildGpuGroupskeeps an e2e-dominated point; eligibility is decided downstream from the selected axesThe
overview-data.tsAPI-doc digest was part of this PR until the rebase. #743 changed that file again and re-reviewed the entry itself, soapi-route-catalog.tsis no longer touched here.E2E
beforeEach. Cypress clears intercepts between tests; these specs relied on the old default fetching duringbeforewhile the stub was alive, after which React Query held the result for the whole spec. Nothing fetches on load anymore, so a later switch into that mode issued a live request and rendered no chart at all.overlay-optimal-onlyneeded a new fixture point. It asserted overlay points are hidden because trace-less rows cannot join the canonical frontier. They now stand or fall on the selected axes like any other point, and all five fixture configs are non-dominated there — throughput falls monotonically as interactivity rises — so the suite would have asserted five visible whatever the filter did. It gets a sixth, deliberately dominated overlay config (DOMINATED_CONFIG), giving Optimal Only something to drop.interceptOverlayRun's newoverlayConfigsoption defaults toREAL_CONFIGS, so the other four consumers are untouched.Verification (re-run on
b367003)retries=0, re-run after rebuilding against feat(data): preserve additive append-only recipe variants #743 — which touched overlay and chart consumers, so this was re-verified rather than assumed.DOMINATED_CONFIGnon-dominated failsdrops only the dominated overlay point…(1 visible vs 5 expected).lint/fmt/typecheckclean.The two local e2e failures are not from this change:
inference-replayasserts a scrubber advances after a wall-clockcy.wait(800)— it imports nothing touched here and passed CI. (zh-pagesfailed once under parallel load and passes on re-run.)Note for @cquil11
#736 left
restrictAgenticPointsToE2eFrontierin the tree with zero callers, and its own tests still pass, so it now reads as live policy. Separately,MODELS.mdin the InferenceX repo still documents the North-star Pareto policy as binding ("a point cannot appear on any AgentX Pareto frontier unless it is both a North Star winner and non-dominated on the selected chart"), which the app no longer enforces. Neither is touched here — deleting dead code and reconciling that doc are your calls, not a test fix's.中文说明
已 rebase 到
b367003(#743 之后)。#743 的说明中亦将本 PR 称为"针对该漂移的纯测试对齐修复"。master自 #736 起转红。该 PR 有意移除了端到端归一化交互性的前沿门控(其摘要即为"各 Pareto 前沿仅依据所选坐标轴计算,不再受端到端归一化交互性门控约束"),且合并时"刻意跳过完整检查",但断言旧门控的测试未随之更新。已二分定位:08e1f7d通过,7fded35(#736)失败。本 PR 不改动任何src行为,只是把测试对齐到既定行为;此后 #737 与 #743 均遇到同一组失败并带红合并。单测:概览档位读数不再因总 token 被支配且 E2E 较慢而丢弃该点,tier-50 读数落在
8100而非null;buildGpuGroups保留被 e2e 支配的点。overview-data.ts的 API 文档摘要原本也在本 PR 范围内,但 #743 已再次修改该文件并重新复核了该条目,故api-route-catalog.ts在此不再改动。E2E:Advanced 弹出菜单已移除,四个模式改为平铺标签页,四个原先打开该菜单的套件改为直接点击标签页;派生指标 stub 移至
beforeEach(Cypress 会在测试间清除拦截,这些套件此前依赖旧默认值在before期间完成抓取并由 React Query 缓存,现在加载时不再抓取,后续切换会发出真实请求并导致图表无法渲染);URL 恢复模式的回归测试改用 TTFT;overlay-optimal-only新增一个刻意被支配的配置DOMINATED_CONFIG,否则五个真实配置在交互性坐标轴上均非被支配,该断言无论过滤器如何都会通过。interceptOverlayRun新增的overlayConfigs默认为REAL_CONFIGS,其余四处调用不受影响。验证(已在
b367003上重跑):单测 3372 项全部通过;本地 e2e 600/602,四个修复套件在retries=0下 30/30——因 #743 改动了叠加层与图表消费方,此处为重新构建后实测而非沿用旧结论;新断言已通过变异测试验证;lint/fmt/typecheck 均通过;rebase 前的版本在 CI 全部 8 个浏览器分片上均为绿色。两项本地失败与本 PR 无关:inference-replay依赖真实时钟的动画断言且未引用本 PR 改动的任何内容,在 CI 中通过(zh-pages仅在并行负载下偶发超时,重跑通过)。另请 @cquil11 注意:#736 之后
restrictAgenticPointsToE2eFrontier仍留在代码中但已无任何调用方,其自身测试仍通过,容易被误读为仍在生效的策略;此外 InferenceX 仓库的MODELS.md仍将 North-star Pareto 政策记为强制约束,而应用已不再执行。这两点本 PR 均未触碰——删除死代码与同步该文档应由你决定。Note
Low Risk
Only test and Cypress fixture changes; production code is untouched.
Overview
Test-only alignment after #736 removed E2E normalized-interactivity frontier gating and reshaped the agentic chart UI. No
srcbehavior changes in this PR.Unit tests now expect Pareto and tier logic on the selected axes only:
buildGpuGroupskeeps e2e-dominated agentic points, and overview AgentX tier reads can land on points that are dominated on total tokens / slower on E2E (8100instead ofnull).Cypress drops the Advanced popover flow—four x-axis modes are flat tabs; agentic defaults to Interactivity (not E2E Normalized Interactivity). Derived-metrics intercepts move to
beforeEachbecause the default no longer fetches on load and Cypress clears intercepts between tests. Overlay specs assert Optimal Only hides only dominated overlay points (not all trace-less overlays);DOMINATED_CONFIGandoverlayConfigsoninterceptOverlayRunmake that test meaningful. URL-restore regression uses TTFT so it can still detect clobbering against the new default.Reviewed by Cursor Bugbot for commit 6094873. Bugbot is set up for automated code reviews on this repo. Configure here.