Skip to content

Merge mixed speculative decoding points on agentic curves - #695

Open
cquil11 wants to merge 4 commits into
masterfrom
fix/agentic-mixed-spec-decode
Open

Merge mixed speculative decoding points on agentic curves#695
cquil11 wants to merge 4 commits into
masterfrom
fix/agentic-mixed-spec-decode

Conversation

@cquil11

@cquil11 cquil11 commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Treat speculative decoding as point-level metadata for agentic_traces, so one hardware/framework curve can contain MTP, EAGLE, or standard-decoding points.
  • Keep fixed-sequence behavior unchanged: speculative decoding remains part of those curve keys.
  • Align agentic availability and workflow-run filtering with the merged series key.
  • Preserve spec_method on every point, include it in point-level identity, and label each agentic point as STP, MTP, or its other speculative method.
  • Keep each merged agentic series within one workflow run, including latest charts, date comparisons, history, replay, AI charts, and AgentX overview scoring.
  • Enable agentic Performance Over Time and replay history without numeric ISL/OSL, while keeping tracked MTP and standard-decoding points distinct.
  • Align AgentX overview links and changelog matching with the merged curve identity, and version affected persistent caches.
  • Works for both official runs and ?unofficialrun= overlays; verified locally through the shared transform/tooltip regression tests and Cypress overlay coverage, and on the Vercel preview with both MTP and standard-decoding point tooltips.

Root cause

The dashboard used spec_method in the hardware key for every scenario. That assumed an entire curve used one decode method, so mixed AgentX submissions were split into separate legend lines and availability/run filters continued to treat them as distinct series.

Main InferenceX repository assessment

No structural master.yaml change is required in SemiAnalysisAI/InferenceX. Its current agentic schema already defines spec-decoding on each agentic-coding.search-space entry. The matrix generator propagates the value per job, launchers route from the per-job value, and single-node and multi-node artifact filenames already include _spec-${SPEC_DECODING}. A mixed curve can therefore be submitted under one top-level agentic config with different spec-decoding values on individual search-space entries. The spec suffix must remain in job and artifact filenames because it prevents collisions; this PR removes it only from agentic display-series identity.

Reference: Slack discussion

Validation

  • bun run lint
  • bun run fmt
  • bun run typecheck
  • bun run test:unit — 3,745 tests passed
  • Cypress component smoke — 22 tests passed
  • Cypress integration smoke — 63 tests passed, including unofficial-run overlay coverage
  • bun run build — passed
  • bun run security — reports the repository's existing 20 lockfile advisories; this PR changes no dependencies

Note

Medium Risk
Touches core chart identity, deduplication, and history API paths across inference, overview, and DB queries; incorrect merging could mis-attribute agentic points or drop comparison/history data.

Overview
Agentic traces now treat speculative decoding as point-level metadata instead of splitting curves by spec_method. getHardwareKey, availability keys, default precision counting, overview/AgentX grouping, and changelog legend matching all omit spec suffixes for agentic_traces while fixed-sequence behavior stays the same.

Shared point-identity helpers keep scatter, replay, and “track over time” distinct for overlapping STP/MTP points; agentic scatter labels and tooltips show decode mode. New benchmark-run-selection dedupes agentic series by workflow run (latest chart, history, comparisons, replay, AI charts).

The benchmark history API accepts benchmarkType=agentic_traces without ISL/OSL; DB/history queries return workflow_run_id / run_started_at. Cache keys bump to benchmark-history-v2 and workflow-info-v2.

Reviewed by Cursor Bugbot for commit ee605cb. Bugbot is set up for automated code reviews on this repo. Configure here.

Treat speculative decoding as point-level metadata for agentic scenarios while retaining fixed-sequence curve identity. Keep availability and run selection aligned, preserve overlapping point identity, and expose the method in bilingual tooltips.

中文:修复(推理):合并智能体场景中的投机解码点。智能体曲线不再按投机解码方式拆分,同时保持定长场景的现有曲线标识;同步可用性与运行筛选,避免重叠点丢失,并在中英文 tooltip 中展示每个点的投机解码方式。
@cquil11
cquil11 requested a review from adibarra as a code owner August 7, 2026 21:31
@vercel

vercel Bot commented Aug 7, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
inferencemax-app Ready Ready Preview Aug 7, 2026 10:21pm

Request Review

Comment thread packages/app/src/components/inference/InferenceContext.tsx
Keep performance-over-time selections and historical matches distinct for agentic MTP and standard-decoding points while preserving fixed-sequence behavior.\n\n中文:区分智能体场景中被跟踪的投机解码模式,避免 Performance Over Time 将相同拓扑的 MTP 与标准解码点合并,同时保持定长场景行为不变。
@cquil11 cquil11 changed the title Merge mixed speculative decoding points on agentic curves / 合并智能体曲线中的混合投机解码点 Merge mixed speculative decoding points on agentic curves Aug 7, 2026
Address independent review findings across run selection, history, overview, tracking, replay, labels, and changelog matching while preserving fixed-sequence behavior.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 4bf754c. Configure here.

Comment thread packages/app/src/hooks/api/use-ai-chart.ts Outdated
Apply the existing agentic latest-run selector to live AI chart benchmark data so mixed decode curves cannot combine workflows.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant