-
Notifications
You must be signed in to change notification settings - Fork 27
feat(dashboard): Add telemetry reporting #471
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
+2,055
−446
Merged
Changes from all commits
Commits
Show all changes
17 commits
Select commit
Hold shift + click to select a range
f9e344d
feat(dashboard): Add telemetry metric tooltips
dcramer fad6c81
fix(dashboard): Address telemetry review feedback
dcramer f95aafb
fix(dashboard): Address remaining telemetry feedback
dcramer d216976
fix(dashboard): Keep duration chart labels aligned
dcramer 5fd84e8
fix(dashboard): Use structured requester identity
dcramer 80a8260
fix(dashboard): Tighten reporting contracts
dcramer 1cd8573
fix(dashboard): Align duration chart contracts
dcramer 6e6a369
fix(dashboard): Clean up metric tooltips
dcramer 910f1d4
fix(dashboard): Simplify metric hover affordances
dcramer 7467dfc
fix(dashboard): Deemphasize transcript tool events
dcramer 1bba295
fix(dashboard): Refine transcript activity rendering
dcramer 1b6a0fa
fix(dashboard): Center collapsed system prompt
dcramer af11c38
fix(dashboard): Clean up telemetry transcript types
dcramer 9034489
fix(dashboard): Share tool invocation matching
dcramer 2f185e8
fix(dashboard): Label conversation row runtime
dcramer a26c9dd
fix(dashboard): Preserve telemetry header context
dcramer d716e5c
fix(dashboard): Tighten telemetry metric rendering
dcramer File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
159 changes: 159 additions & 0 deletions
159
packages/junior-dashboard/src/client/components/Metric.tsx
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,159 @@ | ||
| import { | ||
| useId, | ||
| useRef, | ||
| useState, | ||
| type CSSProperties, | ||
| type ReactNode, | ||
| } from "react"; | ||
|
|
||
| import { cn } from "../styles"; | ||
|
|
||
| export type MetricTooltipLine = { | ||
| label?: string; | ||
| labelStyle?: "code"; | ||
| value: string; | ||
| }; | ||
|
|
||
| export type MetricListItem = { | ||
| content: ReactNode; | ||
| key: string; | ||
| }; | ||
|
|
||
| type TooltipPosition = { | ||
| left: number; | ||
| top: number; | ||
| width: number; | ||
| }; | ||
|
|
||
| function clamp(value: number, min: number, max: number): number { | ||
| return Math.min(Math.max(value, min), max); | ||
| } | ||
|
|
||
| function tooltipPosition( | ||
| trigger: HTMLElement, | ||
| align: "left" | "right" | undefined, | ||
| ): TooltipPosition { | ||
| const margin = 16; | ||
| const viewportWidth = window.innerWidth; | ||
| const width = Math.min(320, Math.max(256, viewportWidth - margin * 2)); | ||
| const rect = trigger.getBoundingClientRect(); | ||
| const preferredLeft = align === "right" ? rect.right - width : rect.left; | ||
| return { | ||
| left: Math.round( | ||
| clamp(preferredLeft, margin, viewportWidth - width - margin), | ||
| ), | ||
| top: Math.round(rect.bottom + 8), | ||
| width, | ||
| }; | ||
| } | ||
|
|
||
| /** Render compact metadata text with an optional styled hover/focus tooltip. */ | ||
| export function MetricValue(props: { | ||
| align?: "left" | "right"; | ||
| children: ReactNode; | ||
| className?: string; | ||
| tooltip?: MetricTooltipLine[]; | ||
| }) { | ||
| const tooltipId = useId(); | ||
| const triggerRef = useRef<HTMLSpanElement>(null); | ||
| const [position, setPosition] = useState<TooltipPosition | null>(null); | ||
| const tooltip = props.tooltip?.filter((line) => line.value.trim()); | ||
| if (!tooltip?.length) { | ||
| return <span className={props.className}>{props.children}</span>; | ||
| } | ||
|
|
||
| const showTooltip = () => { | ||
| if (!triggerRef.current) return; | ||
| setPosition(tooltipPosition(triggerRef.current, props.align)); | ||
| }; | ||
| const hideTooltip = () => setPosition(null); | ||
| const tooltipStyle: CSSProperties | undefined = position | ||
| ? { | ||
| left: position.left, | ||
| top: position.top, | ||
| width: position.width, | ||
| } | ||
| : undefined; | ||
|
|
||
| return ( | ||
| <span className={cn("relative inline-flex", props.className)}> | ||
| <span | ||
| aria-describedby={position ? tooltipId : undefined} | ||
| className="border-b border-dotted border-white/20 outline-none transition-colors hover:border-white/45 focus-visible:border-white/45" | ||
| onBlur={hideTooltip} | ||
| onFocus={showTooltip} | ||
| onMouseEnter={showTooltip} | ||
| onMouseLeave={hideTooltip} | ||
| ref={triggerRef} | ||
| tabIndex={0} | ||
| > | ||
| {props.children} | ||
| </span> | ||
| {position ? ( | ||
| <span | ||
| className="pointer-events-none fixed z-30 border border-white/15 bg-[#050505] px-3 py-2 text-left text-[0.76rem] font-normal leading-relaxed text-[#b8b8b8] shadow-xl shadow-black/35" | ||
| id={tooltipId} | ||
| role="tooltip" | ||
| style={tooltipStyle} | ||
| > | ||
| <span className="grid max-h-72 grid-cols-[minmax(0,1fr)_auto] gap-x-3 gap-y-1.5 overflow-y-auto"> | ||
| {tooltip.map((line, index) => ( | ||
| <span | ||
| className={ | ||
| line.label | ||
| ? "contents" | ||
| : "col-span-2 block min-w-0 break-words text-[#d6d6d6]" | ||
| } | ||
| key={`${index}-${line.label ?? ""}-${line.value}`} | ||
| > | ||
| {line.label ? ( | ||
| <span | ||
| className={cn( | ||
| "min-w-0 break-words font-medium text-[#888]", | ||
| line.labelStyle === "code" && | ||
| "break-all font-mono text-[0.74rem] text-[#d6d6d6]", | ||
| )} | ||
| > | ||
| {line.label} | ||
| </span> | ||
| ) : null} | ||
| {line.label ? ( | ||
| <span className="whitespace-nowrap text-right text-[#d6d6d6]"> | ||
| {line.value} | ||
| </span> | ||
| ) : ( | ||
| line.value | ||
| )} | ||
| </span> | ||
| ))} | ||
| </span> | ||
| </span> | ||
| ) : null} | ||
| </span> | ||
| ); | ||
| } | ||
|
|
||
| /** Render inline metadata with consistent dot spacing across dashboard headers. */ | ||
| export function MetricList(props: { | ||
| className?: string; | ||
| items: MetricListItem[]; | ||
| }) { | ||
| return ( | ||
| <div | ||
| className={cn( | ||
| "flex flex-wrap items-center gap-x-1.5 gap-y-1", | ||
| props.className, | ||
| )} | ||
| > | ||
| {props.items.map((item, index) => ( | ||
| <span | ||
| className="inline-flex min-w-0 items-center gap-x-1.5" | ||
| key={item.key} | ||
| > | ||
| {index > 0 ? <span className="text-[#666]">·</span> : null} | ||
| <span className="min-w-0">{item.content}</span> | ||
| </span> | ||
| ))} | ||
| </div> | ||
| ); | ||
| } |
124 changes: 124 additions & 0 deletions
124
packages/junior-dashboard/src/client/components/TelemetryMetrics.tsx
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,124 @@ | ||
| import { | ||
| formatCompactNumber, | ||
| formatMs, | ||
| formatTime, | ||
| formatTokenSummary, | ||
| type MessageSummary, | ||
| type TokenUsageSummary, | ||
| type ToolCallSummary, | ||
| } from "../format"; | ||
| import { MetricValue, type MetricTooltipLine } from "./Metric"; | ||
|
|
||
| function plural(label: string, count: number): string { | ||
| return `${formatCompactNumber(count)} ${label}${count === 1 ? "" : "s"}`; | ||
| } | ||
|
|
||
| function isMetricTooltipLine( | ||
| line: MetricTooltipLine | undefined, | ||
| ): line is MetricTooltipLine { | ||
| return Boolean(line); | ||
| } | ||
|
|
||
| function tokenTooltip(summary: TokenUsageSummary): MetricTooltipLine[] { | ||
| const lines: Array<MetricTooltipLine | undefined> = [ | ||
| summary.inputTokens !== undefined | ||
| ? { label: "input", value: formatCompactNumber(summary.inputTokens) } | ||
| : undefined, | ||
| summary.outputTokens !== undefined | ||
| ? { label: "output", value: formatCompactNumber(summary.outputTokens) } | ||
| : undefined, | ||
| summary.cachedInputTokens !== undefined | ||
| ? { | ||
| label: "cached", | ||
| value: formatCompactNumber(summary.cachedInputTokens), | ||
| } | ||
| : undefined, | ||
| summary.cacheCreationTokens !== undefined | ||
| ? { | ||
| label: "cache write", | ||
| value: formatCompactNumber(summary.cacheCreationTokens), | ||
| } | ||
| : undefined, | ||
| summary.providerTotalTokens !== undefined | ||
| ? { | ||
| label: "provider", | ||
| value: formatCompactNumber(summary.providerTotalTokens), | ||
| } | ||
| : undefined, | ||
| ]; | ||
| return lines.filter(isMetricTooltipLine); | ||
| } | ||
|
|
||
| /** Render total token usage with a hoverable breakdown. */ | ||
| export function TokenMetric(props: { | ||
| align?: "left" | "right"; | ||
| summary: TokenUsageSummary | undefined; | ||
| }) { | ||
| if (!props.summary) return null; | ||
| return ( | ||
| <MetricValue align={props.align} tooltip={tokenTooltip(props.summary)}> | ||
| {formatTokenSummary(props.summary)} | ||
| </MetricValue> | ||
| ); | ||
| } | ||
|
|
||
| /** Render a duration value with start/end timestamps in the tooltip. */ | ||
| export function DurationMetric(props: { | ||
| align?: "left" | "right"; | ||
| endedAt?: string; | ||
| label: string; | ||
| startedAt?: string; | ||
| }) { | ||
| if (!props.label || props.label === "none") return null; | ||
| const lines: Array<MetricTooltipLine | undefined> = [ | ||
| props.startedAt | ||
| ? { label: "started", value: formatTime(props.startedAt) } | ||
| : undefined, | ||
| props.endedAt | ||
| ? { label: "ended", value: formatTime(props.endedAt) } | ||
| : undefined, | ||
| ]; | ||
| const tooltip = lines.filter(isMetricTooltipLine); | ||
| return ( | ||
| <MetricValue align={props.align} tooltip={tooltip}> | ||
| {props.label} | ||
| </MetricValue> | ||
| ); | ||
| } | ||
|
|
||
| /** Render a tool-call count with top tool names, counts, and matched duration. */ | ||
| export function ToolCallsMetric(props: { | ||
| align?: "left" | "right"; | ||
| loading?: boolean; | ||
| summary: ToolCallSummary | undefined; | ||
| }) { | ||
| if (props.loading) return <span>tool calls loading</span>; | ||
| if (!props.summary || props.summary.total <= 0) return null; | ||
| const tooltip = props.summary.items.map((item) => ({ | ||
| label: item.name, | ||
| labelStyle: "code" as const, | ||
| value: [ | ||
| plural("call", item.count), | ||
| item.totalDurationMs !== undefined | ||
| ? formatMs(item.totalDurationMs) | ||
| : undefined, | ||
| ] | ||
| .filter(Boolean) | ||
| .join(" · "), | ||
| })); | ||
| return ( | ||
| <MetricValue align={props.align} tooltip={tooltip}> | ||
| {plural("tool call", props.summary.total)} | ||
| </MetricValue> | ||
| ); | ||
| } | ||
|
|
||
| /** Render a conversational message count. */ | ||
| export function MessagesMetric(props: { | ||
| loading?: boolean; | ||
| summary: MessageSummary | undefined; | ||
| }) { | ||
| if (props.loading) return <span>messages loading</span>; | ||
| if (!props.summary) return null; | ||
| return <MetricValue>{plural("message", props.summary.total)}</MetricValue>; | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.