Skip to content

[Agent] capture_pprof reports zero worker-wedge gauges and lacks a capture throttle #2422

Description

@ToddHebebrand

Description

Two gaps in the new capture_pprof diagnostic command (#2394/#2408 seam):

  1. The embedded runtime snapshot never includes worker-pool wedge gauges. agent/internal/heartbeat/handlers_diag.go:29,64handleCapturePprof(_ *Heartbeat, cmd Command) discards its *Heartbeat receiver and calls collectors.CollectRuntimeStats() directly instead of h.collectAgentRuntime(now) (heartbeat.go:3766-3769), the only wrapper that populates CommandsInFlight/CommandsOverdue via h.inFlightCommandStats(now). So every capture reports commandsInFlight: 0, commandsOverdue: 0 — an operator investigating an overdue-commands heartbeat gauge (the exact stated purpose of feat(agent): runtime memory gauges on heartbeat + on-demand pprof capture (#2389) #2394: "correlate the profile with the heartbeat trend") gets a snapshot that falsely shows a healthy pool.

  2. No rate limit on capture. handlers_diag.go:71 unconditionally calls runtime.GC() on every heap/all capture. It is a server-queued command (up to 10 concurrent, 100 queued), so several queued captures can force back-to-back stop-the-world GCs. fix(agent): stop heartbeat watchdog from firing every heartbeat on slow links (#2386) #2392 added exactly this throttle pattern for the heartbeat watchdog stack dump (heartbeatWatchdogTryAcquireDump + interval gate) in the same week; reuse it here (e.g. min 30s between captures, reject with an honest error otherwise).

Affected Files

  • agent/internal/heartbeat/handlers_diag.go:29,64,71
  • agent/internal/heartbeat/heartbeat.go:3766-3769 (existing wrapper to call)

Found during pre-v0.95.0 whole-range review (v0.94.0..main).

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions