What
For MoE models (e.g. Mixtral/Qwen-MoE/gpt-oss class), track which experts get activated during inference and surface that as a mesh-wide statistic — purely as a curiosity/observability feature.
Why
- MoE routing is invisible today. We split MoE models across nodes but have no idea how balanced the expert activation actually is.
- Interesting signal for split planning: if a handful of experts dominate, expert-aware placement could beat naive layer splits.
- Fun/insightful thing to show in the console — a live histogram of expert activation across the mesh.
Sketch
- Expose per-token expert selection counts from the embedded staged runtime (skippy) as a lightweight counter, off by default or cheap enough to always run.
- Aggregate locally per model, then gossip a compact summary (additive field, ignorable by older nodes) so any node can render mesh-wide totals.
- Surface via
/api/status (or a dedicated stats route) and a small console visualization; optionally emit as OTLP metrics through skippy-metrics.
Notes
- Must be additive on the wire — older peers should ignore the new gossip field.
- Keep the hot path cost near zero; prefer counters incremented in the runtime over per-token event streams.
- Privacy: counts only, no prompt or token content.
What
For MoE models (e.g. Mixtral/Qwen-MoE/gpt-oss class), track which experts get activated during inference and surface that as a mesh-wide statistic — purely as a curiosity/observability feature.
Why
Sketch
/api/status(or a dedicated stats route) and a small console visualization; optionally emit as OTLP metrics through skippy-metrics.Notes