Skip to content

Skippy placement: account for inter-stage activation bandwidth #1128

Description

@micspiral

Problem

Skippy placement considers capacity and inter-node RTT, but not the usable bandwidth of each adjacent stage edge. That can produce a low-latency plan whose cold prefill is still dominated by activation transfer.

The client sends compact text only to stage 0. The large traffic is between stages: every prompt token's boundary activation crosses each adjacent edge once. For a 128K prompt:

  • typical Qwen, width 2,048 with F16 wire: about 512 MiB per edge
  • Inkling, width 6,144 with F32 wire: exactly 3 GiB per edge
  • Inkling with a validated F16/Q8 wire would fall to 1.5 GiB/768 MiB respectively

Prefill chunks pipeline across stages, so wall time tends toward the slowest edge rather than the sum of all edge times. Decode sends one token at a time, so RTT remains important there.

Proposed change

Make split placement aware of both directed edge bandwidth and RTT:

  • measure or advertise usable peer-to-peer bandwidth as an optional, additive capability
  • estimate boundary bytes from prompt/context size, activation width, selected wire dtype, and any activation sideband multiplier
  • score prefill placement by its bottleneck adjacent edge while retaining RTT weight for decode
  • prefer co-located or high-bandwidth adjacency for high-volume boundaries; do not confuse client-to-stage-0 ingress with inter-stage bandwidth
  • expose estimated bytes and transfer time per edge in plan/readiness diagnostics
  • use a conservative fallback when bandwidth is unknown, preserving mixed-version mesh compatibility

Quantisation must use the actual validated activation wire dtype; KV quantisation does not reduce boundary traffic.

Done when

  • deterministic planner tests choose higher-bandwidth adjacency for a large-context/wide-activation case with otherwise similar capacity and RTT
  • small-context/decode-oriented cases still prefer lower RTT where appropriate
  • plan diagnostics show the bandwidth assumption, boundary volume, bottleneck edge, and predicted transfer floor
  • a real multi-node run confirms the selected plan and observed prefill transport are directionally consistent

Related: #1025 covers decode/speculative latency hiding; this issue covers placement for activation bandwidth.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions