Skip to content

libkineto: native Perfetto (.pftrace) trace export#5

Open
ajassani wants to merge 1 commit into
mainfrom
pftrace-export
Open

libkineto: native Perfetto (.pftrace) trace export#5
ajassani wants to merge 1 commit into
mainfrom
pftrace-export

Conversation

@ajassani

Copy link
Copy Markdown
Owner

Summary

Adds native Perfetto protobuf (.pftrace) export to libkineto, so PyTorch profiler traces can be written directly in the same format consumed by Perfetto and TraceLens pipelines, with no intermediate Chrome-JSON → pftrace conversion.

  • New output_perfetto.{h,cpp}: a PerfettoTraceLogger implementing ActivityLogger on top of the Perfetto in-process tracing SDK. Each Kineto ActivityType maps to a compile-time-registered TrackEvent category; linked activities are emitted as Perfetto flows; activity metadata is attached as debug annotations.
  • Registers a pftrace:// protocol in ActivityLoggerFactory and routes any save() target ending in .pftrace to it (existing .json behavior is untouched).
  • New CMake option KINETO_ENABLE_PERFETTO (default OFF). When ON, the build compiles the Perfetto SDK amalgamation from PERFETTO_SOURCE_DIR into a static kineto_perfetto target and links it into libkineto.
  • Gated unit test (PerfettoTraceLoggerTest) asserting a non-empty .pftrace is produced.

The path is fully optional. With KINETO_ENABLE_PERFETTO=OFF the new translation unit compiles to nothing and there is no behavior or dependency change.

Why

.pftrace is the native Perfetto format; emitting it directly avoids a lossy/slow JSON round-trip and lets profiler output drop straight into Perfetto-based analysis.

Validation

Built into a full PyTorch profiler stack with KINETO_ENABLE_PERFETTO=ON and exercised end-to-end via torch.profiler on both CPU and GPU (MI210) workloads:

  • CPU profile → non-empty .pftrace (~8.7 KB)
  • GPU matmul profile ([CPU, CUDA] activities) → non-empty .pftrace (~33 KB)

Both files open in Perfetto with per-thread/per-device tracks and flows.

Build

cmake -S libkineto -B build \
  -DKINETO_ENABLE_PERFETTO=ON \
  -DPERFETTO_SOURCE_DIR=/path/to/perfetto

PERFETTO_SOURCE_DIR must contain sdk/perfetto.h and sdk/perfetto.cc.

Notes / follow-ups

  • The Perfetto SDK amalgam is compiled with reduced optimization to keep build time reasonable; this only affects the vendored Perfetto translation unit, not libkineto itself.
  • The unit test is gated on KINETO_ENABLE_PERFETTO; wiring it into the default test target matrix can be a follow-up.

Add an optional PerfettoTraceLogger that writes traces directly in the
Perfetto protobuf (.pftrace) format, so PyTorch profiler output can be
consumed natively by Perfetto/TraceLens pipelines without a JSON->pftrace
conversion step.

- New output_perfetto.{h,cpp} implementing ActivityLogger on top of the
  Perfetto in-process tracing SDK, mapping each Kineto ActivityType to a
  registered TrackEvent category and emitting flows for linked activities.
- Register a "pftrace://" protocol in the ActivityLoggerFactory and route
  any save() target ending in .pftrace to it.
- Gate the whole path behind a new KINETO_ENABLE_PERFETTO CMake option
  (OFF by default). When ON, the build compiles the Perfetto SDK amalgam
  from PERFETTO_SOURCE_DIR and links it into libkineto.
- Add a gated unit test asserting a non-empty .pftrace is produced.

The feature is fully optional: with KINETO_ENABLE_PERFETTO=OFF the new
source compiles to nothing and behavior is unchanged.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant