RunAnywhere Pipeline Profiler

A production-grade profiling toolkit for the RunAnywhere SDK. Instruments the full STT → LLM → TTS pipeline, exports structured trace JSON, and renders an interactive Perfetto-style timeline in the browser.

runanywhere-profiler/
├── swift/
│   └── InferenceProfiler.swift   # iOS/macOS SDK instrumentation
├── kotlin/
│   └── InferenceProfiler.kt      # Android SDK instrumentation
└── viewer/
    └── index.html                # Self-contained trace viewer (no build step)

---

Quick start

iOS / macOS

1. Copy InferenceProfiler.swift into your RunAnywhere SDK package.
2. Hook the beginSpan / endSpan calls into your pipeline methods (see comments at the bottom of the file for exact integration points).
3. Enable and export:

// AppDelegate or your AI manager:
RunAnywhere.enableProfiling()

// After running a few voice turns:
if let jsonData = try? RunAnywhere.exportTraceJSON() {
    // Save to Documents directory:
    let url = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask)[0]
        .appendingPathComponent("ra-trace.json")
    try? jsonData.write(to: url)
}

4. AirDrop or share the ra-trace.json file from your device.
5. Open viewer/index.html in Chrome and drop the file onto the viewer.

The profiler also emits os_signpost events — open Instruments.app with a "Time Profile" instrument and you'll see RunAnywhere spans natively annotated.

---

Android

1. Copy InferenceProfiler.kt into your RunAnywhere SDK module.
2. Add dependencies to your build.gradle.kts:

   implementation("org.jetbrains.kotlinx:kotlinx-serialization-json:1.6.3")
   implementation("org.jetbrains.kotlinx:kotlinx-coroutines-android:1.7.3")

3. Hook into your pipeline:

// In your RunAnywhere voice pipeline:
RunAnywhereProfiler.enable()

// Wrap pipeline stages:
val requestId = UUID.randomUUID().toString()
val key = InferenceProfiler.instance.beginSpan(PipelineStage.STT_TRANSCRIBE, requestId)
val transcript = whisper.transcribe(audioBytes)
InferenceProfiler.instance.endSpan(key, mapOf("audioSeconds" to "2.1"))

// Or use the coroutine helper:
val transcript = InferenceProfiler.instance.profileSuspend(
    PipelineStage.STT_TRANSCRIBE, requestId
) {
    whisper.transcribe(audioBytes)
}

// Export after 5+ requests:
val json = RunAnywhereProfiler.exportTraceJson()
// Save to external storage or send via share sheet

4. The profiler also emits android.os.Trace sections visible in Android Studio Profiler → System Trace.

---

Trace viewer

viewer/index.html is fully self-contained — no npm, no build step, no server. Just open it in Chrome (local file:// works fine).

Features:

• Timeline view: Perfetto-style per-stage swimlanes with hover tooltips
• Flame graph view: percentage breakdown of cumulative time per stage
• Memory overlay: real-time allocated MB chart synced to the timeline
• Zoom slider: 1× to 20× zoom on the timeline
• Works with both iOS and Android traces (same JSON schema)

---

Trace JSON schema

{
  "schemaVersion": "1.0",
  "capturedAt": 1712500000000,
  "deviceInfo": {
    "chip": "Apple M4 Max",
    "totalMemoryMB": 768,
    "osVersion": "iOS 18.3",
    "sdkVersion": "0.17.2"
  },
  "spans": [
    {
      "id": "uuid",
      "requestId": "req-0",
      "stage": "stt",
      "stageDisplayName": "STT",
      "color": "#06b6d4",
      "startMs": 20.0,
      "endMs": 121.4,
      "durationMs": 101.4,
      "metadata": { "audioSeconds": "2.1" },
      "threadName": "ai.runanywhere.queue"
    }
  ],
  "memorySamples": [
    { "timestampMs": 0.0, "allocatedMB": 284.3, "availableMB": 483.7 }
  ],
  "summary": {
    "totalRequests": 6,
    "avgPipelineMs": 245.8,
    "p50PipelineMs": 238.1,
    "p95PipelineMs": 312.4,
    "p99PipelineMs": 341.0,
    "avgSttMs": 98.2,
    "avgLlmPrefillMs": 7.4,
    "avgLlmDecodeMs": 72.1,
    "avgTtsMs": 53.8,
    "avgTokensPerSecond": 641.2,
    "peakMemoryMB": 341.0
  }
}

---

Pipeline stages

Stage key	Display name	Notes
vad	VAD	Voice activity detection
model_load	Model Load	First-time model initialization
stt	STT	Whisper transcription
llm_prefill	LLM Prefill	KV cache population (TTFT)
llm_decode	LLM Decode	Token generation loop
tts	TTS	Piper synthesis
pipeline_total	Pipeline Total	Full turn latency
custom	Custom	App-defined spans

---

Why this exists

RunAnywhere's benchmarks show 63ms first-audio latency on M4 Max. Developers building real apps on real devices can't reproduce that without knowing where their latency is actually going. This tooling closes that gap — giving every RunAnywhere developer the same observability the founders use internally.

Built as a contribution to the RunAnywhere SDK ecosystem.