Skip to content

feat: add package wrapper install telemetry #260

Description

@parsakhaz

Intent

Add first-class telemetry for the runpane npm and PyPI wrappers so we can see the full package-wrapper funnel, not just the final Pane artifact download. This matters because download_started with source=npm or source=pip proves a wrapper requested a release artifact, but it does not show whether someone installed the package, invoked the wrapper, chose client vs daemon setup, succeeded, failed, or fell back to GitHub.

Scope

Track wrapper-level activity for both package surfaces:

  • npm ecosystem: npm, npx, pnpm dlx, yarn dlx, bunx, global npm/pnpm installs where detectable.
  • Python ecosystem: pip, pipx, uvx, and python -m runpane where detectable.
  • Commands that should be distinguishable: setup, install client, install daemon, and update.
  • Outcomes that should be distinguishable: command started, artifact download requested, download succeeded, download failed, install/setup succeeded, install/setup failed, and fallback-to-GitHub used.
  • Preserve the current download_started source attribution (source=npm / source=pip) for the actual artifact redirect.

Starting Points

These references are non-exhaustive; investigate further before implementation.

  • packages/runpane/src/cli.ts
  • packages/runpane/src/releases.ts
  • packages/runpane-py/src/runpane/cli.py
  • packages/runpane-py/src/runpane/releases.py
  • docs/RUNPANE_CLI_CONTRACT.md
  • runpane website docs: docs/posthog-event-catalog.md and docs/analytics-attribution.md now document package downloads as download_started segmented by properties.source.

Acceptance Criteria

  • We can answer, in PostHog, how many users invoked the npm wrapper versus the PyPI wrapper over a selected time window.
  • We can distinguish npx/npm-family usage from pipx/pip/uvx-style usage when the runtime makes that reasonably detectable.
  • We can distinguish client install, daemon install, setup, and update wrapper commands.
  • We can see started, succeeded, failed, and fallback outcomes without relying on raw logs.
  • Events include non-sensitive context such as wrapper, wrapper_version, invocation, command, platform, arch, pane_version, and download_source.
  • Events must not include local paths, usernames, hostnames, tokens, connection codes, raw errors, full URLs with sensitive query params, or environment dumps.
  • Existing /api/download telemetry still emits download_started with source=npm or source=pip when the wrapper requests a Pane artifact.
  • The PostHog event catalog and analytics attribution docs are updated with the final event names and properties.
  • Tests cover the telemetry property builder/sanitizer for npm and Python paths, including failure/fallback cases.

Inputs Needed

Decide whether wrapper telemetry should be sent directly from the package clients to PostHog/runpane.com, or whether it should go through a new runpane.com API route that validates and forwards sanitized events. Prefer a server route if direct client capture would expose keys or make schema evolution harder.

Notes

Current observed data shows package-wrapper artifact downloads are present via download_started: source=npm and source=pip. That is useful but incomplete: it only records artifact fetches through /api/download, not package install, wrapper invocation, command intent, success/failure, or GitHub fallback.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions