fix: improve analytics observability for retention debugging#79
Conversation
- Use Effect error _tag instead of constructor.name for run:failed error_tag, fixing minified class names in production builds - Instrument missing cookies:sync_choice and cookies:toggled events - Only emit cookies:browser_selection when browsers are selected Made-with: Cursor
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
commit: |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 2 potential issues.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 6e3a625. Configure here.
| "enabled": true | ||
| } | ||
| } | ||
| } |
There was a problem hiding this comment.
Cursor IDE settings file accidentally committed to repository
Low Severity
The .cursor/settings.json file enables the Cursor IDE PostHog telemetry plugin. This is IDE-specific configuration unrelated to the PR's application analytics improvements. The .gitignore already covers .cursor/plans but not this file, suggesting it was unintentionally included alongside the analytics changes.
Reviewed by Cursor Bugbot for commit 6e3a625. Configure here.
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
1 issue found across 4 files
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="apps/cli/src/data/execution-atom.ts">
<violation number="1" location="apps/cli/src/data/execution-atom.ts:192">
P2: Guard `error` as a non-null object before using `"_tag" in error`; otherwise telemetry can be dropped when failures are primitives/null.</violation>
</file>
Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review, or fix all with cubic.
Test Results✅ Website Test: passed13 passed, 0 failed out of 14 steps — 338s
Session Recordinghttps://github.com/millionco/expect/releases/download/ci-pr-79/36b597dd55976825a89bac6b9e4f50f5.webm |
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
![vercel[bot]](https://avatars.githubusercontent.com/in/8329?s=60&v=4){kind=small}
| @@ -0,0 +1,7 @@ | |||
| { | |||
There was a problem hiding this comment.
IDE-specific settings file (.cursor/settings.json) was committed to the repository when it should only be in .gitignore
| @@ -0,0 +1,7 @@ | |||
| { | |||
| "plugins": { | |||
| "posthog": { | |||
- Updated error handling logic to cast errors as any before checking instance type, ensuring compatibility with various error structures. - This change addresses potential type issues while maintaining existing functionality.
2 participants
Summary
run:failedevents now use Effect's_tagproperty instead oferror.constructor.name, which gets minified in production builds. This means error types likeHeadlessRunTimeoutErrorwill appear as readable strings in PostHog instead ofem,rm, etc.cookies:sync_choiceandcookies:toggledwere typed inEventMapbut never emitted. Now fired from the cookie sync confirm screen and the Ctrl+K toggle in the main menu.cookies:browser_selection: Previously fired even when zero browsers were selected. Now only emits whenkeys.length > 0.Context
Retention analysis surfaced several observability gaps:
run:failedevents had opaque minified error tags, making crash diagnosis impossiblecookies:sync_choiceshowed historical data in PostHog but was no longer instrumented after a refactorTest plan
pnpm typecheck— only pre-existingagent.tserrors, no new type errorsrun:failedin dev mode and verifyerror_tagcontains the_tagstringcookies:sync_choice,cookies:browser_selection, andcookies:toggledevents fire in PostHog (or dev logger)cookies:cleared+cookies:toggledboth fireMade with Cursor
Note
Low Risk
Low risk analytics-only changes: adds/adjusts PostHog event emissions and improves
run:failedtagging without altering execution flow.Overview
Improves CLI analytics observability by emitting the previously-unused
cookies:sync_choiceandcookies:toggledevents, and by only firingcookies:browser_selectionwhen at least one browser is selected (cookie sync confirm + Ctrl+K clear path).Updates
run:failedanalytics to prefer Effect error_tagvalues overerror.constructor.nameto avoid minified error type strings in production builds. Also adds.cursor/settings.jsonenabling the PostHog Cursor plugin.Reviewed by Cursor Bugbot for commit 6e3a625. Bugbot is set up for automated code reviews on this repo. Configure here.
Summary by cubic
Improves analytics for retention debugging by making error tagging robust and restoring cookie event tracking. Failed runs and cookie flows now report accurate, readable data in PostHog.
Bug Fixes
_tagforrun:failederror_tagand add safe fallback toErrorname to avoid minified tags.cookies:browser_selectiononly when at least one browser is selected.New Features
cookies:sync_choiceandcookies:toggled(fires on enable/disable, including Ctrl+K clear).Written for commit bdfaf7f. Summary will update on new commits.