Skip to content

feat: Add Structured Telemetry Logging for dApp Generation & Compile Metrics - #49

Open
ugoocreates-pixel wants to merge 2 commits into
Ender-Stellar:mainfrom
ugoocreates-pixel:feature/telemetry-logging
Open

feat: Add Structured Telemetry Logging for dApp Generation & Compile Metrics#49
ugoocreates-pixel wants to merge 2 commits into
Ender-Stellar:mainfrom
ugoocreates-pixel:feature/telemetry-logging

Conversation

@ugoocreates-pixel

Copy link
Copy Markdown

this pr closes #32

📖 Description

We currently lack visibility into which AI models and generation parameters (such as the target blockchain network, prompts, or edits) lead to successful builds versus causing Vite compile or HMR errors in the sandbox.

This PR introduces structured telemetry logging in MongoDB to track these metrics. By assigning a unique generationId to each code generation stream, we are able to correlate initial LLM generation parameters with subsequent client-side compilation outcomes (either a clean load success or a list of specific compile-error objects).


🛠️ Key Changes

1. Database Layer (lib/mongo.ts)

  • Added a unique index on generationId in the telemetry collection to ensure fast and structured queries.

2. Backend Telemetry and Tracking

  • app/api/generate-ai-code-stream/route.ts:
    • Generates a unique generationId at the start of a request.
    • Logs an initial telemetry record containing: generationId, sandboxId, model, chain, isEdit, promptLength, promptSummary, and initial status ('generating').
    • Streams the generationId back to the frontend in the first status event.
    • Updates the telemetry status to 'generated' on stream completion, or 'generation-failed' (along with the error message) if generation fails.
  • app/api/telemetry/route.ts (New Endpoint):
    • Created a POST endpoint /api/telemetry to receive compile status updates ('success' or 'compile-error') from the client and update/upsert the corresponding record in MongoDB.

3. Client Integration (app/generation/page.tsx)

  • Declared a React state currentGenerationId and status checking tracking refs.
  • Extracted and tracked generationId from streaming status updates.
  • Added a useEffect hook that listens to the buildErrors state and immediately posts a 'compile-error' status to the telemetry API with the error details.
  • Scheduled a 10-second success check when code application is finalized; if the timeout expires with no build errors, it reports 'success'.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

backend: Add structured telemetry logging to track generation success/failure metrics

1 participant