feat: structured JSON logger with per-request id and access log#156
Open
Shawnaldinho wants to merge 1 commit into
Open
feat: structured JSON logger with per-request id and access log#156Shawnaldinho wants to merge 1 commit into
Shawnaldinho wants to merge 1 commit into
Conversation
Previously every backend log was a free-form console.log/error, so
there was no way to (a) correlate the streaming error you see in a
client with the server-side line that produced it, or (b) filter on
status / route / user_id from a log aggregator. The blog post about
mikeoss called out the absence of structured logging, request
correlation, and error context, so this PR puts the floor in.
Three pieces:
1. backend/src/lib/logger.ts — ~80-line zero-dep JSON line logger.
- logger.{debug,info,warn,error}({ fields }, msg) emits one JSON
object per call to stdout/stderr.
- level is controlled by LOG_LEVEL env var, defaulting to "info"
in production and "debug" elsewhere.
- logger.child({ fields }) returns a logger that prefixes every
event with the given fields — used by the request middleware to
bind request_id.
- errFields(err) helper extracts name/message/stack from an Error.
- Kept hand-written instead of pulling in pino because we only need
~30 lines today; the call sites won't change if it's later swapped.
2. backend/src/middleware/requestContext.ts — assigns a per-request id
(from inbound X-Request-Id if present, else a UUID), echoes it on
the response, parks a child logger on res.locals.log, and on
response close emits one access-log line with method/route/status/
elapsed_ms/user_id. Wired into index.ts before the rate limiters.
3. Demonstrate adoption: replace console.error in the two streaming
error paths (POST /chat, POST /projects/:projectId/chat) and the
generate-title handler. Each unhandled error log now ships with the
request_id so it correlates to the access log line. Left the
debug-style console.log calls in chatTools.ts / documents.ts alone
— they're a separate, broader cleanup.
The X-Request-Id header is echoed on every response so a curl user
or a frontend retry can grep their request out of the logs.
amal66
added a commit
to amal66/mike
that referenced
this pull request
May 25, 2026
Chapter: 05 - Observability. Plain-English map: Add Pino JSON logging and attach a request ID to each request so related log lines can be traced together in production. Why it matters: Plain text logs are hard to search once many users are active. Structured logs let operators answer practical questions like "what happened during this one failed upload?" Principle: Production logs should be queryable, consistent, and tied to request context. Precedent borrowed: Upstream PR willchen96#156 and common production logging practice in hosted services. Upstream base: willchen96/mike@d39f580. Original local commit: 6f59af5.
amal66
added a commit
to amal66/mike
that referenced
this pull request
May 25, 2026
Chapter: 26 - Sensitive-path observability. Plain-English map: Replace remaining console logging in `chatTools.ts` with the same structured Pino logger used elsewhere. Why it matters: `chatTools.ts` assembles prompts and handles model tool calls. It should have the same traceability and privacy discipline as the rest of the API. Principle: Observability conventions should be consistent, especially in sensitive code. Precedent borrowed: Upstream PR willchen96#156 and the structured logging baseline from Chapter 05. Upstream base: willchen96/mike@d39f580. Original local commit: 921ef5b.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Backend logs were all free-form `console.log`/`console.error`, so an error you see on the client can't be correlated to the line that produced it on the server, and there's no way to filter by route/status/user_id from a log aggregator. This PR puts the floor in: a tiny structured logger, a per-request id, and an access log line per response.
Changes
Why
Concrete things this enables that weren't possible before:
Addresses the observability concern from https://insights.flank.ai/where-mikeoss-falls-short.html (gap 14). Doesn't try to also tackle tracing / metrics / Sentry — those are reasonable follow-ups built on this foundation.
Testing