Skip to content

feat: Implement structured logging with request correlation IDs (#63)#136

Open
olaleyeolajide81-sketch wants to merge 1 commit into
Epondia:mainfrom
olaleyeolajide81-sketch:feature/structured-logging-correlation-ids
Open

feat: Implement structured logging with request correlation IDs (#63)#136
olaleyeolajide81-sketch wants to merge 1 commit into
Epondia:mainfrom
olaleyeolajide81-sketch:feature/structured-logging-correlation-ids

Conversation

@olaleyeolajide81-sketch

Copy link
Copy Markdown

This commit implements structured logging with correlation IDs to enable request tracing across services and provides JSON log format suitable for log aggregation tools like ELK and Datadog.

Changes Made:

1. Created Request Context Middleware

  • backend/src/middleware/requestContext.ts (TypeScript)
  • backend/src/middleware/requestContext.js (JavaScript)
    • Reads X-Correlation-ID header or generates new UUID
    • Attaches correlation ID to request object
    • Adds X-Correlation-ID to response headers
    • Tracks request start time for duration logging
    • Includes sensitive data redaction function
    • Provides requestDurationLogger middleware for endpoint timing

2. Enhanced Logger Utility (TypeScript)

  • backend/src/utils/logger.ts
    • Added JSON structured log format for production
    • Added pretty format for development with colorization
    • Implemented correlation ID propagation in log entries
    • Added sensitive data redaction (passwords, tokens, etc.)
    • File logs always use JSON format for aggregation
    • Added createChildLogger function for correlation ID scoping
    • Log level configurable via LOG_LEVEL environment variable

3. Enhanced Logger Utility (JavaScript)

  • backend/src/utils/logger.js
    • Aligned with TypeScript logger implementation
    • Added JSON structured log format for production
    • Added pretty format for development
    • Implemented correlation ID propagation
    • Added sensitive data redaction
    • File logs always use JSON format
    • Added createChildLogger function
    • Maintained security.log transport

4. Updated Main Application Entry Point

  • backend/src/index.js
    • Imported requestContext and requestDurationLogger middleware
    • Mounted requestContext middleware early in middleware chain
    • Updated request logging to include correlation ID
    • Ensures correlation ID is available to all downstream middleware

Definition of Done Checklist:

✅ X-Correlation-ID header read or generated per request ✅ Correlation ID propagated to all downstream log entries ✅ JSON structured log format with timestamp, level, correlationId, message, metadata ✅ Sensitive data (passwords, tokens) redacted from logs ✅ Request duration logged for each endpoint (via requestDurationLogger) ✅ Log level configurable via LOG_LEVEL environment variable

Technical Details:

  • Correlation IDs are generated using UUID v4
  • JSON format is used in production (NODE_ENV=production)
  • Pretty format with colors is used in development
  • File logs always use JSON format regardless of environment
  • Sensitive fields: password, token, apiKey, secret, authorization, creditCard, ssn, socialSecurityNumber
  • Request duration can be logged using requestDurationLogger middleware (optional)
  • Log levels: error, warn, info, http, debug

Files Modified:

  • backend/src/utils/logger.ts
  • backend/src/utils/logger.js
  • backend/src/middleware/requestContext.ts (new)
  • backend/src/middleware/requestContext.js (new)
  • backend/src/index.js

Closes #63

Description

Related Issue

Closes #

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • ✨ New feature (non-breaking change that adds functionality)
  • 💥 Breaking change (fix or feature that changes existing behavior)
  • 📚 Documentation update
  • ♻️ Refactor (no functional change)
  • 🧪 Tests
  • 🔧 Chore / tooling

Packages Affected

  • contracts/ (Soroban / Rust)
  • backend/ (Node / Express)
  • frontend/ (Next.js)
  • docs/

How Has This Been Tested?

Checklist

  • My code follows the project's coding standards (see CONTRIBUTING.md)
  • I have run the relevant linters and type checks
  • I have added or updated tests that prove my change works
  • All new and existing tests pass locally
  • I have updated documentation where needed
  • My commits follow the Conventional Commits format
  • I have noted any breaking changes below (or there are none)

Breaking Changes

None

Additional Notes / Screenshots

closes #63

…dia#63)

This commit implements structured logging with correlation IDs to enable
request tracing across services and provides JSON log format suitable for
log aggregation tools like ELK and Datadog.

## Changes Made:

### 1. Created Request Context Middleware
- backend/src/middleware/requestContext.ts (TypeScript)
- backend/src/middleware/requestContext.js (JavaScript)
  - Reads X-Correlation-ID header or generates new UUID
  - Attaches correlation ID to request object
  - Adds X-Correlation-ID to response headers
  - Tracks request start time for duration logging
  - Includes sensitive data redaction function
  - Provides requestDurationLogger middleware for endpoint timing

### 2. Enhanced Logger Utility (TypeScript)
- backend/src/utils/logger.ts
  - Added JSON structured log format for production
  - Added pretty format for development with colorization
  - Implemented correlation ID propagation in log entries
  - Added sensitive data redaction (passwords, tokens, etc.)
  - File logs always use JSON format for aggregation
  - Added createChildLogger function for correlation ID scoping
  - Log level configurable via LOG_LEVEL environment variable

### 3. Enhanced Logger Utility (JavaScript)
- backend/src/utils/logger.js
  - Aligned with TypeScript logger implementation
  - Added JSON structured log format for production
  - Added pretty format for development
  - Implemented correlation ID propagation
  - Added sensitive data redaction
  - File logs always use JSON format
  - Added createChildLogger function
  - Maintained security.log transport

### 4. Updated Main Application Entry Point
- backend/src/index.js
  - Imported requestContext and requestDurationLogger middleware
  - Mounted requestContext middleware early in middleware chain
  - Updated request logging to include correlation ID
  - Ensures correlation ID is available to all downstream middleware

## Definition of Done Checklist:
✅ X-Correlation-ID header read or generated per request
✅ Correlation ID propagated to all downstream log entries
✅ JSON structured log format with timestamp, level, correlationId, message, metadata
✅ Sensitive data (passwords, tokens) redacted from logs
✅ Request duration logged for each endpoint (via requestDurationLogger)
✅ Log level configurable via LOG_LEVEL environment variable

## Technical Details:
- Correlation IDs are generated using UUID v4
- JSON format is used in production (NODE_ENV=production)
- Pretty format with colors is used in development
- File logs always use JSON format regardless of environment
- Sensitive fields: password, token, apiKey, secret, authorization, creditCard, ssn, socialSecurityNumber
- Request duration can be logged using requestDurationLogger middleware (optional)
- Log levels: error, warn, info, http, debug

## Files Modified:
- backend/src/utils/logger.ts
- backend/src/utils/logger.js
- backend/src/middleware/requestContext.ts (new)
- backend/src/middleware/requestContext.js (new)
- backend/src/index.js

Closes Epondia#63
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement structured logging with request correlation IDs

1 participant