[DevOps] Add structured logging and system observability

[Atharv777]

Summary

Introduce structured logging and basic observability to track system behavior, errors, and performance.

Why this matters

Without proper logging:

• debugging is difficult
• production issues are hard to diagnose
• system health is unclear

Observability is essential for maintainability and scaling.

Scope

• Implement structured JSON logging
• Add logging for:
  • API requests
  • execution lifecycle
  • errors and exceptions
• Add request identifiers for tracing
• Add log levels (info, warning, error)
• Ensure logs are consistent across modules

Acceptance Criteria

• Logs are structured and consistent
• Key events (execution, errors, auth) are logged
• Logs help trace issues across requests
• Logging does not significantly impact performance

Files Involved

• server/start.py
• server/utils/*
• server/agent.py
• server/project_routes.py

Difficulty

Medium

Labels: devops backend enhancement