Skip to content

Add LangSmith tracing and system-of-record webhook#1

Open
snowsecure wants to merge 1 commit intomainfrom
codex/evaluate-project-for-enterprise-deployment
Open

Add LangSmith tracing and system-of-record webhook#1
snowsecure wants to merge 1 commit intomainfrom
codex/evaluate-project-for-enterprise-deployment

Conversation

@snowsecure
Copy link
Owner

Motivation

  • Enable structured tracing and sampling for LLM calls using LangSmith so API calls and responses can be observed and correlated across runs.
  • Provide an optional system-of-record webhook to notify downstream systems when an abstract finishes processing, enabling eventual integration with external systems of record.

Description

  • Add a new LangSmith helper module app.core.langsmith and settings in app.core.config (LANGSMITH_*), plus environment examples in .env.example and docs in backend/README.md to configure tracing and sampling.
  • Wire LangSmith initialization into FastAPI startup (app.main) and Celery startup (app.workers.celery_app) via configure_langsmith().
  • Instrument LLM clients (app.core.gemini_client, app.core.clau de_client, app.core.azure_openai_client) to start/end sampled LangSmith runs and attach input/output previews and token metadata using build_llm_inputs, build_llm_outputs, and start_langsmith_run.
  • Add an optional system-of-record publisher app.core.system_of_record and call publish_abstract_completed() from Celery processing tasks in app.workers.tasks after abstracts are saved; also add corresponding config flags and .env.example entries, and add langsmith to backend/requirements.txt.

Testing

  • No automated tests were executed as part of this change.
  • Existing test suite (pytest) was not run during this rollout; updates are limited to wiring/tracing and webhook publish calls and should be covered by integration/smoke tests in downstream validation.

Codex Task

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant