Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 17 additions & 20 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -1,29 +1,26 @@
# Braintrust SDK
# Braintrust JavaScript SDK Monorepo

JavaScript client for Braintrust, plus wrapper libraries for OpenAI, Anthropic, and other AI providers.
JavaScript/TypeScript SDKs and integrations for Braintrust.

This repo uses `pnpm` as it's package manager.
This repository uses `pnpm` workspaces.

## Structure
## Repository Structure

```text
.
├── js/ # Main `braintrust` package (see js/CLAUDE.md)
├── integrations/ # Integration packages (@braintrust/*)
├── docs/ # Docs and reference material
└── internal/ # Internal test fixtures and golden projects
```
sdk/
├── js/ # JavaScript SDK (see js/CLAUDE.md)
└── core/ # Shared core library
```

## Quick Reference

| Task | Command |
| ------------- | ---------------- |
| Run all tests | `pnpm run test` |
| Build | `pnpm run build` |
| Lint check | `pnpm run lint` |
| Auto-fix | `pnpm run fix` |

## Setup
## Common Commands (repo root)

```bash
pnpm install # Install dependencies
pnpm run build # Build all packages
pnpm install # Install dependencies
pnpm run build # Build all workspace packages
pnpm run test # Run workspace tests (via turbo)
pnpm run lint # Prettier + ESLint checks
pnpm run fix # Auto-fix Prettier + ESLint
make test # Full JS-oriented test flow used in this repo
```
85 changes: 42 additions & 43 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,72 +1,71 @@
# Braintrust SDK
# Braintrust JavaScript SDKs

[Braintrust](https://www.braintrust.dev/) is a platform for evaluating and shipping AI products. To learn more about Braintrust or sign up for free,
visit our [website](https://www.braintrust.dev/) or check out the [docs](https://www.braintrust.dev/docs).
[Braintrust](https://www.braintrust.dev/) is a platform for evaluating and shipping AI products. Learn more at [braintrust.dev](https://www.braintrust.dev/) and in the [docs](https://www.braintrust.dev/docs).

This repository contains the JavaScript SDK for Braintrust. The SDK includes utilities to:
This repository contains Braintrust's JavaScript/TypeScript SDKs and integrations, including:

- Log experiments and datasets to Braintrust
- Run evaluations (via the `Eval` framework)
- The main `braintrust` SDK package (`./js`)
- Integration packages under `./integrations`
- Shared tooling and smoke tests for JavaScript SDK development

## Quickstart

First, install the Braintrust SDK:
Install the main SDK and autoeval scorers:

```bash
npm install braintrust autoevals
```

or
Create `tutorial.eval.ts`:

```bash

yarn add braintrust autoevals

```

Then, create a file named `tutorial.eval.ts` with the following code:

```typescript
```ts
import { Eval } from "braintrust";
import { LevenshteinScorer } from "autoevals";

Eval("Say Hi Bot", {
data: () => {
return [
{
input: "Foo",
expected: "Hi Foo",
},
{
input: "Bar",
expected: "Hello Bar",
},
]; // Replace with your eval dataset
},
task: (input) => {
return "Hi " + input; // Replace with your LLM call
},
data: () => [
{ input: "Foo", expected: "Hi Foo" },
{ input: "Bar", expected: "Hello Bar" },
],
task: (input) => "Hi " + input,
scores: [LevenshteinScorer],
});
```

Then, run the following command:
Run it:

```bash
BRAINTRUST_API_KEY=<YOUR_API_KEY> \
npx braintrust eval tutorial.eval.ts
BRAINTRUST_API_KEY=<YOUR_API_KEY> npx braintrust eval tutorial.eval.ts
```

## Integrations

Braintrust provides integrations with several popular AI development tools and platforms:
## Packages

- **LangChain.js**: A callback handler to automatically log LangChain.js executions to Braintrust. [Learn more](integrations/langchain-js)
- **Val Town**: Examples and templates for using Braintrust with Val Town's serverless JavaScript/TypeScript environment. [Learn more](integrations/val.town)
- **Vercel AI SDK**: Integration with Vercel's AI SDK for building AI-powered applications. [Learn more](integrations/vercel-ai-sdk)
| Package | Purpose | npm | Docs |
| --------------------------- | ------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------- |
| `braintrust` | Core JavaScript/TypeScript SDK for logging, tracing, evals, and CLI. | [![npm: braintrust](https://img.shields.io/npm/v/braintrust.svg)](https://www.npmjs.com/package/braintrust) | [js/README.md](js/README.md) |
| `@braintrust/browser` | Browser-focused SDK integration with AsyncLocalStorage polyfill support. | [![npm: @braintrust/browser](https://img.shields.io/npm/v/%40braintrust%2Fbrowser.svg)](https://www.npmjs.com/package/@braintrust/browser) | [integrations/browser-js/README.md](integrations/browser-js/README.md) |
| `@braintrust/langchain-js` | LangChain.js callback handler integration for automatic Braintrust logging. | [![npm: @braintrust/langchain-js](https://img.shields.io/npm/v/%40braintrust%2Flangchain-js.svg)](https://www.npmjs.com/package/@braintrust/langchain-js) | [integrations/langchain-js/README.md](integrations/langchain-js/README.md) |
| `@braintrust/openai-agents` | OpenAI Agents tracing integration for Braintrust. | [![npm: @braintrust/openai-agents](https://img.shields.io/npm/v/%40braintrust%2Fopenai-agents.svg)](https://www.npmjs.com/package/@braintrust/openai-agents) | [integrations/openai-agents-js/README.md](integrations/openai-agents-js/README.md) |
| `@braintrust/otel` | OpenTelemetry span processor and compatibility helpers for Braintrust tracing. | [![npm: @braintrust/otel](https://img.shields.io/npm/v/%40braintrust%2Fotel.svg)](https://www.npmjs.com/package/@braintrust/otel) | [integrations/otel-js/README.md](integrations/otel-js/README.md) |
| `@braintrust/temporal` | Temporal client/worker plugin and workflow interceptors for Braintrust tracing. | [![npm: @braintrust/temporal](https://img.shields.io/npm/v/%40braintrust%2Ftemporal.svg)](https://www.npmjs.com/package/@braintrust/temporal) | [integrations/temporal-js/README.md](integrations/temporal-js/README.md) |

## Documentation

For more information, check out the [docs](https://www.braintrust.dev/docs):
- TypeScript SDK docs: https://www.braintrust.dev/docs/reference/sdks/typescript
- Release notes: https://www.braintrust.dev/docs/reference/release-notes

## License

Apache-2.0

## Repository History

This repository was previously named `braintrust-sdk` and was renamed to `braintrust-sdk-javascript`. The rename happened when Python SDK code was split out of this repository. The Python code now lives in a dedicated [`Braintrust Python SDK`](https://github.com/braintrustdata/braintrust-sdk-python) repository.

## Contributors

Thanks to everyone who contributed to the Braintrust JavaScript SDK!

- [TypeScript](https://www.braintrust.dev/docs/reference/sdks/typescript)
<a href="https://github.com/braintrustdata/braintrust-sdk-javascript/graphs/contributors">
<img src="https://contributors-img.web.app/image?repo=braintrustdata/braintrust-sdk-javascript" />
</a>
32 changes: 15 additions & 17 deletions integrations/browser-js/README.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,12 @@
# Braintrust Browser SDK

[![npm version](https://img.shields.io/npm/v/%40braintrust%2Fbrowser.svg)](https://www.npmjs.com/package/@braintrust/browser)

Official browser-only SDK for [Braintrust](https://braintrust.dev).

This is an integration package that provides browser-optimized builds of the Braintrust SDK with AsyncLocalStorage polyfill support for standard browsers.

Note: This package supports limited functionality in the browser. We're aware of a CORS issue that users may run into if they use this outside of the braintrust.dev domain. If you'd like to use this package in your own domain, please provide us feedback [support@braintrust.dev](mailto:support@braintrust.dev)
This package supports limited functionality in the browser. There is a known CORS limitation when used outside the `braintrust.dev` domain. If you need this for your own domain, contact [support@braintrust.dev](mailto:support@braintrust.dev).

## Installation

Expand All @@ -16,9 +18,14 @@ pnpm add @braintrust/browser braintrust
yarn add @braintrust/browser braintrust
```

Note: `braintrust` is a peer dependency and must be installed alongside `@braintrust/browser`.
Note: Install both packages so your browser integration and main SDK stay on compatible versions.

## Requirements

## Usage
- `braintrust` (installed alongside `@braintrust/browser`)
- `zod` (`^3.25.34 || ^4.0`)

## Quickstart

```typescript
import * as braintrust from "@braintrust/browser";
Expand All @@ -37,29 +44,20 @@ const result = await braintrust.traced(
);
```

## Differences from Main Package

This package:

- **Includes** `als-browser` polyfill for AsyncLocalStorage (bundled)
- **Requires** `braintrust`

## When to Use
## Notes

Use `@braintrust/browser` when:

- Building browser-only applications
- Need AsyncLocalStorage support in standard browsers
- Needing AsyncLocalStorage support in standard browsers

Use `braintrust` directly when:

- Building Node.js applications
- Using in Next.js or other full-stack frameworks (with proper module resolution)
- Need CLI tools or filesystem access

## Features
- Using Next.js or other full-stack frameworks
- Needing CLI tools or filesystem access

All browser-compatible features from the main SDK:
Browser-compatible features include:

- Logging and tracing
- Experiments and datasets
Expand Down
24 changes: 6 additions & 18 deletions integrations/langchain-js/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# @braintrust/langchain-js

[![npm version](https://img.shields.io/npm/v/%40braintrust%2Flangchain-js.svg)](https://www.npmjs.com/package/@braintrust/langchain-js)

SDK for integrating [Braintrust](https://braintrust.dev) with [LangChain.js](https://langchain.com/js). This package provides a callback handler to automatically log LangChain.js executions to Braintrust.

## Installation
Expand All @@ -17,16 +19,14 @@ pnpm add @braintrust/langchain-js
- Node.js >= 16
- LangChain.js >= 0.3.42 (incl. 1.0.0)

## Usage
## Quickstart

First, make sure you have your Braintrust API key set in your environment:

```bash
export BRAINTRUST_API_KEY="your-api-key"
```

### Basic Usage

```typescript
import { ChatOpenAI } from "@langchain/openai";
import {
Expand Down Expand Up @@ -77,19 +77,7 @@ The callback handler supports logging for:

Review the [LangChain.js documentation](https://js.langchain.com/docs/how_to/#callbacks) for more information on how to use callbacks.

## Development

Contributions are welcomed!

```bash
git clone https://github.com/braintrustdata/sdk.git
## Documentation

cd sdk/integrations/langchain-js

pnpm install

# work on the code

pnpm test
pnpm build
```
- Braintrust docs: [https://www.braintrust.dev/docs](https://www.braintrust.dev/docs)
- LangChain callback docs: [https://js.langchain.com/docs/how_to/#callbacks](https://js.langchain.com/docs/how_to/#callbacks)
18 changes: 17 additions & 1 deletion integrations/openai-agents-js/README.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,25 @@
# @braintrust/openai-agents

[![npm version](https://img.shields.io/npm/v/%40braintrust%2Fopenai-agents.svg)](https://www.npmjs.com/package/@braintrust/openai-agents)

SDK for integrating Braintrust with OpenAI Agents.

## Installation

```bash
npm install braintrust @braintrust/openai-agents @openai/agents
# or
yarn add braintrust @braintrust/openai-agents @openai/agents
# or
pnpm add braintrust @braintrust/openai-agents @openai/agents
```

## Usage
## Requirements

- `braintrust`
- `@openai/agents`

## Quickstart

```typescript
import { initLogger } from "braintrust";
Expand Down Expand Up @@ -52,3 +63,8 @@ new OpenAIAgentsTraceProcessor(options?: OpenAIAgentsTraceProcessorOptions)

- `logger?: Logger` - A Braintrust `Span`, `Experiment`, or `Logger` to use for logging. If undefined, the current span, experiment, or logger will be selected exactly as in `startSpan`.
- `maxTraces?: number` - Maximum number of concurrent traces to keep in memory (default: 10000). When exceeded, oldest traces are evicted using LRU policy to prevent memory leaks.

## Documentation

- Braintrust docs: [https://www.braintrust.dev/docs](https://www.braintrust.dev/docs)
- OpenAI Agents docs: [https://openai.github.io/openai-agents-js/](https://openai.github.io/openai-agents-js/)
43 changes: 6 additions & 37 deletions integrations/otel-js/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# @braintrust/otel

[![npm version](https://img.shields.io/npm/v/%40braintrust%2Fotel.svg)](https://www.npmjs.com/package/@braintrust/otel)

SDK for integrating [Braintrust](https://braintrust.dev) with [OpenTelemetry](https://opentelemetry.io/). This package enables you to send OpenTelemetry spans to Braintrust for logging and observability, and provides seamless interoperability between Braintrust and OpenTelemetry tracing contexts.

## Installation
Expand Down Expand Up @@ -29,16 +31,16 @@ npm install @opentelemetry/context-async-hooks

This package supports both OpenTelemetry 1.x and 2.x versions.

Note: You may run into a known [OpenTelemetry browser bug](https://github.com/open-telemetry/opentelemetry-js/issues/3545)) with `@opentelemetry/exporter-trace-otlp-http` < 0.205.0. If you encounter this issue, upgrade the `@opentelemetry/exporter-trace-otlp-http` and related packages to a minimum of:
Note: You may run into a known [OpenTelemetry browser bug](https://github.com/open-telemetry/opentelemetry-js/issues/3545) with `@opentelemetry/exporter-trace-otlp-http` < 0.205.0. If you encounter this issue, upgrade the `@opentelemetry/exporter-trace-otlp-http` and related packages to a minimum of:

- `@opentelemetry/exporter-trace-otlp-http` >= 0.205.0
- `@opentelemetry/sdk-trace-base` >= 2.1.0
- `@opentelemetry/api` >= 2.1.0
- `@opentelemetry/core` >= 2.1.0

It's **important** you do not mix `@opentelemetry/core` and related packages with 1.x with 2.x packages.
It's **important** that `@opentelemetry/core` and related packages all use the same major version (1.x or 2.x).

## Overview
## Capabilities

This integration provides two main capabilities:

Expand Down Expand Up @@ -218,40 +220,7 @@ The following environment variables can be used to configure the integration:
- `BRAINTRUST_PARENT`: Parent project identifier (e.g., `project_name:my_project`)
- `BRAINTRUST_API_URL`: Braintrust API endpoint (defaults to `https://api.braintrust.dev`)

## Development

This package is designed to work with both OpenTelemetry 1.x and 2.x. The development structure ensures compatibility across versions:

### Testing Structure

- **`src/`**: Main source code that is compatible with both OpenTelemetry 1.x and 2.x
- **`otel-v1/`**: Test package that runs tests against OpenTelemetry 1.x dependencies
- **`otel-v2/`**: Test package that runs tests against OpenTelemetry 2.x dependencies

### Running Tests

```bash
# Run tests for both OpenTelemetry versions
pnpm test

# Run tests for OpenTelemetry 1.x only
pnpm test:v1

# Run tests for OpenTelemetry 2.x only
pnpm test:v2
```

### How It Works

The `otel-v1` and `otel-v2` directories are separate packages that:

1. Reference the main `src/` code from the parent directory
2. Install different versions of OpenTelemetry dependencies
3. Run the same test suites against their respective OpenTelemetry versions

This approach ensures the package works correctly with both major OpenTelemetry versions while maintaining a single source codebase.

## Learn More
## Documentation

- [Braintrust Documentation](https://www.braintrust.dev/docs)
- [OpenTelemetry Documentation](https://opentelemetry.io/docs/)
Expand Down
Loading
Loading