diff --git a/README.md b/README.md index 3034b88c..1a1717d4 100644 --- a/README.md +++ b/README.md @@ -1,60 +1,57 @@ -# Braintrust SDK +# Braintrust Python SDKs -[Braintrust](https://www.braintrust.dev/) is a platform for evaluating and shipping AI products. To learn more about Braintrust or sign up for free, -visit our [website](https://www.braintrust.dev/) or check out the [docs](https://www.braintrust.dev/docs). +[Braintrust](https://www.braintrust.dev/) is a platform for evaluating and shipping AI products. Learn more at [braintrust.dev](https://www.braintrust.dev/) and in the [docs](https://www.braintrust.dev/docs). -This repository contains the Python SDK for Braintrust. The SDK includes utilities to: +This repository contains Braintrust's Python SDKs and integrations, including: -- Log experiments and datasets to Braintrust -- Run evaluations (via the `Eval` framework) -- Manage an on-premises installation of Braintrust +- The main `braintrust` SDK package in [`./py`](./py) +- Integration packages under [`./integrations`](./integrations) +- Examples, tests, and local development tooling for Python SDK development -## Quickstart: Python +## Quickstart -Install the library with pip. +Install the main SDK and scorer package: ```bash pip install braintrust autoevals ``` -Then, create a file named `eval_tutorial.py` with the following code: +Create `tutorial_eval.py`: ```python -from braintrust import Eval from autoevals import LevenshteinScorer +from braintrust import Eval Eval( - "Say Hi Bot", - data=lambda: [ - { - "input": "Foo", - "expected": "Hi Foo", - }, - { - "input": "Bar", - "expected": "Hello Bar", - }, - ], # Replace with your eval dataset - task=lambda input: "Hi " + input, # Replace with your LLM call - scores=[LevenshteinScorer], + "Say Hi Bot", + data=lambda: [ + {"input": "Foo", "expected": "Hi Foo"}, + {"input": "Bar", "expected": "Hello Bar"}, + ], + task=lambda input: "Hi " + input, + scores=[LevenshteinScorer], ) ``` -Then, run the following command: +Run it: ```bash -BRAINTRUST_API_KEY= \ - braintrust eval eval_tutorial.py +BRAINTRUST_API_KEY= braintrust eval tutorial_eval.py ``` -## Integrations +## Packages -Braintrust provides integrations with several popular AI development tools and platforms: - -- **LangChain Python**: Integration for logging LangChain Python executions to Braintrust. [Learn more](integrations/langchain-py) +| Package | Purpose | PyPI | Docs | +| --- | --- | --- | --- | +| `braintrust` | Core Python SDK for logging, tracing, evals, and CLI workflows. | [![PyPI - braintrust](https://img.shields.io/pypi/v/braintrust.svg)](https://pypi.org/project/braintrust/) | [py/README.md](py/README.md) | +| `braintrust-langchain` | LangChain callback integration for automatic Braintrust logging. | [![PyPI - braintrust-langchain](https://img.shields.io/pypi/v/braintrust-langchain.svg)](https://pypi.org/project/braintrust-langchain/) | [integrations/langchain-py/README.md](integrations/langchain-py/README.md) | +| `braintrust-adk` | Deprecated Google ADK integration package. New ADK support lives in `braintrust`. | [![PyPI - braintrust-adk](https://img.shields.io/pypi/v/braintrust-adk.svg)](https://pypi.org/project/braintrust-adk/) | [integrations/adk-py/README.md](integrations/adk-py/README.md) | ## Documentation -For more information, check out the [docs](https://www.braintrust.dev/docs): +- Python SDK docs: https://www.braintrust.dev/docs/reference/sdks/python +- Release notes: https://www.braintrust.dev/docs/reference/release-notes + +## License -- [Python](https://www.braintrust.dev/docs/reference/sdks/python) +Apache-2.0 diff --git a/integrations/adk-py/README.md b/integrations/adk-py/README.md index b0c53158..e1552283 100644 --- a/integrations/adk-py/README.md +++ b/integrations/adk-py/README.md @@ -1,5 +1,7 @@ # braintrust-adk (DEPRECATED) +[![PyPI version](https://img.shields.io/pypi/v/braintrust-adk.svg)](https://pypi.org/project/braintrust-adk/) + SDK for integrating [Braintrust](https://braintrust.dev) with [Google ADK (Agent Development Kit)](https://github.com/google/adk-python). This package provides automatic tracing and logging of ADK agent executions to Braintrust. > **This package is deprecated.** The Google ADK integration is now included in the main [`braintrust`](https://pypi.org/project/braintrust/) package. @@ -37,10 +39,10 @@ pip install braintrust-adk ## Requirements -- Python >= 3.9 +- Python >= 3.10 - Google ADK >= 1.14.1 -## Quick Start +## Quickstart The `braintrust-adk` integration automatically traces your ADK agents' execution, including: @@ -49,8 +51,6 @@ The `braintrust-adk` integration automatically traces your ADK agents' execution - Parallel execution flows - Multi-step agent reasoning -### Basic Usage - ```python from google.adk.agents import LlmAgent from braintrust_adk import setup_adk @@ -73,9 +73,9 @@ response = agent.send_message("What's the weather like in New York?") print(response.text) ``` -### Advanced Configuration +## Advanced Configuration -#### Using Project ID +### Using Project ID If you know your Braintrust project ID, you can use it directly: @@ -86,7 +86,7 @@ setup_adk( ) ``` -#### Custom Tools with Tracing +### Custom Tools with Tracing Other braintrust functions like `traced` work seamlessly with this integration. @@ -118,16 +118,12 @@ response = agent.send_message( ) ``` -### Manual Patching +## Manual Patching The `setup_adk` will automatically patch Google ADK Runner, Agent, and Flow classes to automatically trace all agent interactions. If you prefer to manually patch classes, you can use the `wrap_agent`, `wrap_runner`, and `wrap_flow` functions. Take a look at the [manual example](./examples/manual.py). Note that, as of writing, `adk web` does not support [custom Runners](https://github.com/google/adk-web/issues/72) and you will need to use `setup_adk` if you would like LLM traces. -## Examples - -The `examples/` directory contains complete working examples: - ## Viewing Traces in Braintrust Once you've set up the integration, you can view your traces in the Braintrust dashboard: @@ -140,28 +136,8 @@ Once you've set up the integration, you can view your traces in the Braintrust d - Token usage and latency metrics - Any errors or warnings -## Development - -To contribute to this integration: - -```bash -# Clone the repository -git clone https://github.com/braintrustdata/braintrust-sdk-python.git -cd sdk/integrations/adk-py - -uv sync - -# Run examples -cd examples - -# simple programmatic agent call -uv run manual.py - -# or use the adk web UI -uv run adk web --port 8888 -``` - -## Related Resources +## Documentation - [Braintrust Documentation](https://www.braintrust.dev/docs) +- [Braintrust Python SDK Documentation](https://www.braintrust.dev/docs/reference/sdks/python) - [Google ADK Documentation](https://github.com/google/genai-agent-dev-kit) diff --git a/integrations/langchain-py/README.md b/integrations/langchain-py/README.md index ef290891..ec00daef 100644 --- a/integrations/langchain-py/README.md +++ b/integrations/langchain-py/README.md @@ -1,5 +1,7 @@ # braintrust-langchain +[![PyPI version](https://img.shields.io/pypi/v/braintrust-langchain.svg)](https://pypi.org/project/braintrust-langchain/) + SDK for integrating [Braintrust](https://braintrust.dev) with [LangChain](https://langchain.com/). This package provides a callback handler to automatically log LangChain executions to Braintrust. ## Installation @@ -10,10 +12,10 @@ pip install braintrust-langchain ## Requirements -- Python >= 3.9 -- LangChain >= 0.1.0 +- Python >= 3.10 +- LangChain >= 0.3.27 -## Usage +## Quickstart First, make sure you have your Braintrust API key set in your environment: @@ -21,8 +23,6 @@ First, make sure you have your Braintrust API key set in your environment: export BRAINTRUST_API_KEY="your-api-key" ``` -### Basic Usage - ```python import asyncio from braintrust import init_logger @@ -54,6 +54,8 @@ if __name__ == "__main__": asyncio.run(main()) ``` +## Passing Handlers Explicitly + If you'd prefer to pass the callback handler to specific LangChain calls instead of setting it globally, you can do so using the `callbacks` config option: ```python @@ -83,18 +85,8 @@ The callback handler supports logging for: Review the [LangChain documentation](https://python.langchain.com/docs/modules/callbacks/) for more information on how to use callbacks. -## Development - -Contributions are welcomed! - -```bash -git clone https://github.com/braintrustdata/sdk.git - -cd sdk/integrations/langchain-py - -pip install -e ".[dev]" +## Documentation -# work on the code - -pytest -``` +- Braintrust docs: https://www.braintrust.dev/docs +- Braintrust Python SDK docs: https://www.braintrust.dev/docs/reference/sdks/python +- LangChain callback docs: https://python.langchain.com/docs/modules/callbacks/ diff --git a/py/README.md b/py/README.md index 4d9eb072..5d5e0c26 100644 --- a/py/README.md +++ b/py/README.md @@ -1,57 +1,73 @@ -## Braintrust +# Braintrust Python SDK -A Python library for logging data to Braintrust. `braintrust` is distributed as -a [library on PyPI](https://pypi.org/project/braintrust/). It is open source and -[available on GitHub](https://github.com/braintrustdata/braintrust-sdk-python/tree/main/py). +[![PyPI version](https://img.shields.io/pypi/v/braintrust.svg)](https://pypi.org/project/braintrust/) -### Quickstart +The official Python SDK for logging, tracing, and evaluating AI applications with [Braintrust](https://www.braintrust.dev/). -Install the library with pip. +## Installation -```bash -pip install braintrust -``` - -**Performance tip**: For 3-5x faster JSON serialization, install with the optional `performance` extra: - -```bash -pip install braintrust[performance] -``` - -Or install `orjson` separately: +Install the SDK: ```bash -pip install orjson +pip install braintrust ``` -The SDK automatically detects and uses orjson if available, with seamless fallback to standard json. See [ORJSON_OPTIMIZATION.md](ORJSON_OPTIMIZATION.md) for details. +## Quickstart -Then, run a simple experiment with the following code (replace `YOUR_API_KEY` with -your Braintrust API key): +Run a simple evaluation: ```python from braintrust import Eval + def is_equal(expected, output): return expected == output + Eval( - "Say Hi Bot", - data=lambda: [ - { - "input": "Foo", - "expected": "Hi Foo", - }, - { - "input": "Bar", - "expected": "Hello Bar", - }, - ], # Replace with your eval dataset - task=lambda input: "Hi " + input, # Replace with your LLM call - scores=[is_equal], + "Say Hi Bot", + data=lambda: [ + {"input": "Foo", "expected": "Hi Foo"}, + {"input": "Bar", "expected": "Hello Bar"}, + ], + task=lambda input: "Hi " + input, + scores=[is_equal], ) ``` -# Performance Optimization +Then run: + +```bash +BRAINTRUST_API_KEY= braintrust eval tutorial_eval.py +``` + +## Optional Extras + +Install extras as needed for specific workflows: + +```bash +pip install "braintrust[cli]" +pip install "braintrust[openai-agents]" +pip install "braintrust[otel]" +pip install "braintrust[temporal]" +pip install "braintrust[all]" +``` + +Available extras: + +- `performance`: installs `orjson` for faster JSON serialization +- `cli`: installs optional dependencies used by the Braintrust CLI +- `openai-agents`: installs OpenAI Agents integration support +- `otel`: installs OpenTelemetry integration dependencies +- `temporal`: installs Temporal integration dependencies +- `all`: installs all optional extras + +## Documentation + +- Python SDK docs: https://www.braintrust.dev/docs/reference/sdks/python +- Braintrust docs: https://www.braintrust.dev/docs +- Source code: https://github.com/braintrustdata/braintrust-sdk-python/tree/main/py + +## License -For 3-5x faster JSON serialization, install `orjson`. The SDK automatically detects and uses orjson if available, with seamless fallback to standard json. +Apache-2.0