-
Notifications
You must be signed in to change notification settings - Fork 62
Add LangChain integration to main package with auto_instrument() support #1320
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
Closed
Changes from all commits
Commits
Show all changes
3 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
4 changes: 2 additions & 2 deletions
4
integrations/langchain-py/src/braintrust_langchain/__init__.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,4 +1,4 @@ | ||
| from .callbacks import BraintrustCallbackHandler | ||
| from .context import set_global_handler | ||
| from .context import clear_global_handler, set_global_handler | ||
|
|
||
| __all__ = ["BraintrustCallbackHandler", "set_global_handler"] | ||
| __all__ = ["BraintrustCallbackHandler", "set_global_handler", "clear_global_handler"] | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -13,6 +13,7 @@ | |
| - Agno | ||
| - Claude Agent SDK | ||
| - DSPy | ||
| - LangChain | ||
| """ | ||
|
|
||
| import braintrust | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,35 @@ | ||
| """ | ||
| Example: LangChain with auto_instrument() | ||
|
|
||
| This example demonstrates automatic tracing of LangChain operations | ||
| using braintrust.auto_instrument(). | ||
|
|
||
| Run with: python examples/langchain/auto.py | ||
| """ | ||
|
|
||
| import braintrust | ||
|
|
||
| # One-line instrumentation - call this BEFORE importing LangChain | ||
| results = braintrust.auto_instrument() | ||
| print(f"LangChain instrumented: {results.get('langchain', False)}") | ||
|
|
||
| # Initialize logging | ||
| logger = braintrust.init_logger(project="langchain-auto-example") | ||
|
|
||
| # Now import LangChain - all operations are automatically traced | ||
| from langchain_core.prompts import ChatPromptTemplate | ||
| from langchain_openai import ChatOpenAI | ||
|
|
||
| # Create a simple chain | ||
| prompt = ChatPromptTemplate.from_template("What is {number} + {number}?") | ||
| model = ChatOpenAI(model="gpt-4o-mini") | ||
| chain = prompt | model | ||
|
|
||
| # Wrap in a span to get a link | ||
| with braintrust.start_span(name="langchain_auto_example") as span: | ||
| print("Running LangChain chain...") | ||
| result = chain.invoke({"number": "5"}) | ||
| print(f"Result: {result.content}") | ||
| span.log(output=result.content) | ||
|
|
||
| print(f"\nView trace: {span.link()}") |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,62 @@ | ||
| """ | ||
| Example: LangChain with manual setup | ||
|
|
||
| This example demonstrates using setup_langchain() for global handler registration | ||
| and BraintrustCallbackHandler for per-call tracing. | ||
|
|
||
| Run with: python examples/langchain/manual.py | ||
| """ | ||
|
|
||
| import braintrust | ||
| from braintrust.wrappers.langchain import ( | ||
| BraintrustCallbackHandler, | ||
| set_global_handler, | ||
| setup_langchain, | ||
| ) | ||
|
|
||
| # Initialize logging | ||
| logger = braintrust.init_logger(project="langchain-manual-example") | ||
|
|
||
| # Method 1: Global handler via setup_langchain() | ||
| # This registers a handler that traces ALL LangChain operations automatically | ||
| print("Method 1: Global handler") | ||
| setup_langchain() | ||
|
|
||
| from langchain_core.prompts import ChatPromptTemplate | ||
| from langchain_openai import ChatOpenAI | ||
|
|
||
| prompt = ChatPromptTemplate.from_template("What is the capital of {country}?") | ||
| model = ChatOpenAI(model="gpt-4o-mini") | ||
| chain = prompt | model | ||
|
|
||
| # All operations are traced automatically | ||
| result = chain.invoke({"country": "France"}) | ||
| print(f" Capital: {result.content}\n") | ||
|
|
||
|
|
||
| # Method 2: Per-call handler | ||
| # This is useful when you want more control over which calls are traced | ||
| print("Method 2: Per-call handler") | ||
|
|
||
| # Create a handler with a specific logger | ||
| handler = BraintrustCallbackHandler(logger=logger) | ||
|
|
||
| # Pass the handler explicitly to chain.invoke() | ||
| result = chain.invoke( | ||
| {"country": "Japan"}, | ||
| config={"callbacks": [handler]} | ||
| ) | ||
| print(f" Capital: {result.content}\n") | ||
|
|
||
|
|
||
| # Method 3: Global handler with custom handler instance | ||
| print("Method 3: Custom global handler") | ||
|
|
||
| # Create a custom handler and set it globally | ||
| custom_handler = BraintrustCallbackHandler(logger=logger) | ||
| set_global_handler(custom_handler) | ||
|
|
||
| result = chain.invoke({"country": "Brazil"}) | ||
| print(f" Capital: {result.content}\n") | ||
|
|
||
| print("Check Braintrust dashboard for traces!") |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -43,6 +43,7 @@ | |
| "openai", | ||
| "openai-agents", | ||
| # pydantic_ai is NOT included here - it has dedicated test sessions with version-specific handling | ||
| # langchain is NOT included here - it has dedicated test sessions with version-specific handling | ||
| "autoevals", | ||
| "braintrust_core", | ||
| "litellm", | ||
|
|
@@ -73,6 +74,8 @@ | |
| DSPY_VERSIONS = (LATEST,) | ||
| # temporalio 1.19.0+ requires Python >= 3.10; skip Python 3.9 entirely | ||
| TEMPORAL_VERSIONS = (LATEST, "1.20.0", "1.19.0") | ||
| # langchain requires Python >= 3.10 | ||
| LANGCHAIN_VERSIONS = (LATEST, "0.3.27") | ||
|
|
||
|
|
||
| @nox.session() | ||
|
|
@@ -193,6 +196,20 @@ def test_dspy(session, version): | |
| _run_tests(session, f"{WRAPPER_DIR}/test_dspy.py") | ||
|
|
||
|
|
||
| @nox.session() | ||
| @nox.parametrize("version", LANGCHAIN_VERSIONS, ids=LANGCHAIN_VERSIONS) | ||
| def test_langchain(session, version): | ||
| """Test LangChain integration.""" | ||
| # langchain requires Python >= 3.10 | ||
| if sys.version_info < (3, 10): | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. we don't support 3.9 anymore |
||
| session.skip("langchain tests require Python >= 3.10") | ||
| _install_test_deps(session) | ||
| _install(session, "langchain", version) | ||
| session.install("langchain-openai", "langchain-anthropic", "langgraph") | ||
| _run_tests(session, f"{WRAPPER_DIR}/test_langchain.py") | ||
| _run_core_tests(session) | ||
|
|
||
|
|
||
| @nox.session() | ||
| @nox.parametrize("version", AUTOEVALS_VERSIONS, ids=AUTOEVALS_VERSIONS) | ||
| def test_autoevals(session, version): | ||
|
|
@@ -267,6 +284,8 @@ def pylint(session): | |
| session.install("opentelemetry.instrumentation.openai") | ||
| # langsmith is needed for the wrapper module but not in VENDOR_PACKAGES | ||
| session.install("langsmith") | ||
| # langchain dependencies for the langchain wrapper | ||
| session.install("langchain", "langchain-openai", "langchain-anthropic", "langgraph") | ||
|
|
||
| result = session.run("git", "ls-files", "**/*.py", silent=True, log=False) | ||
| files = result.strip().splitlines() | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
not sure we needed this change. should we just kill the source in the repo? the published pypi may be enough. perhaps we can save a tag or branch if we need to provide patch fixes.