-
Notifications
You must be signed in to change notification settings - Fork 1
Refactor API #10
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Refactor API #10
Conversation
Add I prefix to interface names.
Simplifies scorer instantiation by removing the IScorer.Of factory method and making FunctionScorer public. Updates the IScorer interface to use a Name property instead of GetName(), and modernizes code to use newer C# features. These changes improve API clarity and usability for defining custom scorers.
Refactors environment override handling in BraintrustConfig to use tuple-based syntax and nullable values, improving type safety and clarity. Updates constructors and tests to support explicit null overrides and removes legacy key-value array logic.
Replaces the custom ICursor interface with C# async streams for dataset case iteration. This modernizes the API, improves scalability, and simplifies usage by leveraging await foreach and IAsyncEnumerable throughout the codebase.
This helps discoverability (it is not expected to find static methods on interfaces) and allows users to take advantage of type inference.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This pull request refactors the Braintrust SDK API to modernize the codebase with async/await patterns, improved type safety, and better organization. The refactoring eliminates sync-over-async anti-patterns, consolidates types into separate files following one-type-per-file convention, and replaces cursor-based iteration with IAsyncEnumerable.
Changes:
- Converted all API methods to async, removing sync-over-async patterns throughout IBraintrustApiClient and BraintrustApiClient
- Refactored config API from variadic string pairs to tuple-based overrides for better type safety and inference
- Replaced cursor-based dataset iteration with IAsyncEnumerable pattern for cleaner async enumeration
- Renamed interfaces with I prefix (Task → ITask, Scorer → IScorer, Dataset → IDataset) for .NET conventions
- Extracted types into separate files (ApiException, InstrumentedChatClient, FunctionScorer, MockBraintrustApiClient)
- Converted Score, TaskResult, and EvalResult to readonly structs for better performance
- Updated BaseConfig to use IParsable constraint, simplifying type conversion logic
Reviewed changes
Copilot reviewed 39 out of 39 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| src/Braintrust.Sdk/Api/BraintrustApiClient.cs | Converted all methods to async, added ConfigureAwait patterns |
| src/Braintrust.Sdk/Api/IBraintrustApiClient.cs | Updated interface to async Task-returning methods |
| src/Braintrust.Sdk/Api/ApiException.cs | Extracted to separate file following one-type-per-file |
| src/Braintrust.Sdk/Config/BaseConfig.cs | Refactored to use IParsable constraint, changed to abstract class |
| src/Braintrust.Sdk/Config/BraintrustConfig.cs | Updated to tuple-based config overrides API |
| src/Braintrust.Sdk/Eval/Eval.cs | Converted Build to BuildAsync, RunAsync uses IAsyncEnumerable |
| src/Braintrust.Sdk/Eval/IDataset.cs | Replaced ICursor with IAsyncEnumerable |
| src/Braintrust.Sdk/Eval/DatasetInMemoryImpl.cs | Implemented GetCasesAsync with iterator pattern |
| src/Braintrust.Sdk/Eval/IScorer.cs | Extracted interface, renamed from Scorer |
| src/Braintrust.Sdk/Eval/FunctionScorer.cs | Made public, extracted to separate file |
| src/Braintrust.Sdk/Eval/Score.cs | Converted to readonly record struct |
| src/Braintrust.Sdk/Eval/TaskResult.cs | Converted to readonly record struct |
| src/Braintrust.Sdk/Eval/EvalResult.cs | Converted to readonly struct with ToString override |
| src/Braintrust.Sdk/Braintrust.cs | Changed ProjectUri to async GetProjectUriAsync |
| src/Braintrust.Sdk/Instrumentation/OpenAI/InstrumentedChatClient.cs | Extracted to separate file |
| tests/Braintrust.Sdk.Tests/Eval/MockBraintrustApiClient.cs | Extracted mock to separate file with async methods |
| examples/*/Program.cs | Updated all examples to use async Main and await async APIs |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| public override ClientResult<ChatCompletion> CompleteChat(IEnumerable<ChatMessage> messages, ChatCompletionOptions? options = null, CancellationToken cancellationToken = default) | ||
| { | ||
| // Start a span for the chat completion | ||
| var activity = _activitySource.StartActivity("Chat Completion", ActivityKind.Client); |
Copilot
AI
Jan 12, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This variable is manually disposed in a finally block - consider a C# using statement as a preferable resource management technique.
| public override async Task<ClientResult<ChatCompletion>> CompleteChatAsync(IEnumerable<ChatMessage> messages, ChatCompletionOptions? options = null, CancellationToken cancellationToken = default) | ||
| { | ||
| // Start a span for the chat completion | ||
| var activity = _activitySource.StartActivity("Chat Completion", ActivityKind.Client); |
Copilot
AI
Jan 12, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This variable is manually disposed in a finally block - consider a C# using statement as a preferable resource management technique.
| OrganizationAndProjectInfo? orgAndProject; | ||
|
|
||
| if (_projectId == null) | ||
| { | ||
| orgAndProject = await _apiClient.GetProjectAndOrgInfo().ConfigureAwait(false) | ||
| ?? throw new InvalidOperationException("Unable to retrieve project and org info"); | ||
| } | ||
| else | ||
| { | ||
| orgAndProject = await _apiClient.GetProjectAndOrgInfo(_projectId).ConfigureAwait(false) | ||
| ?? throw new InvalidOperationException($"Invalid project id: {_projectId}"); | ||
| } | ||
|
|
Copilot
AI
Jan 12, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Both branches of this 'if' statement write to the same variable - consider using '?' to express intent better.
| OrganizationAndProjectInfo? orgAndProject; | |
| if (_projectId == null) | |
| { | |
| orgAndProject = await _apiClient.GetProjectAndOrgInfo().ConfigureAwait(false) | |
| ?? throw new InvalidOperationException("Unable to retrieve project and org info"); | |
| } | |
| else | |
| { | |
| orgAndProject = await _apiClient.GetProjectAndOrgInfo(_projectId).ConfigureAwait(false) | |
| ?? throw new InvalidOperationException($"Invalid project id: {_projectId}"); | |
| } | |
| OrganizationAndProjectInfo? orgAndProject = _projectId == null | |
| ? await _apiClient.GetProjectAndOrgInfo().ConfigureAwait(false) | |
| ?? throw new InvalidOperationException("Unable to retrieve project and org info") | |
| : await _apiClient.GetProjectAndOrgInfo(_projectId).ConfigureAwait(false) | |
| ?? throw new InvalidOperationException($"Invalid project id: {_projectId}"); |
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Copilot <[email protected]>
IprefixIAsyncEnumerableIParsable<T>inBaseConfigto make code simpler and cleaner