Skip to content

.Net: Add Provider Support to ONNX Connector (+CUDA Sample)#12861

Merged
markwallace-microsoft merged 11 commits into
microsoft:mainfrom
loicbtd:features/dotnet_onnx_provider_choice
Aug 7, 2025
Merged

.Net: Add Provider Support to ONNX Connector (+CUDA Sample)#12861
markwallace-microsoft merged 11 commits into
microsoft:mainfrom
loicbtd:features/dotnet_onnx_provider_choice

Conversation

@loicbtd

@loicbtd loicbtd commented Aug 5, 2025

Copy link
Copy Markdown
Contributor

Motivation and Context

  1. Why is this change required?
    Currently, users can configure ONNX execution providers by manually creating a genai_config.json file next to the ONNX model.

  2. What problem does it solve?
    This change eliminates the need for manual JSON configuration files by providing a programmatic API for provider configuration, making it easier for developers to configure ONNX execution providers directly through code with the extension methods.

  3. What scenario does it contribute to?
    This improves the developer experience when working with ONNX models in Semantic Kernel, particularly for scenarios where:

    • Developers want to dynamically select providers based on runtime conditions
    • Teams prefer code-based configuration over file-based configuration
  4. If it fixes an open issue, please link to the issue here.
    Closes .Net: Add Provider Choice Support to ONNX Connector #12828

Description

Changes Made:

  • New Provider Class: Added Provider class to encapsulate provider configuration with Id and Options properties
  • Enhanced Chat Completion Service: Extended OnnxRuntimeGenAIChatCompletionService to accept the providers parameter.
  • Updated Builder Extension: Modified OnnxKernelBuilderExtensions.AddOnnxRuntimeGenAIChatCompletion to accept the providers parameter.
  • New Demo Project: Created OnnxWithProviderChoice demo showcasing the new provider configuration API
  • Package Configuration: Add the Microsoft.ML.OnnxRuntime.Gpu package version in the Directory.Packages.props

Design Approach:

The implementation leverages ONNX Runtime GenAI's existing Config API to programmatically set providers and their options. The Provider class provides a clean abstraction for specifying provider ID (e.g., "cuda", "cpu") and custom options. During service initialization, the providers are configured using the underlying ONNX Runtime GenAI configuration system.

Usage Example:

builder.AddOnnxRuntimeGenAIChatCompletion(
    modelId: "onnx",
    modelPath: modelPath,
    providers: [new Provider { Id = "cuda" }]
);

Backward Compatibility:
This change maintains full backward compatibility. Existing code continues to work without modification, and the manual genai_config.json approach remains supported.

Contribution Checklist

@moonbox3 moonbox3 added .NET Issue or Pull requests regarding .NET code kernel Issues or pull requests impacting the core kernel labels Aug 5, 2025
@loicbtd loicbtd marked this pull request as ready for review August 5, 2025 12:23
@loicbtd loicbtd requested a review from a team as a code owner August 5, 2025 12:23
@rogerbarreto

Copy link
Copy Markdown
Member

@loicbtd Thanks for the contribution, I added some extra updates to this PR related to your change and other outstanding changes I had plan myself to add to the connector. Overall LTGM.

@loicbtd

loicbtd commented Aug 6, 2025

Copy link
Copy Markdown
Contributor Author

Thanks for the update 👍

@rogerbarreto rogerbarreto changed the title .Net: Add Provider Choice Support to ONNX Connector .Net: Add Provider Support to ONNX Connector (+CUDA Sample) Aug 6, 2025
Comment thread dotnet/SK-dotnet.slnx Outdated
@markwallace-microsoft markwallace-microsoft added this pull request to the merge queue Aug 7, 2025
Merged via the queue into microsoft:main with commit fe30769 Aug 7, 2025
@loicbtd loicbtd deleted the features/dotnet_onnx_provider_choice branch August 27, 2025 14:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation kernel Issues or pull requests impacting the core kernel .NET Issue or Pull requests regarding .NET code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

.Net: Add Provider Choice Support to ONNX Connector

4 participants