Skip to content

[EP ABI] Signatures for compatibility info methods #25454

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jul 22, 2025

Conversation

adrastogi
Copy link
Contributor

@adrastogi adrastogi commented Jul 18, 2025

Description

This PR proposes two new ABI methods to better support precompiled models. We introduce the concept of a compatibility information string, which is a piece of data produced by an EP implementor to describe the stack that may have been used to compile a model. The GetCompiledModelCompatibilityInfo function on the OrtEp interface creates this string.

ORT will get the compatibility information string when precompiling a model and will store the output in the model's metadata. When creating sessions against any precompiled models, ORT will use the ValidateCompiledModelCompatibilityInfo function to confirm that the model is still EP-compatible. The EP implementor decides how compatibility is defined for their stack and communicates this via the possible states in the OrtCompiledModelCompatibility enum.

The validation API is stored on the OrtEpFactory to allow for cases where a model may not be present on the device yet (e.g., perhaps an application is trying to decide on a pre-compiled model to download based on the state of the system).

Motivation and Context

The APIs proposed in this PR address two requirements:

  1. Apps that have an already pre-compiled model on device need a way to determine if the pre-compiled app is still valid (given the EPs / drivers / etc. on the system).
  2. Apps may have many different pre-compiled versions of a model stored on a remote server, and want to figure out which of those models they should download for the device where they are running.

@adrastogi adrastogi force-pushed the adrastogi/compat-string-interface branch from 853e43c to ed770dd Compare July 21, 2025 17:19
@adrastogi adrastogi marked this pull request as ready for review July 21, 2025 20:21
@javier-intel
Copy link
Contributor

LGTM

@thevishalagarwal
Copy link
Contributor

Looks good. Thanks

@jywu-msft jywu-msft merged commit af4b232 into main Jul 22, 2025
87 of 91 checks passed
@jywu-msft jywu-msft deleted the adrastogi/compat-string-interface branch July 22, 2025 05:00
*
* \since Version 1.23.
*/
OrtStatus*(ORT_API_CALL* ValidateCompiledModelCompatibilityInfo)(_In_ OrtEpFactory* this_ptr,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should use ORT_API2_STATUS so we get consistent usage of noexcept.

Comment on lines +699 to +700
const char*(ORT_API_CALL* GetCompiledModelCompatibilityInfo)(_In_ OrtEp* this_ptr,
_In_ const OrtGraph* graph);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should use ORT_API_T so we get consistent usage of noexcept.

skottmckay pushed a commit that referenced this pull request Jul 23, 2025
### Description
This change corrects the signatures on the new precompiled model
compatibility APIs to use the recommended macros.

### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->
This fix addresses PR feedback from #25454 .

---------

Co-authored-by: Aditya Rastogi <[email protected]>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants