-
Notifications
You must be signed in to change notification settings - Fork 3.4k
[EP ABI] Signatures for compatibility info methods #25454
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
853e43c
to
ed770dd
Compare
LGTM |
Looks good. Thanks |
* | ||
* \since Version 1.23. | ||
*/ | ||
OrtStatus*(ORT_API_CALL* ValidateCompiledModelCompatibilityInfo)(_In_ OrtEpFactory* this_ptr, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should use ORT_API2_STATUS so we get consistent usage of noexcept
.
const char*(ORT_API_CALL* GetCompiledModelCompatibilityInfo)(_In_ OrtEp* this_ptr, | ||
_In_ const OrtGraph* graph); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should use ORT_API_T so we get consistent usage of noexcept
.
### Description This change corrects the signatures on the new precompiled model compatibility APIs to use the recommended macros. ### Motivation and Context <!-- - Why is this change required? What problem does it solve? - If it fixes an open issue, please link to the issue here. --> This fix addresses PR feedback from #25454 . --------- Co-authored-by: Aditya Rastogi <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Description
This PR proposes two new ABI methods to better support precompiled models. We introduce the concept of a compatibility information string, which is a piece of data produced by an EP implementor to describe the stack that may have been used to compile a model. The
GetCompiledModelCompatibilityInfo
function on theOrtEp
interface creates this string.ORT will get the compatibility information string when precompiling a model and will store the output in the model's metadata. When creating sessions against any precompiled models, ORT will use the
ValidateCompiledModelCompatibilityInfo
function to confirm that the model is still EP-compatible. The EP implementor decides how compatibility is defined for their stack and communicates this via the possible states in theOrtCompiledModelCompatibility
enum.The validation API is stored on the
OrtEpFactory
to allow for cases where a model may not be present on the device yet (e.g., perhaps an application is trying to decide on a pre-compiled model to download based on the state of the system).Motivation and Context
The APIs proposed in this PR address two requirements: