Skip to content

Conversation

@bopeng1234
Copy link
Contributor

pass the total_seq_length into the internal::GQA, and keep it there.

Details:

if not, the onnx perf test will report error log, that onnx model input is not matching with ov compiled model, because the total_sequence_length is dropped by us.

onnxruntime::openvino_ep::BasicBackend::BasicBackend] [OpenVINO-EP] Loaded model to the plugin
←[1;31m2025-11-21 16:39:44.3499912 [E:onnxruntime:, inference_session.cc:2522 onnxruntime::InferenceSession::Initialize::<lambda_41c120c1b014dbad66e82a684676c107>::operator ()] Exception during initialization: E:\pengbo\ortgenai_rep\onnxruntime\onnxruntime\core/providers/openvino/backends/basic_backend.h:87 auto __cdecl onnxruntime::openvino_ep::OnnxToOvNetworkBindings::{ctor}::<lambda_1>::operator ()<class std::vector<struct onnxruntime::openvino_ep::ParameterInfo,class std::allocator<struct onnxruntime::openvino_ep::ParameterInfo> >,class std::vector<class ov::Output<class ov::Node const >,class std::allocator<class ov::Output<class ov::Node const > > >>(class std::vector<struct onnxruntime::openvino_ep::ParameterInfo,class std::allocator<struct onnxruntime::openvino_ep::ParameterInfo> > &,const class std::unordered_map<class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> >,unsigned int,struct std::hash<class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > >,struct std::equal_to<class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > >,class std::allocator<struct std::pair<class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > const ,unsigned int> > > &,const class std::vector<class ov::Output<class ov::Node const >,class std::allocator<class ov::Output<class ov::Node const > > > &) const matched_names was false.

[OpenVINO-EP] Input names mismatch between OpenVINO and ONNX. total_seq_len doesn't exist in the list of OpenVINO input tensor names
←[m

Tickets:

@github-actions github-actions bot added category: transformations OpenVINO Runtime library - Transformations category: ONNX FE OpenVINO ONNX FrontEnd category: NPU OpenVINO NPU plugin category: NPUW NPUW plugin labels Nov 21, 2025
@bopeng1234 bopeng1234 marked this pull request as ready for review November 21, 2025 09:45
@bopeng1234 bopeng1234 requested review from a team as code owners November 21, 2025 09:45
@bopeng1234 bopeng1234 requested review from mryzhov and removed request for a team November 21, 2025 09:45
…t into the internal::GQA, and keep it there for later use
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

category: NPU OpenVINO NPU plugin category: NPUW NPUW plugin category: ONNX FE OpenVINO ONNX FrontEnd category: transformations OpenVINO Runtime library - Transformations

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant