File upstream ORT bug: .onnx serializer mishandles Loop-subgraph initializer scoping

[christopherthompson81]

Why this is open

PR #59 worked around the underlying ORT bug by falling back to the .ort binary format for Loop-bearing graphs. The underlying bug — .onnx serializer can't faithfully round-trip a graph with a Loop subgraph — is still live in ORT and likely affects other users. Worth reporting upstream so it can actually get fixed.

Reproducer

From the investigation in docs/ort_loop_cache_investigation.md (Run 1):

import onnxruntime as ort

src = "conditional_decoder_loop.onnx"   # any graph with a Loop op
opts = ort.SessionOptions()
opts.graph_optimization_level = ort.GraphOptimizationLevel.ORT_DISABLE_ALL  # bug happens at every level
opts.optimized_model_filepath = "round_trip.onnx"

# Write: succeeds
ort.InferenceSession(src, opts, providers=["CUDAExecutionProvider", "CPUExecutionProvider"])

# Re-load: fails
reload_opts = ort.SessionOptions()
reload_opts.graph_optimization_level = ort.GraphOptimizationLevel.ORT_DISABLE_ALL
ort.InferenceSession("round_trip.onnx", reload_opts, providers=["CUDAExecutionProvider", "CPUExecutionProvider"])
# → INVALID_GRAPH: ... initializer name is not unique

Same OptimizedModelFilePath write with session.save_model_format = ORT round-trips cleanly at every opt level. So it's specifically the .onnx writer's handling of Loop-subgraph initializer scoping, not the optimizer or loader.

Draft body for the upstream issue

A starting point — refine before posting to https://github.com/microsoft/onnxruntime/issues:

Title: .onnx serializer of OptimizedModelFilePath produces an unreadable file for graphs containing Loop subgraphs

Affected versions: confirmed on ORT 1.23.2 and 1.24.4 (Python and .NET); likely older.

Description: When OptimizedModelFilePath is set and the input graph contains a Loop op with a body subgraph that has its own initializers, the resulting .onnx file fails to load with "initializer name is not unique" on every GraphOptimizationLevel including ORT_DISABLE_ALL. Switching to session.save_model_format = ORT produces a file that round-trips cleanly at every level, so the bug is isolated to the .onnx serializer for subgraph-scoped initializers (not the optimizer or loader). Source graph passes onnx.checker.check_model and contains no duplicate initializer names at any scope.

Reproducer: see above (plus a minimal-merged-graph attachment).

Workaround in our codebase: Vernacula PR #59 added a layered cache (.onnx primary → .ort fallback after detected round-trip failure → sentinel last). Works but costs ~1.5 s of additional load time per affected graph vs a fixed .onnx writer.

What we know

• Bug happens at every opt level including DISABLE_ALL → not an optimizer pass issue, it's the .onnx writer for subgraph-scoped initializers.
• Source graph has zero duplicate initializer names at any scope (counted both outer and body).
• The names ORT complains about (const_transpose_optimizer_token_NN) are created by ORT's own transpose-folding pass — they're cosmetic noise, not contributors to the failure.
• .ort binary format works at every opt level — different code path inside ORT.
• A minimal repro graph is in /tmp/cb_dyn5/conditional_decoder_loop.onnx (~168 MB; can be regenerated via scripts/_export_utils/merge_cond_decoder_loop.py).

Related

• Issue OrtSessionBuilder cache fails to round-trip ORT-optimized models with Loop subgraphs #56 (closed by PR OrtSessionBuilder: skip caching after failed round-trip (closes #56) #58 with sentinel workaround; PR Real fix for #56: layered .onnx → .ort → sentinel cache #59 with proper layered fallback)
• docs/ort_loop_cache_investigation.md (full probe results and dead-end findings)
• scripts/_export_utils/merge_cond_decoder_loop.py (script that produces the affected graph)