Skip to content

🔧 Harden installed QDMI runtime staging #2231

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
burgholzer wants to merge 1 commit into from
Draft

🔧 Harden installed QDMI runtime staging #2231

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 40 additions & 0 deletions .agent/plans/qdmi-installed-consumer-deployment-c3.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# Installed QDMI runtime deployment

Status: independently rebased and locally validated; hosted Windows CI pending.

## Motivation and scope

Installed CMake consumers need the same complete runtime layout as in-tree
applications. The existing mqt_copy_qdmi_runtime helper must stage imported
Client, driver, device libraries, manifests, provider assets and Windows DLLs.

This is Core PR #2231 on #2230, targeting Core 4.1 / QDMI 1.4. It does not
depend on metadata removal, batching, or payload capabilities. No payload-format
header is introduced by the driver workstream.

## Decisions

Reuse the existing imported-device fixture as a real find_package consumer. Use
copy_if_different for local and imported runtime targets. Imported targets must
not become build dependencies. For Windows, retain the non-imported
linker-language-bearing closure used to compute transitive imported DLLs.
Preserve device metadata and asset copying, and use the build RPATH while
running staged build-tree applications.

This changes deployment only. It does not add Client APIs, providers, compiler
behavior, or a second package-consumer harness.

## Validation

Run the release build, both imported-device fixture tests, and the full native
suite. The fixture must resolve installed Core targets, execute the consumer,
and compare staged libraries, manifest, assets and Windows dependency files.
Check that the helper disables BUILD_WITH_INSTALL_RPATH on its consumer. Run
repository lint; Windows hosted CI remains necessary for real DLL loading.

The release build and native suite pass: 3,873 tests pass and one existing
optional-device test skips. Both installed-consumer fixture tests pass.

Keep useful commits, human attribution and existing review threads. Do not
create archive branches or request reviews. Published artifacts require released
dependency pins.
8 changes: 5 additions & 3 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -110,9 +110,10 @@ releases may include breaking changes.
open targeted sessions through the packaged driver's optional private
configuration extension ([#2230]) ([**@burgholzer**])
- 💥 Load one replaceable QDMI 1.4 Client driver through a validated function
table, split `MQT::CoreQDMI` from the packaged `MQT::CoreQDMIDriver`, and use
stable Client device IDs across C++, Python, MLIR, Qiskit, PennyLane, and
Slurm ([#2229]) ([**@burgholzer**])
table, split `MQT::CoreQDMI` from the packaged `MQT::CoreQDMIDriver`, and
stage their complete runtime dependencies for installed consumers. Use stable
Client device IDs across C++, Python, MLIR, Qiskit, PennyLane, and Slurm
([#2229], [#2231]) ([**@burgholzer**])
- 💥 Drop support for x86 macOS and stop publishing the respective wheels
([#2259]) ([**@denialhaag**])
- ⬆️ Raise the macOS deployment target to 13.3 to enable `std::format` in libc++
Expand Down Expand Up @@ -914,6 +915,7 @@ for previous changelogs._
[#2257]: https://github.com/munich-quantum-toolkit/core/pull/2257
[#2246]: https://github.com/munich-quantum-toolkit/core/pull/2246
[#2240]: https://github.com/munich-quantum-toolkit/core/pull/2240
[#2231]: https://github.com/munich-quantum-toolkit/core/pull/2231
[#2230]: https://github.com/munich-quantum-toolkit/core/pull/2230
[#2229]: https://github.com/munich-quantum-toolkit/core/pull/2229
[#2232]: https://github.com/munich-quantum-toolkit/core/pull/2232
Expand Down
60 changes: 43 additions & 17 deletions cmake/AddMQTQDMIDevice.cmake
View file
Open in desktop

@burgholzer burgholzer Aug 26, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This feels overly complicated and slightly confusing, especially because of the terminology being used. "closure", "staging", "concrete", "runtime".
I think this is important work to ensure a robust installation and usage; but it needs to be prepared in a better fashion.

Our top distribution channel is the mqt-core Python package. It is either directly used, which needs to work with the shipped libraries. Or it is loaded in other Python-only libraries (also not critical), or it is used to feed a find_package(mqt-core) call in another project (such as MQT QCEC, MQT DDSIM, or MQT QMAP). The latter case is probably the one that is hardest to get right across all systems when it comes to using the QDMI-related functionality.
However, I still have the feeling that most of that was already working quite alright.

The second distribution channel is likely installations in HPC centers. These would typically build MQT Core from source via tools like spack. I do not really foresee major problems there with the solution that we previously had in place (maybe I am missing something though). The key aspect here is that this is almost exclusively for Linux (x86 and arm64), while the Python distribution channel is much broader and needs to cover macOS and Windows.

Maybe one can simplify around that information and already include these thoughts in the iteration on the lower layers.

Original file line number Diff line number Diff line change
Expand Up @@ -147,7 +147,33 @@ function(mqt_get_qdmi_device_targets result)
PARENT_SCOPE)
endfunction()

# Copy in-tree QDMI runtime libraries and manifests beside a runtime consumer.
function(_mqt_qdmi_runtime_files result target)
set(runtime_files "$<TARGET_FILE:${target}>")
if(WIN32)
get_target_property(concrete_target ${target} ALIASED_TARGET)
if(NOT concrete_target)
set(concrete_target ${target})
endif()
get_target_property(imported ${concrete_target} IMPORTED)
if(imported)
string(MAKE_C_IDENTIFIER "${concrete_target}-runtime-closure" runtime_closure_target)
if(NOT TARGET ${runtime_closure_target})
add_library(${runtime_closure_target} MODULE EXCLUDE_FROM_ALL
"${CMAKE_CURRENT_FUNCTION_LIST_FILE}")
set_property(TARGET ${runtime_closure_target} PROPERTY LINKER_LANGUAGE CXX)
target_link_libraries(${runtime_closure_target} PRIVATE ${target})
endif()
set(runtime_files "$<TARGET_RUNTIME_DLLS:${runtime_closure_target}>")
else()
list(APPEND runtime_files "$<TARGET_RUNTIME_DLLS:${target}>")
endif()
endif()
set(${result}
${runtime_files}
PARENT_SCOPE)
endfunction()

# Copy the QDMI Client library, driver, device libraries, and manifests beside a runtime consumer.
function(mqt_copy_qdmi_runtime target)
if(NOT TARGET ${target})
message(FATAL_ERROR "Unknown QDMI runtime consumer target: ${target}")
Expand All @@ -163,19 +189,22 @@ function(mqt_copy_qdmi_runtime target)
if(NOT runtime_concrete_target)
set(runtime_concrete_target ${runtime_target})
endif()
get_target_property(runtime_imported ${runtime_concrete_target} IMPORTED)
if(NOT runtime_imported AND NOT consumer_target STREQUAL runtime_concrete_target)
add_dependencies(${consumer_target} ${runtime_concrete_target})
set(runtime_files "$<TARGET_FILE:${runtime_target}>")
if(WIN32)
list(APPEND runtime_files "$<TARGET_RUNTIME_DLLS:${runtime_target}>")
if(NOT consumer_target STREQUAL runtime_concrete_target)
string(MAKE_C_IDENTIFIER "${consumer_target}-${runtime_target}-copy" runtime_copy_target)
if(NOT TARGET ${runtime_copy_target})
get_target_property(runtime_imported ${runtime_concrete_target} IMPORTED)
_mqt_qdmi_runtime_files(runtime_files ${runtime_target})
add_custom_target(
${runtime_copy_target}
COMMAND ${CMAKE_COMMAND} -E make_directory "$<TARGET_FILE_DIR:${consumer_target}>"
COMMAND ${CMAKE_COMMAND} -E copy_if_different ${runtime_files}
"$<TARGET_FILE_DIR:${consumer_target}>"
COMMAND_EXPAND_LISTS)
if(NOT runtime_imported)
add_dependencies(${runtime_copy_target} ${runtime_concrete_target})
endif()
endif()
add_custom_command(
TARGET ${consumer_target}
POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_if_different ${runtime_files}
"$<TARGET_FILE_DIR:${consumer_target}>"
COMMAND_EXPAND_LISTS)
add_dependencies(${consumer_target} ${runtime_copy_target})
endif()
endif()
endforeach()
Expand Down Expand Up @@ -222,10 +251,7 @@ function(mqt_copy_qdmi_runtime target)
if(NOT device_imported)
add_dependencies(${target} ${device})
endif()
set(device_files "$<TARGET_FILE:${device}>")
if(WIN32 AND NOT device_imported)
list(APPEND device_files "$<TARGET_RUNTIME_DLLS:${device}>")
endif()
_mqt_qdmi_runtime_files(device_files ${device})
add_custom_command(
TARGET ${target}
POST_BUILD
Expand Down
10 changes: 8 additions & 2 deletions docs/qdmi/configuration.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -247,7 +247,12 @@ mqt_copy_qdmi_runtime(my-application MQT::CoreQDMIScDevice MQT::CoreQDMI_DDSIM_D

Inside an MQT Core build, omitting the device list copies every device
registered through `mqt_configure_qdmi_device`. Installed consumers select the
exported device targets they need, as shown above.
exported device targets they need, as shown above. The helper stages
`MQT::CoreQDMI` and `MQT::CoreQDMIDriver` when those targets are available,
whether they come from the current build or an installed CMake package. On
Windows, it also stages the transitive runtime DLLs of both Core libraries and
each selected device. During a build, the consumer uses its build RPATH rather
than an unrelated final install RPATH.

An external device implementation does not need MQT Core as a build dependency.
It can export its stable ID and prefix as target metadata:
Expand All @@ -267,4 +272,5 @@ When `mqt_copy_qdmi_runtime` receives that built or imported target, it
generates the relocatable manifest while copying the device. Device targets may
also declare `RUNTIME_FILES` through `mqt_configure_qdmi_device`; their exported
`QDMI_RUNTIME_FILES` basenames are copied beside the provider as part of the
same operation.
same operation. The result is one colocated directory with the Client, Driver,
provider, manifest, provider assets, and required Windows DLLs.
4 changes: 3 additions & 1 deletion test/qdmi/driver/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -139,11 +139,13 @@ if(TARGET MQT::CoreQDMIDriver)
add_dependencies(${TARGET_NAME} mqt-core-qdmi-session-device)

set(imported_device_build_dir "${CMAKE_CURRENT_BINARY_DIR}/imported-device-consumer")
set(imported_device_install_dir "${imported_device_build_dir}/install")
set(imported_device_configure_command
${CMAKE_COMMAND} -S "${CMAKE_CURRENT_SOURCE_DIR}/imported_device" -B
"${imported_device_build_dir}" -G "${CMAKE_GENERATOR}"
"-DMQT_CORE_QDMI_DEVICE_TARGETS=${metadata_device_export}"
"-DMQT_CORE_QDMI_HELPER=${PROJECT_SOURCE_DIR}/cmake/AddMQTQDMIDevice.cmake")
"-DMQT_CORE_BUILD_DIR=${CMAKE_BINARY_DIR}" "-DMQT_CORE_BUILD_CONFIG=$<CONFIG>"
"-DMQT_CORE_INSTALL_DIR=${imported_device_install_dir}" "-DCMAKE_BUILD_TYPE=$<CONFIG>")
if(CMAKE_GENERATOR_PLATFORM)
list(APPEND imported_device_configure_command -A "${CMAKE_GENERATOR_PLATFORM}")
endif()
Expand Down
154 changes: 149 additions & 5 deletions test/qdmi/driver/imported_device/CMakeLists.txt
View file
Open in desktop

@burgholzer burgholzer Aug 26, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This definitely goes out of its way to prove that something works.
I'd personally be fine if this was demonstrated locally but not directly included in the test suite.

Original file line number Diff line number Diff line change
Expand Up @@ -9,14 +9,76 @@
cmake_minimum_required(VERSION 3.24)
project(mqt-core-qdmi-imported-device-test LANGUAGES CXX)

if(NOT MQT_CORE_QDMI_DEVICE_TARGETS OR NOT MQT_CORE_QDMI_HELPER)
message(FATAL_ERROR "Device targets and the QDMI helper are required")
if(NOT MQT_CORE_BUILD_CONFIG
OR NOT MQT_CORE_BUILD_DIR
OR NOT MQT_CORE_INSTALL_DIR
OR NOT MQT_CORE_QDMI_DEVICE_TARGETS)
message(FATAL_ERROR "The MQT Core build, install prefix, and device target export are required")
endif()

file(REMOVE_RECURSE "${MQT_CORE_INSTALL_DIR}")
foreach(component IN ITEMS qdmi_Runtime qdmi_Development mqt-core_Runtime mqt-core_Development)
execute_process(
COMMAND ${CMAKE_COMMAND} --install "${MQT_CORE_BUILD_DIR}" --config "${MQT_CORE_BUILD_CONFIG}"
--prefix "${MQT_CORE_INSTALL_DIR}" --component "${component}"
RESULT_VARIABLE install_result)
if(install_result)
message(FATAL_ERROR "Failed to stage installed component ${component}")
endif()
endforeach()

list(PREPEND CMAKE_PREFIX_PATH "${MQT_CORE_INSTALL_DIR}")
find_package(mqt-core CONFIG REQUIRED PATHS "${MQT_CORE_INSTALL_DIR}" NO_DEFAULT_PATH)
include("${MQT_CORE_QDMI_DEVICE_TARGETS}")
include("${MQT_CORE_QDMI_HELPER}")

get_target_property(client_imported MQT::CoreQDMI IMPORTED)
get_target_property(driver_imported MQT::CoreQDMIDriver IMPORTED)
if(NOT client_imported OR NOT driver_imported)
message(FATAL_ERROR "The installed QDMI Client and Driver targets must be imported")
endif()

set(device mqt-core-qdmi-metadata-device)
if(WIN32)
set(client_runtime_dependency "${CMAKE_CURRENT_BINARY_DIR}/client-runtime-dependency.dll")
set(client_runtime_import_library "${CMAKE_CURRENT_BINARY_DIR}/client-runtime-dependency.lib")
file(WRITE "${client_runtime_dependency}" "client runtime dependency\n")
file(WRITE "${client_runtime_import_library}" "")
add_library(qdmi-client-runtime-dependency SHARED IMPORTED)
set_target_properties(
qdmi-client-runtime-dependency PROPERTIES IMPORTED_IMPLIB "${client_runtime_import_library}"
IMPORTED_LOCATION "${client_runtime_dependency}")
set_property(
TARGET MQT::CoreQDMI
APPEND
PROPERTY INTERFACE_LINK_LIBRARIES qdmi-client-runtime-dependency)

set(driver_runtime_dependency "${CMAKE_CURRENT_BINARY_DIR}/driver-runtime-dependency.dll")
set(driver_runtime_import_library "${CMAKE_CURRENT_BINARY_DIR}/driver-runtime-dependency.lib")
file(WRITE "${driver_runtime_dependency}" "driver runtime dependency\n")
file(WRITE "${driver_runtime_import_library}" "")
add_library(qdmi-driver-runtime-dependency SHARED IMPORTED)
set_target_properties(
qdmi-driver-runtime-dependency PROPERTIES IMPORTED_IMPLIB "${driver_runtime_import_library}"
IMPORTED_LOCATION "${driver_runtime_dependency}")
set_property(
TARGET MQT::CoreQDMIDriver
APPEND
PROPERTY INTERFACE_LINK_LIBRARIES qdmi-driver-runtime-dependency)

set(device_runtime_dependency "${CMAKE_CURRENT_BINARY_DIR}/device-runtime-dependency.dll")
set(device_runtime_import_library "${CMAKE_CURRENT_BINARY_DIR}/device-runtime-dependency.lib")
file(WRITE "${device_runtime_dependency}" "device runtime dependency\n")
file(WRITE "${device_runtime_import_library}" "")
add_library(qdmi-device-runtime-dependency SHARED IMPORTED)
set_target_properties(
qdmi-device-runtime-dependency PROPERTIES IMPORTED_IMPLIB "${device_runtime_import_library}"
IMPORTED_LOCATION "${device_runtime_dependency}")
set_property(
TARGET ${device}
APPEND
PROPERTY INTERFACE_LINK_LIBRARIES qdmi-device-runtime-dependency)
endif()

get_target_property(device_id ${device} QDMI_DEVICE_ID)
get_target_property(device_prefix ${device} QDMI_DEVICE_PREFIX)
get_target_property(runtime_files ${device} QDMI_RUNTIME_FILES)
Expand All @@ -26,13 +88,95 @@ if(NOT device_id STREQUAL "test.metadata-only"
message(FATAL_ERROR "Exported QDMI device metadata was not preserved")
endif()

file(WRITE "${CMAKE_CURRENT_BINARY_DIR}/main.cpp" "int main() { return 0; }\n")
file(
WRITE "${CMAKE_CURRENT_BINARY_DIR}/main.cpp"
[=[#ifdef _WIN32
#include <windows.h>

int main(int argc, char** argv) {
if (argc != 3) {
return 1;
}
const auto client = LoadLibraryA(argv[1]);
const auto driver = LoadLibraryA(argv[2]);
if (driver != nullptr) {
FreeLibrary(driver);
}
if (client != nullptr) {
FreeLibrary(client);
}
return client != nullptr && driver != nullptr ? 0 : 1;
}
#else
#include "qdmi/Client.hpp"

int main() {
const qdmi::Session session;
return 0;
}
#endif
]=])
add_executable(imported-device-consumer "${CMAKE_CURRENT_BINARY_DIR}/main.cpp")
if(NOT WIN32)
target_link_libraries(imported-device-consumer PRIVATE MQT::CoreQDMI)
endif()
set_property(TARGET imported-device-consumer PROPERTY BUILD_WITH_INSTALL_RPATH TRUE)
mqt_copy_qdmi_runtime(imported-device-consumer ${device})
get_target_property(build_with_install_rpath imported-device-consumer BUILD_WITH_INSTALL_RPATH)
if(build_with_install_rpath)
message(FATAL_ERROR "QDMI runtime staging must use the build RPATH")
endif()

string(MAKE_C_IDENTIFIER "imported-device-consumer-${device}" manifest_stem)
set(manifest_name "${manifest_stem}.qdmi.json")
add_custom_command(
TARGET imported-device-consumer
POST_BUILD
COMMAND
${CMAKE_COMMAND} -E compare_files
"$<TARGET_FILE_DIR:imported-device-consumer>/$<TARGET_FILE_NAME:MQT::CoreQDMI>"
"$<TARGET_FILE:MQT::CoreQDMI>"
COMMAND
${CMAKE_COMMAND} -E compare_files
"$<TARGET_FILE_DIR:imported-device-consumer>/$<TARGET_FILE_NAME:MQT::CoreQDMIDriver>"
"$<TARGET_FILE:MQT::CoreQDMIDriver>"
COMMAND
${CMAKE_COMMAND} -E compare_files
"$<TARGET_FILE_DIR:imported-device-consumer>/$<TARGET_FILE_NAME:${device}>"
"$<TARGET_FILE:${device}>"
COMMAND
${CMAKE_COMMAND} -E compare_files
"$<TARGET_FILE_DIR:imported-device-consumer>/metadata-runtime.json"
"$<TARGET_FILE_DIR:${device}>/metadata-runtime.json")
"$<TARGET_FILE_DIR:${device}>/metadata-runtime.json"
COMMAND
${CMAKE_COMMAND} -E compare_files
"$<TARGET_FILE_DIR:imported-device-consumer>/${manifest_name}"
"${CMAKE_CURRENT_BINARY_DIR}/$<CONFIG>/${manifest_name}")

if(WIN32)
add_custom_command(
TARGET imported-device-consumer
POST_BUILD
COMMAND
${CMAKE_COMMAND} -E compare_files
"$<TARGET_FILE_DIR:imported-device-consumer>/$<TARGET_FILE_NAME:qdmi-client-runtime-dependency>"
"$<TARGET_FILE:qdmi-client-runtime-dependency>"
COMMAND
${CMAKE_COMMAND} -E compare_files
"$<TARGET_FILE_DIR:imported-device-consumer>/$<TARGET_FILE_NAME:qdmi-driver-runtime-dependency>"
"$<TARGET_FILE:qdmi-driver-runtime-dependency>"
COMMAND
${CMAKE_COMMAND} -E compare_files
"$<TARGET_FILE_DIR:imported-device-consumer>/$<TARGET_FILE_NAME:qdmi-device-runtime-dependency>"
"$<TARGET_FILE:qdmi-device-runtime-dependency>")
endif()

add_custom_command(
TARGET imported-device-consumer
POST_BUILD
COMMAND
${CMAKE_COMMAND} -E env
"MQT_CORE_QDMI_DRIVER=$<TARGET_FILE_DIR:imported-device-consumer>/$<TARGET_FILE_NAME:MQT::CoreQDMIDriver>"
"$<TARGET_FILE:imported-device-consumer>"
"$<TARGET_FILE_DIR:imported-device-consumer>/$<TARGET_FILE_NAME:MQT::CoreQDMI>"
"$<TARGET_FILE_DIR:imported-device-consumer>/$<TARGET_FILE_NAME:MQT::CoreQDMIDriver>")
Loading