You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
For users that have more than one GPU (e.g. Laptop with RTX 4060 8GB - gpu-id 0 and eGPU-RTX 5060 Ti 16GB - gpu-id 1) and would like to select the more performant GPU (e.g. gpu-id 1)
Changes
Add optional command line parameter --gpu-id <GPU_ID> to select the GPU. Defaults to 0 for backwards compatibility
Changes hardcoding of device_id=0 to device_type=device_type and device id=gpu_id where appropriate
Review against current upstream/dev for merge safety:
This PR is not safe to merge into dev as-is. Please repost to dev branch.
Merge conflict against dev
The PR is currently based on main, and GitHub reports it clean for that base, but merging the PR head into current upstream/dev conflicts in:
app/processors/workers/frame_worker.py
Repro:
git fetch upstream dev pull/231/head:review-pr-231
git merge-tree upstream/dev review-pr-231
Result includes:
CONFLICT (content): Merge conflict in app/processors/workers/frame_worker.py
Fix: rebase/retarget this PR onto the latest dev branch and resolve the frame_worker.py conflict, then rerun the processor tests.
CPU provider can receive an invalid I/O binding device id when --gpu-id is nonzero
ModelsProcessor.switch_providers_priority() keeps self.gpu_id unchanged when switching to CPU, while many ONNX Runtime I/O bindings now pass device_type=self.models_processor.device_type together with device_id=self.models_processor.gpu_id. Example paths include app/processors/face_masks.py where CPU mode would bind device_type=cpu and device_id=<gpu id>.
That is risky because CPU I/O bindings should use device id 0; running with --gpu-id 1 and then selecting CPU can pass device_id=1 to CPU bindings.
Fix: centralize the binding device id, for example:
Then use self.models_processor.binding_device_id for all ORT bind_input() / bind_output() calls. Alternatively, update gpu_id-based binding sites to use 0 whenever device_type == cpu.
Changes have been rebased to dev branch, resolved the conflict and resolved the CPU provider can receive an invalid I/O binding device id when --gpu-id is nonzero issue. I've also tested this locally on my system to ensure it is still functional
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
For users that have more than one GPU (e.g. Laptop with RTX 4060 8GB - gpu-id 0 and eGPU-RTX 5060 Ti 16GB - gpu-id 1) and would like to select the more performant GPU (e.g. gpu-id 1)
Changes
--gpu-id <GPU_ID>to select the GPU. Defaults to 0 for backwards compatibilitydevice_id=0todevice_type=device_typeanddevice id=gpu_idwhere appropriate