Add vhost-device-media backend for virtio-media#944
Open
aesteve-rh wants to merge 9 commits into
Open
Conversation
aesteve-rh
requested review from
dorindabassey,
epilys,
stefano-garzarella,
stsquad and
vireshk
as code owners
aesteve-rh
force-pushed
the
virtio-media
branch
from
c3a8789 to
4a6c56d
Compare
epilys
reviewed
Mar 10, 2026
Collaborator
|
Thank you for this PR @aesteve-rh |
Member
|
@dorindabassey virtio-video is still under development (see virtio-comment list for v10). It's not replaced by virtio-media. |
aesteve-rh
force-pushed
the
virtio-media
branch
from
4a6c56d to
5ddd48f
Compare
aesteve-rh
force-pushed
the
virtio-media
branch
from
6ffe82f to
0727f40
Compare
Contributor
Author
|
Some |
4 tasks
aesteve-rh
force-pushed
the
virtio-media
branch
from
0727f40 to
42eaca8
Compare
Member
@aesteve-rh yeah, we need to manually update |
Contributor
Author
|
Ah, yes. Makes sense. I think I can manage. Thanks for the pointer! |
Contributor
Author
|
CI update: rust-vmm/rust-vmm-ci#205 |
Member
|
@aesteve-rh it seems there is still something missing in the CI container. I suggest running it locally following https://github.com/rust-vmm/rust-vmm-ci#running-the-tests-locally |
Contributor
Author
|
Another +1 for this one? rust-vmm/rust-vmm-ci#207 edit: |
This commit introduces vhost-user-media device. The virtio-media device first appeared in virtio-spec v1.4 specification[1]. It standardizes how guest applications communicate with media devices on the host, such as encoders, decoders, and cameras. The protocol relies on the V4L2 (Video4Linux2) specification to format the communication between the guest and host, exposing the host's media devices to the virtualized environment. The low-level implementation of this backend for the virtio-media protocol is provided by the device crate from the virtio-media repository[2]. [1] https://docs.oasis-open.org/virtio/virtio/v1.4/virtio-v1.4.html#x1-82200022 [2] https://github.com/chromeos/virtio-media Signed-off-by: Albert Esteve <aesteve@redhat.com>
This commit introduces a memory allocator for managing the shared memory regions used by the virtio-media device. The allocator is responsible for finding and managing free regions within the shared memory, ensuring that buffers are allocated without conflicts. The new `MediaAllocator` tracks available memory pools and provides a mechanism for allocating and releasing memory blocks of a requested size, while also handling alignment requirements. This replaces the previous, more error-prone approach of managing address ranges in a simple Vec. Signed-off-by: Albert Esteve <aesteve@redhat.com>
This commit refactors the vhost-device-media crate to make each backend an optional feature. This allows users to compile only the backends they need, reducing the final binary size and dependency footprint. The following features have been introduced: - `simple-capture`: Enables the simple V4L2 capture device. - `v4l2-proxy`: Enables the V4L2 proxy device. - `xen`: Various rust-vmm dependencies support Xen platforms. The `v4l2-proxy` feature is enabled by default. The `v4l2r` dependency is now optional and only included when a v4l2r-based feature is enabled. The source code has been updated with `#[cfg]` attributes to support conditional compilation of the backends. Signed-off-by: Albert Esteve <aesteve@redhat.com>
This commit introduces a new backend that uses FFmpeg for software-based video decoding. This provides a hardware-independent way to test and use the virtio-media device for video decoding tasks. The new `FfmpegDecoder` backend leverages the `virtio-media-ffmpeg-decoder` crate from the virtio-media repository [1]. It is exposed as a new backend option and can decode common video formats such as H.264, VP8, and VP9 without requiring any specific host hardware. This addition is useful for development and testing and also serves as a reference implementation for a virtio-media decoder device. [1] https://github.com/chromeos/virtio-media/tree/main/extras/ffmpeg-decoder Signed-off-by: Albert Esteve <aesteve@redhat.com>
This change reorganizes the vhost-device-media crate to follow a similar pattern used by other vhost-device crates, including the template, splitting the codebase into a library component (lib.rs) and a minimal binary entry point (main.rs). A new src/lib.rs was added containing the core backend logic, including backend creation functions (create_*_device_config), and serve functions for each backend type. The main.rs file was simplified to focus solely on CLI argument parsing and delegating to the library. BackendType and VuMediaError are now publicly re-exported from the library crate to allow external callers to drive start_backend without going through the binary. Signed-off-by: Albert Esteve <aesteve@redhat.com>
Add unit tests to existing code for vhost-device-media covering: - descriptor_chain: DescriptorChainReader/Writer read/write paths and boundary behaviour. - vhu_media_thread: VhostUserMediaThread queue processing helpers. - vhu_media: VuMediaBackend construction, handle_event dispatch, and error-type conversions. - lib: CLI argument parsing, V4l2DeviceType path resolution, and socket-binding smoke tests for each backend. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Albert Esteve <aesteve@redhat.com>
Add V4l2DeviceType enum class that reads the device path received in the command line and resolves the device type according to the VFL_TYPE_* kernel constants. Added tests to verify path resolution is correct. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Albert Esteve <aesteve@redhat.com>
Add a null backend that is always compiled in, regardless of which optional backend features are enabled. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Albert Esteve <aesteve@redhat.com>
vhost-device-media depends on v4l2r, which uses bindgen in its build script to generate Rust bindings from /usr/include/linux/videodev2.h. That header includes <sys/time.h>, which on Debian/Ubuntu aarch64 is placed under the multiarch path /usr/include/aarch64-linux-gnu/ rather than directly under /usr/include/. Clang, invoked by bindgen, does not add that directory to its default include search path, causing the build to fail with: /usr/include/linux/videodev2.h:60:10: fatal error: 'sys/time.h' file not found Set BINDGEN_EXTRA_CLANG_ARGS to pass the multiarch include directory to clang for all GNU build and test steps. The path is derived from $(uname -m) so it resolves correctly on x86_64 and aarch64 alike, and is a no-op on any platform where the directory does not exist. Additionally, the ffmpeg feature pulls in virtio-media-ffmpeg-decoder, whose build script uses pkg-config to locate libavcodec. pkg-config refuses to run for a musl cross-compilation target, and libavcodec is not available as a musl-linked library regardless. Exclude vhost-device-media from all musl build and test jobs, following the same pattern already used for vhost-device-gpu. Signed-off-by: Albert Esteve <aesteve@redhat.com>
Contributor
Author
|
Looks much cleaner now. One of the remaining issues (with cargo-audit) is related with Gnurou/v4l2r#60, so it is a matter of time we get rid of it, hopefully. The only other issue is with coverage. I will double check where's the gap and see if I can close it. Thanks @stefano-garzarella and @epilys for the reviews and the patience! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary of the PR
This PR introduces vhost-user-media device.
The virtio-media device first appeared in virtio-spec v1.4 specification[1].
It standardizes how guest applications communicate with media devices on
the host, such as encoders, decoders, and cameras.
The protocol relies on the V4L2 (Video4Linux2) specification to
format the communication between the guest and host, exposing the
host's media devices to the virtualized environment.
The low-level implementation of this backend for the virtio-media
protocol is provided by the device crate from the virtio-media
repository[2].
Implementation is still WiP as it is waiting for vhost release and SHMEM_MAP/UNMAP patch to be merged in Qemu (so that the specs align with current implementation).Done!
[1] https://docs.oasis-open.org/virtio/virtio/v1.4/virtio-v1.4.html#x1-82200022
[2] https://github.com/chromeos/virtio-media
Requirements
Before submitting your PR, please make sure you addressed the following
requirements:
git commit -s), and the commit message has max 60 characters for thesummary and max 75 characters for each description line.
test.
Release" section of CHANGELOG.md (if no such section exists, please create one).
unsafecode is properly documented.