Skip to content
This repository was archived by the owner on Jan 3, 2023. It is now read-only.

FFmpeg video analytics release v0.3

Pre-release
Pre-release

Choose a tag to compare

@linxie47 linxie47 released this 24 Oct 11:42
· 7 commits to ffmpeg4.1_va since this release

This release contains FFmpeg* Video Analytics plugins that bring Deep Learning Inference capabilities to open-source framework FFmpeg* and helps developers to build highly efficient and scalable video analytics applications.

New in This Release

In this release, it enables FFmpeg* analytics pipeline with the elementary inference features, including:

  • Support OpenVINO 2019 R2/R3 release.
  • Support Intel® VCAC-A (Harker Height) accelerator card, which is one standard PCIe form factor
    accelerator card. It is equipped with Intel® Core i3 CPU and Movidius Myriad* X VPU.
  • Enable the “throughput” performance mode, executing multiple infer requests asynchronously to
    improve the performance, on both CPU (for example, on XEON® E5) and VCAC-A (with 12 Myraid* X VPUs).
  • Support cropping feature in “detect” filter, so user could specify part of the image for inference.
  • Support HETERO plugin.
  • Support Multi-Device plugin to run inference on multiple devices for higher throughput.
  • New “metaconvert” filter to convert the inference results in AVFrame’s sidedata to pre-defined metadata
    format.
  • The "detect" filter supports YOLO*v3 model for object detection.
  • Support specifying configurations for different inference devices.
  • Support asynchronous pre-processing in the inference filters.
  • Support model reshape API (which is introduced in OpenVINO 2019 R2)

Known Issues/limitations

This release is subject to the following limitations:

  • Running the pipeline with Gen-accelerated hardware decoding and inference on Gen(GPU), there
    might be segfault issue happen (This issue doesn’t exist on OpenVINO 2019 R1).
  • Some models from Open Model Zoo (example, vehicle-license-plate-detection-barrier-0106.xml,
    license-plate-recognition-barrier-0001.xml) doesn’t support batch-size setting. This could be
    workarounded by setting batch-size as 1.
  • When GPU-accelerated hardware decoding is enabled in the ffmpeg command line, there might be
    issue reported that hardware surface is not available. This could be workarounded by
    • setting appropriate “-extra_hw_frames” numbers and “nireq” numbers for each inference filter.
    • setting “-threads 1” to disable multiple-threading for decoding.
  • OpenVINO doesn't support C API up to 2019 R2 release, so we provided the C-API wrapper for the
    inference engine. This C API wrapper will be replaced when OpenVINO support IE C API
    officially.
  • The inference output supports limited number of pre-defined metadata format for use cases, including
    objects detection, emotion, age, gender, license plate, etc.

Release details are available in the attached release notes. Getting started is available on the Wiki or in the attached user guide.