Skip to content

Commit ca68f5c

Browse files
matinlotfalimatinlotfali
authoredFeb 26, 2024··
Merge pull request #95 from MistySOM/develop
Merge from develop to master
2 parents 49a11fe + 6297f0f commit ca68f5c

File tree

2 files changed

+49
-29
lines changed

2 files changed

+49
-29
lines changed
 

‎content/GStreamerDRPAI.md

+45-29
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ you can look for both `/dev/drpai0` and `/dev/udmabuf0` devices on your Linux sh
66
The Userland Direct Memory Access (UDMA) kernel module is required to provide the trained AI model and
77
the input image to the DRPAI hardware. After activating the hardware, it will use the trained model to
88
generate the output which can be read by the UDMA module. While DRPAI is running, the running thread will
9-
go to sleep. Of course, sleep time varies based on the size of the AI model.
9+
go to sleep. Of course, the sleep time varies based on the size of the AI model.
1010

1111
MistyWest team has prepared a [GStreamer plugin for DRPAI](https://github.com/MistySOM/gstreamer1.0-drpai) which can receive any kind of video input,
1212
such as a file (filesrc), a network stream (udpsrc), or a camera device (v4l2src) and outputs a video
@@ -27,27 +27,42 @@ video/x-raw
2727

2828
The plugin also provides you with the following parameters:
2929

30-
| Name | Type | Default | Description |
31-
|-----------------------|---------------------|--------:|----------------------------------------------------------------------|
32-
| **model** | String | --- | The name of the pre-trained model and the directory prefix. |
33-
| **multithread** | Boolean | true | Use a separate thread for object detection. |
34-
| **tracking** | Boolean | true | Track detected objects based on their previous locations. |
35-
| **log-detects** | Boolean | false | Print detected objects in standard output. |
36-
| **show-fps** | Boolean | false | Render frame rates of video and DRPAI at the corner of the video. |
37-
| **stop-error** | Boolean | true | Stop the gstreamer if kernel modules fail to open. |
38-
| **max-video-rate** | Float [0.001 - 120] | 120 | Force maximum video frame rate using thread sleeps. |
39-
| **max-drpai-rate** | Float [0 - 120] | 120 | Force maximum DRPAI frame rate using thread sleeps. |
40-
| **smooth-video-rate** | Float [1 - 1000] | 1 | Number of last video frame rates to average for a more smooth value. |
41-
| **smooth-drpai-rate** | Float [1 - 1000] | 1 | Number of last DRPAI frame rates to average for a more smooth value. |
42-
| **smooth-bbox-rate** | Float [1 - 1000] | 1 | Number of last bounding-box updates to average. (requires tracking) |
43-
| **track-seconds** | Float [0.001 - 100] | 2 | Number of seconds to wait for a tracked undetected object to forget it. |
44-
| **track-iou-thresh** | Float [0 - 1] | 0.25 | The threshold of IOU for tracking bounding-boxes. (1=exact overlap, 0.1=loose overlap) |
45-
| **filter-class** | String | - | A comma-separated list of classes to filter the detection. |
46-
| **filter-left** | Integer [0 - 639] | 0 | The left edge of the region of interest to filter the detection. |
47-
| **filter-top** | Integer [0 - 479] | 0 | The top edge of the region of interest to filter the detection. |
48-
| **filter-width** | Integer [1 - 640] | 640 | The left edge of the region of interest to filter the detection. |
49-
| **filter-height** | Integer [1 - 480] | 480 | The left edge of the region of interest to filter the detection. |
50-
30+
### General Parameters
31+
32+
| Name | Type | Default | Description |
33+
|--------------------------|----------------------|--------:|-----------------------------------------------------------------------------------------------------------|
34+
| **model** | String | --- | (Required) The name of the pre-trained model and the directory prefix. |
35+
| **multithread** | Boolean | true | Use a separate thread for object detection. |
36+
| **log-detects** | Boolean | false | Print detected objects in standard output. |
37+
| **log-server** | Host:Port (String) | --- | Address of logs to send in UDP messages in [JSON format](JSON.md) to the specified port on a remote host. |
38+
| **show-fps** | Boolean | false | Render frame rates of video and DRPAI at the corner of the video. |
39+
| **stop-error** | Boolean | true | Stop the gstreamer if kernel modules fail to open. |
40+
| **max-video-rate** | Float [0.001 - 120] | 120 | Force maximum video frame rate using thread sleeps. |
41+
| **max-drpai-rate** | Float [0 - 120] | 120 | Force maximum DRPAI frame rate using thread sleeps. |
42+
| **smooth-video-rate** | Float [1 - 1000] | 1 | Number of last video frame rates to average for a more smooth value. |
43+
| **smooth-drpai-rate** | Float [1 - 1000] | 1 | Number of last DRPAI frame rates to average for a more smooth value. |
44+
| **smooth-bbox-rate** | Float [1 - 1000] | 1 | Number of last bounding-box updates to average. (requires tracking) |
45+
46+
### Tracking Parameters (YOLO specific)
47+
48+
| Name | Type | Default | Description |
49+
|--------------------------|----------------------|--------:|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
50+
| **tracking** | Boolean | true | Track detected objects based on their previous locations.<br>Each detected object gets an ID that persists across multiple detections<br>based on other tracking properties. |
51+
| **show-track-id** | Boolean | false | Show the track ID on the detection labels. |
52+
| **track-seconds** | Float [0.001 - 100] | 2 | Number of seconds to wait for a tracked undetected object to forget it. |
53+
| **track-doa-thresh** | Float [0.001 - 1000] | 2.25 | The threshold of Distance Over Areas (DOA) for tracking bounding-boxes. |
54+
| **track-history-length** | Integer [0 - 1440] | 60 | Minutes to keep the tracking history. |
55+
56+
### Filtering Parameters (YOLO specific)
57+
58+
| Name | Type | Default | Description |
59+
|--------------------------|-------------------|---------:|-----------------------------------------------------------------------------------|
60+
| **filter-show** | Boolean | false | Show a yellow box where the filter is applied. |
61+
| **filter-class** | String | --- | A comma-separated list of classes to filter the detection.<br>Shows all if empty. |
62+
| **filter-left** | Integer [0 - 639] | 0 | The left edge of the region of interest to filter the detection. |
63+
| **filter-top** | Integer [0 - 479] | 0 | The top edge of the region of interest to filter the detection. |
64+
| **filter-width** | Integer [1 - 640] | 640 | The left edge of the region of interest to filter the detection. |
65+
| **filter-height** | Integer [1 - 480] | 480 | The left edge of the region of interest to filter the detection. |
5166

5267
## AI Model
5368

@@ -61,27 +76,28 @@ MistyWest has already placed compatible `yolov2`, `yolov3`, `tinyyolov2`, and `t
6176
for you to download from [here](https://remote.mistywest.com/download/mh11/models.zip) and extract
6277
into the desired location. For details on how the GStreamer plugin uses them, refer to the next sections.
6378

64-
### Post Processor Dynamic Library
79+
### Model Dynamic Library
6580

6681
Depending on the model you use, even though their input layers are the same, their output layers can be
6782
very different and require additional post-processing to interpret the array of floating point numbers
6883
to a data structure that is used to render the bounding boxes for each inferred object. Therefore,
6984
the plugin uses a shared library that needs to be included with the model and its path is mentioned in
7085
the `{model}/{model}_process_params.txt` file like this:
86+
7187
```
7288
[dynamic_library]
73-
libpostprocess-yolo.so
89+
libgstdrpai-yolo.so
7490
.
7591
.
7692
.
7793
```
7894

79-
#### Yolo Post-Processor Library (libpostprocess-yolo.so)
95+
#### Yolo Dynamic Model Library (libgstdrpai-yolo.so)
8096

81-
The plugin already includes a post-processor library that supports `yolov2`, `yolov3`, `tinyyolov2`,
82-
and `tinyyolov3` models. This post-processor library leverages many similarities between these models and
83-
switches its behaviour based on other parameters that are mentioned in `{model}/{model}_process_params.txt`
84-
file such as the `[best_class_prediction_algorithm]` and `[anchor_divide_size]`.
97+
The plugin already includes a dynamic library that supports `yolov2`, `yolov3`, `tinyyolov2`,
98+
and `tinyyolov3` models. This dynamic library leverages many similarities between these models and
99+
switches its behaviour based on other parameters that are mentioned in `{model}/{model}_process_params.txt`
100+
file such as the `[best_class_prediction_algorithm]` and `[anchor_divide_size]`.
85101

86102
The library also loads the list of all class labels in `{model}/{model}_labels.txt` and the list of all
87103
box anchors in `{model}/{model}_anchors.txt`. This means these 3 files need to be manually included

‎content/GettingStarted.md

+4
Original file line numberDiff line numberDiff line change
@@ -128,6 +128,10 @@ or
128128
```
129129
$ git clone git@github.com:MistySOM/rzv2l.git
130130
```
131+
make sure you have the latest submodules:
132+
```bash
133+
$ git submodule update --init --recursive
134+
```
131135
then enter the `Build/` directory.
132136
```
133137
$ cd Build/

0 commit comments

Comments
 (0)
Please sign in to comment.