Skip to content

Commit ee677c5

Browse files
culhatskerculhatsker
committed
README, test after build feature
1 parent aea6786 commit ee677c5

File tree

13 files changed

+511
-87
lines changed

13 files changed

+511
-87
lines changed

‎refactor/README.md‎

Lines changed: 310 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,8 +4,317 @@
44

55
The tool does not require any non-standard Python packages, it only needs Python 3.10+ present
66

7+
`configs/releases` directory contains configurations for all the supported releases. In general, if you want to build
8+
`ubuntu20_dev:2024.3.0` image you should run this command:
9+
710
```bash
811
python3 image.py 2024.3.0/ubuntu20 --preset dev --build
912
```
13+
This will generate `Dockerfile` and build it, tagging it `localhost/ubuntu20_dev:2024.3.0`
14+
15+
## Current support state
16+
17+
Os support:
18+
* Ubuntu 20: ✅
19+
* Ubuntu 22: ✅
20+
* Ubuntu 24: ❌
21+
* RHEL8: ❌ (TODO)
22+
23+
OpenVINO releases support:
24+
* before 2024.1.0 ❌
25+
* 2024.1.0 ❌(TODO)
26+
* 2024.2.0 ✅
27+
* 2024.3.0 ✅
28+
29+
Device support:
30+
* x86-64 CPU ✅
31+
* aarch64 CPU ❌
32+
* armhf CPU ❌
33+
* Intel GPU ✅
34+
* Intel NPU ✅
35+
36+
Note that even though `Intel NPU` is said to be supported it doesn't mean that every configuration supports it, for example `Ubuntu 20` doesn't support running on `Intel NPU`, only `Ubuntu 22` (and above when applicable) support it. Similar situation exists for Arm CPUs with Intel hardware.
37+
38+
## How to work with it
39+
40+
### When new OpenVINO release
41+
42+
1) Create a new release directory in `config/releases` called after the release version;
43+
2) For each package build for specific os supported by this project, create a json file and use previous versions ad a template.
44+
45+
### When new OS needs to be supported
46+
47+
1) Create corresponding configs in the affected release directories
48+
2) Create a base config for the new os
49+
50+
Try to use the previous os version configs as a template if applicable.
51+
52+
Note: some package versions are hard-coded or refer to a specific build, make sure those are also up-to-date for the new OS.
53+
54+
## How it works
55+
56+
### Config file structure
57+
58+
Configuration in this project is done with a chain of configs. Product configs are stored in `configs/releases`
59+
directory, they inherit from base configs stored in `configs/base` which can also inherit from other base configs.
60+
Config inheritance is defined with `_based_on` property, the config pointed at by `_based_on` will load first and then
61+
the original config will merge with the base config.
62+
63+
Note: recursion is forbidden, that is, the dependency graph must have no cycles.
64+
65+
TODO: check for recursion, right now it will be infinitely loading if recursion appears.
66+
67+
#### Merging rules
68+
69+
1) If either object is null (or if either is missing / is undefined) then the other is returned
70+
2) If objects have different types then an error is returned
71+
3) If objects are dictionaries then they are merged with this algorithm
72+
4) otherwise the new object is returned instead of the old one (including lists)
73+
74+
TODO: ^^^ describe merging better ^^^
75+
76+
Note: lists are not merged, they replace each other
77+
78+
This table shows a simplified example of how the files could be related to each other and how the merged
79+
configuration would look like.
80+
81+
<style>
82+
table {
83+
border-width: 5px;
84+
border-style: solid
85+
}
86+
87+
td {
88+
border-width: 3px;
89+
border-style: solid
90+
}
91+
</style>
92+
<table>
93+
<tbody>
94+
<tr>
95+
<td>common.json</td>
96+
<td>ubuntu.json</td>
97+
<td>ubuntu22.json</td>
98+
<td>releases/…/ubuntu22.json</td>
99+
<td>result (what will be read)</td>
100+
</tr>
101+
<tr>
102+
<td>{</td>
103+
<td>{</td>
104+
<td>{</td>
105+
<td>{</td>
106+
<td>{</td>
107+
</tr>
108+
<tr>
109+
<td></td>
110+
<td>"_based_on": "common"</td>
111+
<td>"_based_on": "ubuntu"</td>
112+
<td>"_based_on": "ubuntu22"</td>
113+
<td></td>
114+
</tr>
115+
<tr>
116+
<td></td>
117+
<td></td>
118+
<td></td>
119+
<td>"_template": "Dockerfile.j2"</td>
120+
<td>"_template": "Dockerfile.j2"</td>
121+
</tr>
122+
<tr>
123+
<td></td>
124+
<td></td>
125+
<td></td>
126+
<td>"package": {</td>
127+
<td>"package": {</td>
128+
</tr>
129+
<tr>
130+
<td></td>
131+
<td></td>
132+
<td></td>
133+
<td>"url": "https://example.com"</td>
134+
<td>"url": "https://example.com"</td>
135+
</tr>
136+
<tr>
137+
<td></td>
138+
<td></td>
139+
<td></td>
140+
<td>"version": "2024.3.0"</td>
141+
<td>"version": "2024.3.0"</td>
142+
</tr>
143+
<tr>
144+
<td></td>
145+
<td></td>
146+
<td></td>
147+
<td>}</td>
148+
<td>}</td>
149+
</tr>
150+
<tr>
151+
<td></td>
152+
<td></td>
153+
<td></td>
154+
<td>"presets": {</td>
155+
<td>"presets": {</td>
156+
</tr>
157+
<tr>
158+
<td></td>
159+
<td></td>
160+
<td></td>
161+
<td>"runtime": ["preset_runtime", "device_gpu"]</td>
162+
<td>"runtime": ["preset_runtime", "device_gpu"]</td>
163+
</tr>
164+
<tr>
165+
<td></td>
166+
<td></td>
167+
<td></td>
168+
<td>…</td>
169+
<td>…</td>
170+
</tr>
171+
<tr>
172+
<td></td>
173+
<td></td>
174+
<td></td>
175+
<td>}</td>
176+
<td>}</td>
177+
</tr>
178+
<tr>
179+
<td></td>
180+
<td></td>
181+
<td>"base_image": "ubuntu:22.04"</td>
182+
<td></td>
183+
<td>"base_image": "ubuntu:22.04"</td>
184+
</tr>
185+
<tr>
186+
<td>"components": {</td>
187+
<td>"components": {</td>
188+
<td>"components": {</td>
189+
<td></td>
190+
<td>"components": {</td>
191+
</tr>
192+
<tr>
193+
<td></td>
194+
<td></td>
195+
<td>"intel-level-zero-gpu": {</td>
196+
<td></td>
197+
<td>"intel-level-zero-gpu": {</td>
198+
</tr>
199+
<tr>
200+
<td></td>
201+
<td></td>
202+
<td>"requires": ["level-zero"]</td>
203+
<td></td>
204+
<td>"requires": ["level-zero"]</td>
205+
</tr>
206+
<tr>
207+
<td></td>
208+
<td></td>
209+
<td>"apt": ["https://..."]</td>
210+
<td></td>
211+
<td>"apt": ["https://..."]</td>
212+
</tr>
213+
<tr>
214+
<td></td>
215+
<td></td>
216+
<td>}</td>
217+
<td></td>
218+
<td>}</td>
219+
</tr>
220+
<tr>
221+
<td></td>
222+
<td></td>
223+
<td>"level-zero": {</td>
224+
<td></td>
225+
<td>"level-zero": {</td>
226+
</tr>
227+
<tr>
228+
<td></td>
229+
<td></td>
230+
<td>"apt": ["https://..."]</td>
231+
<td></td>
232+
<td>"apt": ["https://..."]</td>
233+
</tr>
234+
<tr>
235+
<td></td>
236+
<td></td>
237+
<td>}</td>
238+
<td></td>
239+
<td>}</td>
240+
</tr>
241+
<tr>
242+
<td></td>
243+
<td>"base": {</td>
244+
<td></td>
245+
<td></td>
246+
<td>"base": {</td>
247+
</tr>
248+
<tr>
249+
<td></td>
250+
<td>"apt": ["curl", …]</td>
251+
<td></td>
252+
<td></td>
253+
<td>"apt": ["curl", …]</td>
254+
</tr>
255+
<tr>
256+
<td></td>
257+
<td>}</td>
258+
<td></td>
259+
<td></td>
260+
<td>}</td>
261+
</tr>
262+
<tr>
263+
<td>"preset_runtime": {</td>
264+
<td></td>
265+
<td></td>
266+
<td></td>
267+
<td>"preset_runtime": {</td>
268+
</tr>
269+
<tr>
270+
<td>"requires": ["base"]</td>
271+
<td></td>
272+
<td></td>
273+
<td></td>
274+
<td>"requires": ["base"]</td>
275+
</tr>
276+
<tr>
277+
<td>}</td>
278+
<td></td>
279+
<td></td>
280+
<td></td>
281+
<td>}</td>
282+
</tr>
283+
<tr>
284+
<td>"device_gpu": {</td>
285+
<td></td>
286+
<td></td>
287+
<td></td>
288+
<td>"device_gpu": {</td>
289+
</tr>
290+
<tr>
291+
<td>"requires": ["intel-level-zero-gpu", ...]</td>
292+
<td></td>
293+
<td></td>
294+
<td></td>
295+
<td>"requires": ["intel-level-zero-gpu", ...]</td>
296+
</tr>
297+
<tr>
298+
<td>}</td>
299+
<td></td>
300+
<td></td>
301+
<td></td>
302+
<td>}</td>
303+
</tr>
304+
<tr>
305+
<td>}</td>
306+
<td>}</td>
307+
<td>}</td>
308+
<td>}</td>
309+
<td>}</td>
310+
</tr>
311+
<tr>
312+
<td>}</td>
313+
<td>}</td>
314+
<td>}</td>
315+
<td>}</td>
316+
<td>}</td>
317+
</tr>
318+
</tbody>
319+
</table>
10320

11-
This will generate `Dockerfile` and build it, tagging it `localhost/ubuntu20_dev:2024.3.0`

‎refactor/configs/base/common.json‎

Lines changed: 18 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,19 @@
11
{
22
"components": {
3+
"base": {
4+
"tests": [
5+
"[email protected]",
6+
"local_library_loadable.sh@libopenvino_c.so"
7+
]
8+
},
9+
"python": {
10+
"tests": [
11+
"check_python_openvino.sh"
12+
]
13+
},
314
"device_cpu": {
415
"tests": [
5-
"check_device@CPU"
16+
"local_library_loadable.sh@libopenvino_intel_cpu_plugin.so"
617
]
718
},
819
"device_gpu": {
@@ -13,7 +24,7 @@
1324
"intel-opencl-icd"
1425
],
1526
"tests": [
16-
"check_device@GPU"
27+
"local_library_loadable.sh@libopenvino_intel_gpu_plugin.so"
1728
]
1829
},
1930
"device_npu": {
@@ -22,15 +33,18 @@
2233
"intel-level-zero-npu"
2334
],
2435
"tests": [
25-
"check_device@NPU"
36+
"local_library_loadable.sh@libopenvino_intel_npu_plugin.so"
2637
]
2738
},
2839
"openvino_dev": {
2940
"requires": [
3041
"python",
3142
"build_tools"
3243
],
33-
"extras": "caffe,kaldi,mxnet,onnx,pytorch,tensorflow,tensorflow2"
44+
"extras": "caffe,kaldi,mxnet,onnx,pytorch,tensorflow,tensorflow2",
45+
"tests": [
46+
"check_omz_tools.sh"
47+
]
3448
},
3549
"preset_runtime": {
3650
"requires": [

0 commit comments

Comments
 (0)