@@ -32,6 +32,7 @@ DLI supports inference using the following frameworks:
32
32
- [ RKNN] [ rknn ] (C++ API).
33
33
- [ ncnn] [ ncnn ] (Python API).
34
34
- [ PaddlePaddle] [ PaddlePaddle ] (Python API).
35
+ - [ ExecuTorch] [ executorch ] (C++ API)
35
36
36
37
More information about DLI is available on the web-site
37
38
([ here] [ dli-ru-web-page ] (in Russian)
@@ -47,8 +48,8 @@ Please consider citing the following papers.
47
48
48
49
1 . Kustikova V., Vasilyev E., Khvatov A., Kumbrasiev P., Rybkin R.,
49
50
Kogteva N. DLI: Deep Learning Inference Benchmark //
50
- Communications in Computer and Information Science.
51
- V. 1129. 2019. P. 542-553.
51
+ Communications in Computer and Information
52
+ Science. - 2019. - V. 1129. - P. 542-553.
52
53
53
54
1 . Sidorova A.K., Alibekov M.R., Makarov A.A., Vasiliev E.P.,
54
55
Kustikova V.D. Automation of collecting performance indicators
@@ -73,10 +74,11 @@ Please consider citing the following papers.
73
74
and Programming. - 2024. - Vol. 25(2). - P. 127-141. -
74
75
[ https://num-meth.ru/index.php/journal/article/view/1332/1264 ] [ nummeth2023 ] .
75
76
(In Russian)
77
+
76
78
1 . Mukhin I., Rodimkov Y., Vasiliev E., Volokitin V., Sidorova A.,
77
79
Kozinov E., Meyerov I., Kustikova V. Benchmarking Deep Learning
78
80
Inference on RISC-V CPUs // Springer Lecture Notes in Computer
79
- Science. – 2024 . – Accepted .
81
+ Science. – 2025 . – Vol. 15406. - P. 331-346 .
80
82
81
83
## Repo structure
82
84
@@ -97,12 +99,12 @@ Please consider citing the following papers.
97
99
- ` OpenCV ` is a directory of Dockerfiles for OpenCV.
98
100
- ` OpenVINO_DLDT ` is a directory of Dockerfiles for Intel®
99
101
Distribution of OpenVINO™ Toolkit.
102
+ - ` PaddlePaddle ` is a directory of Dockerfiles for PaddlePaddle.
100
103
- ` PyTorch ` is a directory of Dockerfiles for PyTorch.
101
- - ` TVM ` is a directory of Dockerfiles for Apache TVM.
102
104
- ` TensorFlow ` is a directory of Dockerfiles for Intel® Optimizations
103
105
for TensorFlow.
104
106
- ` TensorFlowLite ` is a directory of Dockerfiles for TensorFlow Lite.
105
- - ` PaddlePaddle ` is a directory of Dockerfiles for PaddlePaddle .
107
+ - ` TVM ` is a directory of Dockerfiles for Apache TVM .
106
108
107
109
- ` docs ` directory contains auxiliary documentation. Please, find
108
110
complete documentation at the [ Wiki page] [ dli-wiki ] .
@@ -138,6 +140,9 @@ Please consider citing the following papers.
138
140
is a table that confirms correctness of inference implementation
139
141
based on Intel® Distribution of OpenVINO™ toolkit for models trained
140
142
by Intel engineers and available in [ Open Model Zoo] [ open-model-zoo ] .
143
+ - [ ` validation_results_paddlepaddle.md ` ] ( results/validation/validation_results_paddlepaddle.md )
144
+ is a table that confirms correctness of inference implementation
145
+ based on PaddlePaddle.
141
146
- [ ` validation_results_pytorch.md ` ] ( results/validation/validation_results_pytorch.md )
142
147
is a table that confirms correctness of inference implementation
143
148
based on PyTorch for [ TorchVision] [ torchvision ] .
@@ -161,9 +166,11 @@ Please consider citing the following papers.
161
166
- [ ` onnxruntime_models_checklist.md ` ] ( results/onnxruntime_models_checklist.md ) contains a list
162
167
of deep models inferred by ONNX Runtime checked in the DLI benchmark.
163
168
- [ ` opencv_models_checklist.md ` ] ( results/opencv_models_checklist.md ) contains a list
164
- of deep models inferred by OpenCV DNN.
169
+ of deep models inferred by OpenCV DNN checked in the DLI benchmark .
165
170
- [ ` openvino_models_checklist.md ` ] ( results/openvino_models_checklist.md ) contains a list
166
171
of deep models inferred by the OpenVINO toolkit checked in the DLI benchmark.
172
+ - [ ` paddlepaddle_models_checklist.md ` ] ( results/paddlepaddle_models_checklist.md )
173
+ containsa list deep models inferred by PaddlePaddle checked in the DLI benchmark.
167
174
- [ ` pytorch_models_checklist.md ` ] ( results/pytorch_models_checklist.md ) contains a list
168
175
of deep models inferred by PyTorch checked in the DLI benchmark.
169
176
- [ ` tensorflow_models_checklist.md ` ] ( results/tensorflow_models_checklist.md ) contains a list
@@ -179,14 +186,17 @@ Please consider citing the following papers.
179
186
using Accuracy Checker of Intel® Distribution of OpenVINO™ toolkit.
180
187
- ` benchmark ` is a set of scripts to estimate inference
181
188
performance of different models at the single local computer.
182
- - ` build_scripts ` is a directory to build inference frameworks for different platforms.
189
+ - ` build_scripts ` is a directory to build inference frameworks for different
190
+ platforms.
183
191
- ` config_maker ` contains GUI-application to make configuration files
184
- of the benchmark components.
192
+ of the benchmark components. Application supports outdated version
193
+ of configuration files. It is required to update (one of the future tasks).
185
194
- ` configs ` contains template configuration files.
186
195
- ` cpp_dl_benchmark ` contains C++ tools that allow to measure
187
196
deep learning models inference performance with
188
197
[ ONNX Runtime] [ onnx-runtime-github ] , [ OpenCV DNN] [ opencv-dnn ] ,
189
- [ PyTorch] [ pytorch ] and [ TensorFlow Lite] [ tensorflow-lite ] in C++ API implementation.
198
+ [ PyTorch] [ pytorch ] , [ ExecuTorch] [ executorch ] and
199
+ [ TensorFlow Lite] [ tensorflow-lite ] in C++ API implementation.
190
200
This implementation inspired by [ OpenVINO Benchmark C++ tool] [ benchmark-app ]
191
201
as a reference and stick to its measurement methodology,
192
202
thus provide consistent performance results.
@@ -199,15 +209,15 @@ Please consider citing the following papers.
199
209
- ` model_converters ` contains converters of deep models.
200
210
- ` node_info ` contains a set of functions to get information about
201
211
computational node.
202
- - ` quantization ` contains scripts to quantize model to INT8-precision
203
- using Post-Training Optimization Tool (POT)
204
- of Intel® Distribution of OpenVINO™ toolkit.
212
+ - ` quantization ` contains scripts to quantize model
213
+ using Post-Training Optimization Tool (POT) of Intel® Distribution
214
+ of OpenVINO™ toolkit, TensorFlow Lite and TVM internal tools .
205
215
- ` remote_control ` contains scripts to execute benchmark
206
216
remotely.
207
217
- ` tvm_autotuning ` contains scripts to optimize Apache TVM models.
208
218
- ` utils ` is a package of auxiliary utilities.
209
219
210
- - ` test ` contains smoke tests.
220
+ - ` tests ` contains smoke tests.
211
221
212
222
- ` requirements.txt ` is a list of special requirements for the DLI
213
223
benchmark without inference frameworks.
@@ -271,6 +281,7 @@ Report questions, issues and suggestions, using:
271
281
[ rknn ] : https://github.com/rockchip-linux/rknn-toolkit2
272
282
[ ncnn ] : https://github.com/Tencent/ncnn
273
283
[ PaddlePaddle ] : https://www.paddlepaddle.org.cn/en
284
+ [ executorch ] : https://pytorch.org/executorch-overview
274
285
[ benchmark-app ] : https://github.com/openvinotoolkit/openvino/tree/master/samples/cpp/benchmark_app
275
286
[ dli-ru-web-page ] : http://hpc-education.unn.ru/dli-ru
276
287
[ dli-web-page ] : http://hpc-education.unn.ru/dli
0 commit comments