OpenVINO2021r4.1-瞎搞YOLOV5模型转换，INT8量化及 C++推理实现
1. 这个⼤神在openvino 2021.2上⼿写代码扩充了accuracy checker的yolov5的⽀持，需要在ov2021.2上打patch, 如果openvino
升级的话，因为AC代码的变动, merge的时候⼤概率会有问题 2. 记得当时转换完测了⼀下，CPU上推理是正常的，但是GPU推理有问题。在那个bus的推理图⽚上，莫名的多出了⼏个识别的红框
框，明显是错了。感觉clDNN在计算上可能有数据溢出的现象，但是时间有点久了，忘了是在FP32还是在INT8时候出错了。
最近OpenVINO 2021r4.1发布了，再回过头来看看2021r4.1版本的yolov5⽀持
1. 搭建环境
安装windows版的openvino 2021.4.1 LTS, 按照官⽹的教程安装ac和pot的依赖项。
2. 下载和准备yolov5的模型
下载github上分享的版本的pytorch模型
运⾏export.py将pytorch模型转成onnx模型
$ python3 models/export.py --weights yolov5l.pt --img-size 640
然后⽤netron观察转换出的onnx⽹络节点图，记录最后3个输出点前的最后⼀个Conv层的名字，等下转IR时候需要指定--output输出节点，这个模型分别是Conv_403， Conv_491， Conv_435
3. 转换yolov5的OpenVINO FP32模型
进⼊openvino环境，将onnx模型转成FP32的IR模型
C:\temp\yolov5_ac_ov2021_4>python "c:\Program Files (x86)\intel\openvino_2021\deployment_tools\model_optimizer\mo.py" --input_model Model Optimizer arguments:
Common parameters:
- Path to the Input Model: C:\temp\yolov5_ac_ov2021_4\
- Path for generated IR: C:\temp\yolov5_ac_ov2021_4\.
- IR output name: yolov5l_v4
- Log level: ERROR
- Batch: Not specified, inherited from the model
- Input layers: Not specified, inherited from the model
- Output layers: Conv_403,Conv_419,Conv_435
- Input shapes: Not specified, inherited from the model
- Mean values: Not specified
- Scale values: Not specified
- Scale factor: 255.0
- Precision of IR: FP32
- Enable fusing: True
- Enable grouped convolutions fusing: True
- Move mean values to preprocess section: None
- Reverse input channels: True
ONNX specific parameters:
-
Inference Engine found in: c:\Program Files (x86)\Intel\openvino_2021\python\python3.7\openvino
Inference Engine version: 2021.4.1-3926-14e67d86634-releases/2021/4
Model Optimizer version: 2021.4.1-3926-14e67d86634-releases/2021/4
[ SUCCESS ] Generated IR version 10 model.
[ SUCCESS ] XML file: C:\temp\yolov5_ac_ov2021_4\l
[ SUCCESS ] BIN file: C:\temp\yolov5_ac_ov2021_4\yolov5l_v4.bin
[ SUCCESS ] Total execution time: 14.90 seconds.
接下来先做⼀下FP32的精度检测，好等下和INT8模型精度做⽐对 1. 下载和准备coco val2017数据集，
2. 下载的原始数据集有点⼤，⽤CPU做完整的AC太费时间，可以参考这⾥对数据库做裁剪 这⾥只截了256张图⽚做实验
accuracy checker
准备yolov5_l配置⽂件
models:
- name: yolo-v5
launchers:
- framework: dlsdk
model: l
weights: yolov5l_v4.bin
adapter:
type: yolo_v3
anchors: "10,13, 16,30, 33,23, 30,61, 62,45, 59,119, 116,90, 156,198, 373,326" num: 9
coords: 4
classes: 80
anchor_masks: [[6, 7, 8], [3, 4, 5], [0, 1, 2], ]
outputs:
- Conv_435
- Conv_419
- Conv_403
datasets:
- name: ms_coco_detection_80_class_without_background
data_source: val2017
annotation_conversion:
converter: mscoco_detection
annotation_file: instances_val2017.json
has_background: False
sort_annotations: True
use_full_label_map: False
annotation: mscoco_det_80.pickle
dataset_meta: mscoco_det_80.json
preprocessing:
- type: resize
size: 640
postprocessing:
- type: resize_prediction_boxes
-
type: filter
apply_to: prediction
min_confidence: 0.001
remove_filtered: True
- type: nms
overlap: 0.5
- type: clip_boxes
apply_to: prediction
metrics:
- type: map
integral: 11point
ignore_difficult: true
presenter: print_scalar
- type: coco_precision
max_detections: 100
threshold: 0.5
运⾏ac命令
C:\temp\yolov5_ac_ov2021_4>accuracy_check -c yolov5_l -s ./ -td CPU
09:01:27 accuracy_checker WARNING: c:\Program Files (x86)\Intel\openvino_2021\python\python3.7\ngraph\utils\types.py:25: DeprecationWarning: `np.bool` is Deprecated in NumPy 1.20; for more details and guidance: /devdocs/release/1.20.0-notes.html#deprecations
(NgraphType.boolean, np.bool),
Processing info:
model: yolo-v5
launcher: dlsdk
device: CPU
dataset: ms_coco_detection_80_class_without_background
OpenCV version: 4.5.2
Annotation for ms_coco_detection_80_class_without_background dataset will be loaded from mscoco_det_80.pickle
Loaded dataset info:
Dataset name: ms_coco_detection_80_class_without_background_1
Accuracy Checker version 0.8.7
Dataset size 256
Conversion parameters:
converter: mscoco_detection
annotation_file: PATH/instances_val2017.json
has_background: False
sort_annotations: True
use_full_label_map: False
ms_coco_detection_80_class_without_background dataset metadata will be loaded from mscoco_det_80.json
IE version: 2021.4.1-3926-14e67d86634-releases/2021/4
Loaded CPU plugin version:
CPU - MKLDNNPlugin: 2.1.2021.4.1-3926-14e67d86634-releases/2021/4
Found model l
Found weights yolov5l_v4.bin
Input info:
Layer name: images
precision: FP32
shape [1, 3, 640, 640]
Output info
Layer name: Conv_403
precision: FP32
shape: [1, 255, 80, 80]
Layer name: Conv_419
precision: FP32
shape: [1, 255, 40, 40]
Layer name: Conv_435
precision: FP32
shape: [1, 255, 20, 20]
09:01:28 accuracy_checker WARNING: c:\users\intel\anaconda3\lib\site-packages\accuracy_checker-0.8.\accuracy_checker\metrics\metric_executor warnings.warn('threshold option is deprecated. Please use abs_threshold instead', DeprecationWarning)
256 objects processed in 133.234 seconds
map: 27.70%
coco_precision: 31.18%
这个精度有问题啊，⽤的代码复现的精度有60%多呢... 但是看⽹上很多⼈已经⽤openvino+yolov5做推理了，精度应该没有⼤问题，还是accuracy checker⾃⼰有问题的可能性⽐较⼤
4. 转换yolov5的OpenVINO INT8模型
⽤pot⼯具做int8模型量化
准备pot配置json⽂件, 量化⽅法选择了最简单的DefaultQuantization, 这种⽅法在量化时不做精度检测对⽐，只是把所有能转成int8的
fp32操作全部转成int8类型
准备yolov5l_v4_int8_simple_mode.json配置⽂件
{
"model": {
"model_name": "yolov5l_v4_int8_cpu",
"model": "yolov5l_l",
"weights": "yolov5l_v4_640.bin"
},
"engine": {
"type": "simplified",
// you can specify path to directory with images or video file
// also you can specify template for file names to filter images to load
// templates are unix style
"data_source": "val2017"
},
"compression": {
"target_device": "CPU",
"algorithms": [
{
"name": "DefaultQuantization",
"params": {
"preset": "performance",
"stat_subset_size": 128
}
}
]
}
}
运⾏pot命令量化
C:\temp\yolov5_ac_ov2021_4>pot -c yolov5l_v4_int8_simple_mode.json
10:12:41 accuracy_checker WARNING: c:\users\intel\appdata\roaming\python\python37\site-packages\defusedxml\__init__.py:30: DeprecationWarning: defusedx from . import cElementTreemsinfo
10:12:41 accuracy_checker WARNING: c:\users\intel\anaconda3\lib\site-packages\pot-1.\compression\algorithms\quantization\optimization\algorithm. 'Nevergrad package could not be imported. If you are planning to use'
10:12:41 accuracy_checker WARNING: c:\users\intel\anaconda3\lib\site-packages\past\builtins\misc.py:4: DeprecationWarning: Using or importing the ABCs from from collections import Mapping
INFO:app.run:Output log dir: ./results\yolov5l_v4_int8_gpu_DefaultQuantization\2021-09-16_10-12-41
INFO:app.run:Creating pipeline:
Algorithm: DefaultQuantization
Parameters:
preset : performance
stat_subset_size : 128
target_device : CPU
model_type : None
dump_intermediate_model : False
exec_log_dir : ./results\yolov5l_v4_int8_gpu_DefaultQuantization\2021-09-16_10-12-41
===========================================================================
INFO:llector:Start computing statistics for algorithms : DefaultQuantization
INFO:llector:Computing statistics finished
INFO:compression.pipeline.pipeline:Start algorithm: DefaultQuantization
INFO:compression.algorithms.quantization.default.algorithm:Start computing statistics for algorithm : ActivationChannelAlignment
INFO:compression.algorithms.quantization.default.algorithm:Computing statistics finished
INFO:compression.algorithms.quantization.default.algorithm:Start computing statistics for algorithms : MinMaxQuantization,FastBiasCorrection
INFO:compression.algorithms.quantization.default.algorithm:Computing statistics finished
INFO:compression.pipeline.pipeline:Finished: DefaultQuantization
===========================================================================
豫公网安备41160202000603
站长QQ:729038198
关于我们
投诉建议