YOLO-NAS-OpenVINO.cpp

C++ OpenVINO OpenCV CMake Object Detection

C++ implementation of YOLO-NAS with an OpenVINO backend.

Source

GitHub release (latest by date) Visual Studio CMake Licence

YOLO-NAS-OpenVino.cpp Link to this heading

YOLO-NAS is a state-of-the-art object detector by Deci AI. This project implements the YOLO-NAS object detector in C++ with an OpenVINO backend to speed up inference performance.

Features Link to this heading

  • Supports both image and video inference.
  • Faster inference speeds.

Getting Started Link to this heading

The following instructions demonstrates how to build this project on a Windows system and Linux systems supported by OpenVINO.

Prerequisites Link to this heading

  • CMake v3.8+ - found at https://cmake.org/

  • MSVC 2017++ (Windows Build) - MinGW will not work on Windows Build as OpenVINO libraries are not compatible with MinGW.

  • GNU (Linux Build) - tested on v11.4.0.

  • OpenVINO Toolkit - tested on 2022.1. Download here.

  • OpenCV v4.0+ - tested on v4.7. Download here.

Building the project Link to this heading

  1. Set the OpenCV_DIR environment variable to point to your ../../opencv/build directory.
  2. Set the OpenVINO_DIR environment variable to point to your ../../openvino/runtime/cmake directory.
  3. Run the following build commands:

a. [Windows] VS Developer Command Prompt:

cd \d <yolo-nas-openvino-cpp-directory>
cmake -S. -Bbuild -DCMAKE_BUILD_TYPE=Release
cd build

MSBuild yolo-nas-openvino-cpp.sln /property:Configuration=Release

b. [Linux] Bash:

cd <yolo-nas-openvino-cpp-directory>
cmake -S. -Bbuild -DCMAKE_BUILD_TYPE=Release
cd build

make
  1. The compiled .exe will be inside the Release folder for Windows build, while the executable will be in root folder for Linux build.

Inference Link to this heading

  1. Export the ONNX file:
from super_gradients.training import models

model = models.get("yolo_nas_s", pretrained_weights="coco")
model.eval()
model.prep_model_for_conversion(input_size=(1, 3, 640, 640))
models.convert_to_onnx(model=model, prep_model_for_conversion_kwargs={"input_size":(1, 3, 640, 640)}, out_path="yolo_nas_s.onnx")
  1. Convert the ONNX model to OpenVINO IR:
mo --input_model yolo_nas_s.onnx -s 255 --reverse_input_channels
  1. To run the inference, execute the following command:
yolo-nas-openvino-cpp --model <OPENVINO_IR_XML_PATH> [-i <IMAGE_PATH> | -v <VIDEO_PATH>] [--imgsz IMAGE_SIZE] [--gpu] [--iou-thresh IOU_THRESHOLD] [--score-thresh CONFIDENCE_THRESHOLD]

Benchmarks Link to this heading

The following benchmarks were done on Google Colab using Intel® Xeon® Processor E5-2699 v4 @ 2.20GHz with 2 vCPUs.

Backend Latency FPS Implementation
PyTorch 867.02ms 1.15 Native (model.predict() in super_gradients)
ONNX C++ (via OpenCV DNN) 962.27ms 1.04 Hyuotu
ONNX Python 626.37ms 1.59 Hyuotu
OpenVINO C++ 628.04ms 1.59 Y-T-G

Authors Link to this heading

Acknowledgements Link to this heading

Thanks to @Hyuto for his work on ONNX implementation of YOLO-NAS in C++ which was utilized in this project.

License Link to this heading

This project is licensed under the MIT License - see the LICENSE file for details.