mirror of https://github.com/azaion/autopilot.git synced 2026-04-22 22:56:33 +00:00

Files

T

Tuomas Järvinen 022e4a1200 Add compile option to use YOLOv8 ONNX models for testing.

- added qmake option yolo_onnx to use normal YOLOv8 ONNX models. This makes possible to test
  gimbals camera inside without real model.
- reduced confidence threshold requirement in AiEngineInferencevOnnxRuntime from 0.5 to 0.2
- make printing prettier with ONNX Runtime
- removed unnecessary cv::Mat::clone()

Type: Improvement
Issue: https://denyspopov.atlassian.net/browse/AZ-39

2024-08-18 17:17:40 +03:00

2.6 KiB

Raw Blame History

rtsp_ai_player

rtsp_ai_player is an application that listens to an RTSP stream, analyzes images with an AI model, and shows the results visually. It also controls a gimbal camera to zoom in on the recognized objects. Application uses YOLOv8 AI models converted to the ONNX format.

How to convert Azaion AI model file to ONNX format

yolo export model=azaion-2024-08-13.pt dynamic=False format=onnx imgsz=640,640

How to use application locally on a Linux PC.

Speed up compilations

echo "export MAKEFLAGS=\"-j8\"" >> ~/.bashrc
echo "export PATH=/usr/lib/ccache:\$PATH" >> ~/.bashrc

Install OpenCV 4.10.0

sudo apt update
sudo apt install libgtk-3-dev libpng-dev cmake ffmpeg libavcodec-dev libavformat-dev libavfilter-dev
wget https://github.com/opencv/opencv/archive/refs/tags/4.10.0.zip
unzip 4.10.0.zip
cd opencv-4.10.0
mkdir build && cd build
cmake -DCMAKE_INSTALL_PREFIX=/opt/opencv-4.10.0 -DBUILD_opencv_world=ON -DOPENCV_GENERATE_PKGCONFIG=ON -DBUILD_PERF_TESTS=OFF -DBUILD_TESTS=OFF ..
make -j8 && sudo make install

Install ONNX runtime 1.18.0

wget https://github.com/microsoft/onnxruntime/releases/download/v1.18.0/onnxruntime-linux-x64-1.18.0.tgz
sudo tar xf onnxruntime-linux-x64-1.18.0.tgz -C /opt

Install ffmpeg and mediamtx RTSP server:

sudo apt update
sudo apt install ffmpeg
wget https://github.com/bluenviron/mediamtx/releases/download/v1.8.4/mediamtx_v1.8.4_linux_amd64.tar.gz
mkdir mediamtx
tar xf mediamtx_v1.8.4_linux_amd64.tar.gz -C mediamtx

If you use video file from the local RTSP server:

cd mediamtx
./mediamtx

Play Azaion mp4 video file from RTSP server ... :

ffmpeg -re -stream_loop -1 -i $HOME/azaion/videos/for_ai_cut.mp4 -c copy -f rtsp rtsp://localhost:8554/live.stream

... or play simple video file from RTSP server:

ffmpeg -re -stream_loop -1 -i $HOME/azaion/videos/table.mp4 -c copy -f rtsp rtsp://localhost:8554/live.stream

Test RTSP streaming with ffplay:

ffplay -rtsp_transport tcp rtsp://localhost:8554/live.stream

Compile and run rtsp_ai_player with YOLOv8 medium AI model:

cd autopilot/misc/rtsp_ai_player
qmake6 && make
./rtsp_ai_player ~/azaion/models/onnx/yolov8m.onnx

Compile and run rtsp_ai_player with Azaion AI model:

cd autopilot/misc/rtsp_ai_player
qmake6 && make
./rtsp_ai_player ~/azaion/models/azaion/azaion-2024-08-13.onnx

Compile and run rtsp_ai_player with YOLOv8 medium model and gimbal camera:

cd autopilot/misc/rtsp_ai_player
qmake6 CONFIG+=gimbal && make
./rtsp_ai_player ~/azaion/models/onnx/yolov8m.onnx

2.6 KiB Raw Blame History