mirror of
https://github.com/azaion/autopilot.git
synced 2026-04-23 01:26:34 +00:00
Alex B.
130 lines
3.8 KiB
Markdown
130 lines
3.8 KiB
Markdown
# rtsp_ai_player
|
|
|
|
`rtsp_ai_player` is an application that listens to an RTSP stream, analyzes images with an AI model, and shows the results visually. It also controls a gimbal camera to zoom in on the recognized objects. Application uses YOLOv8 AI models converted to the ONNX format.
|
|
|
|
### How to convert Azaion AI model file to ONNX format
|
|
```bash
|
|
yolo export model=azaion-2024-08-13.pt dynamic=False format=onnx imgsz=640,640
|
|
```
|
|
|
|
## How to use application locally on a Linux PC.
|
|
|
|
# Speed up compilations
|
|
```bash
|
|
echo "export MAKEFLAGS=\"-j8\"" >> ~/.bashrc
|
|
echo "export PATH=/usr/lib/ccache:\$PATH" >> ~/.bashrc
|
|
```
|
|
|
|
### Install OpenCV 4.10.0
|
|
```bash
|
|
sudo apt update
|
|
sudo apt install libgtk-3-dev libpng-dev cmake ffmpeg libavcodec-dev libavformat-dev libavfilter-dev
|
|
wget https://github.com/opencv/opencv/archive/refs/tags/4.10.0.zip
|
|
unzip 4.10.0.zip
|
|
cd opencv-4.10.0
|
|
mkdir build && cd build
|
|
cmake -DCMAKE_INSTALL_PREFIX=/opt/opencv-4.10.0 -DBUILD_opencv_world=ON -DOPENCV_GENERATE_PKGCONFIG=ON -DBUILD_PERF_TESTS=OFF -DBUILD_TESTS=OFF ..
|
|
make -j8 && sudo make install
|
|
```
|
|
|
|
### Install ONNX runtime 1.18.0
|
|
```bash
|
|
wget https://github.com/microsoft/onnxruntime/releases/download/v1.18.0/onnxruntime-linux-x64-1.18.0.tgz
|
|
sudo tar xf onnxruntime-linux-x64-1.18.0.tgz -C /opt
|
|
```
|
|
|
|
### Install ffmpeg and mediamtx RTSP server:
|
|
```bash
|
|
sudo apt update
|
|
sudo apt install ffmpeg
|
|
|
|
# For Amd64-like platforms:
|
|
wget https://github.com/bluenviron/mediamtx/releases/download/v1.8.4/mediamtx_v1.8.4_linux_amd64.tar.gz
|
|
mkdir mediamtx
|
|
tar xf mediamtx_v1.8.4_linux_amd64.tar.gz -C mediamtx
|
|
|
|
# For OrangePi5 use this lib instead (since it's an ARM platform):
|
|
wget https://github.com/bluenviron/mediamtx/releases/download/v1.9.1/mediamtx_v1.9.1_linux_arm64v8.tar.gz
|
|
mkdir mediamtx
|
|
tar xf mediamtx_v1.9.1_linux_arm64v8.tar.gz -C mediamtx
|
|
```
|
|
|
|
### If you use video file from the local RTSP server:
|
|
```bash
|
|
cd mediamtx
|
|
./mediamtx
|
|
```
|
|
|
|
### Play Azaion mp4 video file from RTSP server ... :
|
|
```bash
|
|
ffmpeg -re -stream_loop -1 -i $HOME/azaion/videos/for_ai_cut.mp4 -c copy -f rtsp rtsp://localhost:8554/live.stream
|
|
```
|
|
|
|
### ... or play simple video file from RTSP server:
|
|
```bash
|
|
ffmpeg -re -stream_loop -1 -i $HOME/azaion/videos/table.mp4 -c copy -f rtsp rtsp://localhost:8554/live.stream
|
|
```
|
|
|
|
### Test RTSP streaming with ffplay:
|
|
```bash
|
|
ffplay -rtsp_transport tcp rtsp://localhost:8554/live.stream
|
|
```
|
|
|
|
### Compile and run rtsp_ai_player with YOLOv8 medium AI model:
|
|
```bash
|
|
cd autopilot/misc/rtsp_ai_player
|
|
qmake6 && make
|
|
./rtsp_ai_player ~/azaion/models/onnx/yolov8m.onnx
|
|
```
|
|
|
|
### Compile and run rtsp_ai_player with Azaion AI model:
|
|
```bash
|
|
cd autopilot/misc/rtsp_ai_player
|
|
qmake6 && make
|
|
./rtsp_ai_player ~/azaion/models/azaion/azaion-2024-08-13.onnx
|
|
```
|
|
### Compile and run rtsp_ai_player with YOLOv8 medium model and gimbal camera:
|
|
```bash
|
|
cd autopilot/misc/rtsp_ai_player
|
|
qmake6 CONFIG+=gimbal && make
|
|
./rtsp_ai_player ~/azaion/models/onnx/yolov8m.onnx
|
|
```
|
|
|
|
## How to use application on Orange PI 5.
|
|
|
|
### Install ffmpeg and mediamtx to Ubuntu PC:
|
|
```bash
|
|
sudo apt update
|
|
sudo apt install ffmpeg
|
|
wget https://github.com/bluenviron/mediamtx/releases/download/v1.8.4/mediamtx_v1.8.4_linux_amd64.tar.gz
|
|
mkdir mediamtx
|
|
tar xf mediamtx_v1.8.4_linux_amd64.tar.gz -C mediamtx
|
|
```
|
|
|
|
### Launch RTSP server in Ubuntu PC
|
|
```bash
|
|
cd mediamtx
|
|
./mediamtx
|
|
```
|
|
|
|
### Play RTSP video stream in Ubuntu PC
|
|
```bash
|
|
ffmpeg -re -stream_loop -1 -i SOME_MP4_VIDEO_FILE -c copy -f rtsp rtsp://localhost:8554/live.stream
|
|
```
|
|
|
|
### Test RTSP stream with ffplay in Ubuntu PC:
|
|
```bash
|
|
ffplay -rtsp_transport tcp rtsp://localhost:8554/live.stream
|
|
```
|
|
|
|
### Compile and launch RTSP AI PLAYER in Orange PI 5:
|
|
Modify autopilot/misc/rtsp_ai_player/aiengineconfig.h and change 192.168.168.91 to IP address of RTSP source
|
|
|
|
```bash
|
|
cd autopilot/misc/rtsp_ai_player
|
|
mkdir build
|
|
cd build
|
|
qmake6 CONFIG+=opi5 .. && make -j8 && ./rtsp_ai_player [RKNN_MODEL_FILE]
|
|
```
|
|
|