autopilot/README.md

# Azaion DroneController and AiController
An autonomous drone controller with AI inference support. Programs uses MAVSDK framework and ArduPilot flight controller. Running the applications requires MAVSDK installation and compiling ArduPilot. Example has been tested in Ubuntu 22.04 and 24.04 environments.

## Install necessary dependencies
```bash
sudo apt update
sudo apt install ccache git build-essential qmake6 qt6-base-dev
```

## Speed up the compilations
```bash
echo "export MAKEFLAGS=\"-j$(($(nproc)))\"" >> ~/.bashrc
echo "export PATH=/usr/lib/ccache:\$PATH"  >> ~/.bashrc
```

## Clone source codes. SSH key must be added before the cloning!
```bash
git clone git@github.com:azaion/autopilot.git
git clone --recursive git@github.com:azaion/autopilot.git
git submodule update --init --recursive
```

## Install MAVSDK for Ubuntu 20.04
```bash
wget https://github.com/mavlink/MAVSDK/releases/download/v2.12.10/libmavsdk-dev_2.12.10_ubuntu20.04_amd64.deb
sudo dpkg -i libmavsdk-dev_2.12.10_ubuntu20.04_amd64.deb
```

## Install MAVSDK for Ubuntu 22.04
```bash
wget https://github.com/mavlink/MAVSDK/releases/download/v2.12.10/libmavsdk-dev_2.12.10_ubuntu22.04_amd64.deb
sudo dpkg -i libmavsdk-dev_2.12.10_ubuntu22.04_amd64.deb
```

## Install ONNX Runtime for Ubuntu (not required for embedded platforms)

### With GPU inference
```bash
wget https://github.com/microsoft/onnxruntime/releases/download/v1.18.0/onnxruntime-linux-x64-gpu-cuda12-1.18.0.tgz
sudo tar xf onnxruntime-linux-x64-gpu-cuda12-1.18.0.tgz -C /opt/
sudo ln -s /opt/onnxruntime-linux-x64-gpu-cuda12 /opt/onnxruntime-linux-x64-1.18.0
```

### With CPU inference
```bash
wget https://github.com/microsoft/onnxruntime/releases/download/v1.18.0/onnxruntime-linux-x64-1.18.0.tgz
sudo tar xf onnxruntime-linux-x64-1.18.0.tgz -C /opt/
```

## Install MAVSDK for Ubuntu 22.04 or 24.04 PCs
```bash
wget https://github.com/mavlink/MAVSDK/releases/download/v2.12.10/libmavsdk-dev_2.12.10_ubuntu22.04_amd64.deb
sudo dpkg -i libmavsdk-dev_2.12.10_ubuntu22.04_amd64.deb
```

## Build ArduPilot
```bash
git clone --recursive https://github.com/ArduPilot/ardupilot.git
cd ardupilot
./Tools/environment_install/install-prereqs-ubuntu.sh -y
. ~/.profile
./waf configure --board=sitl
./waf build
```

## Build and install OpenCV 4.10.0
```bash
sudo apt update
sudo apt install libgtk-3-dev libpng-dev cmake ffmpeg libavcodec-dev libavformat-dev libavfilter-dev
wget https://github.com/opencv/opencv/archive/refs/tags/4.10.0.zip
unzip 4.10.0.zip
cd opencv-4.10.0
mkdir build && cd build
cmake -DCMAKE_INSTALL_PREFIX=/opt/opencv-4.10.0 -DBUILD_opencv_world=ON -DOPENCV_GENERATE_PKGCONFIG=ON -DBUILD_PERF_TESTS=OFF -DBUILD_TESTS=OFF ..
make -j8 && sudo make install
```

## Install MAVSDK for embedded platforms

### Build and install MAVSDK from the sources if needed
```bash
sudo apt-get update
sudo apt-get install build-essential cmake git
git clone https://github.com/mavlink/MAVSDK.git
cd MAVSDK
git checkout tags/v2.12.10
git submodule update --init --recursive
cmake -Bbuild/default -DCMAKE_BUILD_TYPE=Release -H.
cmake --build build/default -j8
sudo cmake --build build/default --target install
```

## Build Azaion applications

### PC builds (uses ONNX Runtime for the AI inference)
```bash
qmake6 && make
```

### OPI5 builds (uses gimbal camera and RKNN libraries for the AI inference)
```bash
qmake6 CONFIG+=opi5 CONFIG+=gimbal && make
```

## Run Azaion drone controller with ArduPilot simulator in Ubuntu PC

### Launch similator in the ArduPilot directory
```bash
./Tools/autotest/sim_vehicle.py --map --console -v ArduCopter
```

### Launch example application in the new terminal window after waiting simulator (around 1 min) to be ready
```bash
./drone_controller/drone_controller ./drone_controller/mission.json quadcopter udp
```

## Run Azaion AI controller in Ubuntu PC

### Install ffmpeg and mediamtx
```bash
sudo apt update
sudo apt install ffmpeg
wget https://github.com/bluenviron/mediamtx/releases/download/v1.8.4/mediamtx_v1.8.4_linux_amd64.tar.gz
mkdir mediamtx
tar xf mediamtx_v1.8.4_linux_amd64.tar.gz -C mediamtx
```

### Launch RTSP server if real camera is not used
```bash
cd mediamtx
./mediamtx
```

### Playback of RTSP video stream when no real camera is used
```bash
ffmpeg -re -stream_loop -1 -i SOME_MP4_VIDEO_FILE -c copy -f rtsp rtsp://localhost:8554/live.stream
```

### Test RTSP stream with ffplay
```bash
ffplay -rtsp_transport tcp rtsp://localhost:8554/live.stream
```

### Compile and run AI controller without camera support
Modify ./ai_controller/aiengineconfig.h and change IP address of RTSP source
```bash
qmake6 CONFIG+=opi5 && make && ./ai_controller/ai_controller [ONNX_MODEL_FILE]
```

### Compile and run AI controller with camera support
Modify ./ai_controller/aiengineconfig.h and change IP address of RTSP source
```bash
qmake6 CONFIG+=opi5 CONFIG+=gimbal && make && ./ai_controller/ai_controller [ONNX_MODEL_FILE]
```

## Run Azaion AI controller in OPI5

### Compile and run AI controller without gimbal camera support
Modify ./ai_controller/aiengineconfig.h and change IP address of RTSP source
```bash
qmake6 CONFIG+=opi5 && make && ./ai_controller/ai_controller [RKNN_MODEL_FILE]
```

### Compile and run AI controller without gimbal camera support
Modify ./ai_controller/aiengineconfig.h and change IP address of RTSP source
```bash
qmake6 CONFIG+=opi5 CONFIG+=gimbal && make && ./ai_controller/ai_controller [RKNN_MODEL_FILE]
```