Live stream video from drone (via Raspberry Pi)

Introduction

Situational awareness is essential in the operation of autonomous drones, with live video streaming being a key component in achieving situational awareness. In this guide, part of a series focused on building a 4G-connected autonomous drone, I detail a practical solution for setting up a live video stream from a drone to a Ground Control Station (GCS).

This guide is particularly aimed at those who are looking to build custom drones using a Raspberry Pi as a companion computer. Whether for surveillance, research, or recreational purposes, this setup offers a reliable way to transmit real-time video.

Prerequisites

Before we dive into creating the GStreamer workflow, there are a few essential components you’ll need for a successful implementation:

1. Raspberry Pi: This will serve as the drone’s companion computer. I used a Raspberry Pi 4B as my companion computer with 8GB of RAM, running the 64-bit version of Raspberry Pi OS. There are other great choices for a companion computer but the required GStreamer plugins will differ.
2. Camera: A key component for capturing video. I used a ArduCam 12MP camera module 3. Ensure your camera is compatible with the Raspberry Pi and properly connected. You can find guidance on setting up your camera in the Raspberry Pi documentation.
3. Ground Control Station (GCS): This is where you’ll receive and monitor the video stream. In my setup, I used a Mac running QGroundControl. Ensure that QGroundControl is installed and configured on your GCS. Installation instructions can be found here.
4. Network connectivity: Establish a reliable connection between your Raspberry Pi and the GCS. Both devices should be on the same network and able to communicate with each other.

By making sure these prerequisites are in place, you’ll be well-prepared to follow the steps in this guide for setting up live video streaming from your drone.

Install GStreamer

GStreamer is a versatile open-source multimedia framework and the backbone of our solution. It enables us to capture, process, and stream video through a customizable pipeline. We’ll use GStreamer to capture video from the Raspberry Pi camera, compress it, and then transmit it over the network. Here’s how to install it:

1. Update your system: Before installing new packages, it’s a good practice to update the list of available packages. Enter the following command:

sudo apt-get update

2. Install GStreamer: Now, install GStreamer along with plugins necessary for video capturing and processing. Enter the following command:

sudo apt install libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev \
  libgstreamer-plugins-bad1.0-dev gstreamer1.0-plugins-base \
  gstreamer1.0-plugins-good gstreamer1.0-plugins-bad gstreamer1.0-libav \
  gstreamer1.0-plugins-ugly gstreamer1.0-tools gstreamer1.0-x \
  gstreamer1.0-gl gstreamer1.0-plugins-base-apps libcamera-dev \
  gstreamer1.0-libcamera

This command installs the GStreamer development libraries, base plugins, and additional plugins for advanced processing.

3. Verify installation: After installation, verify that GStreamer is installed correctly by checking its version:

gst-launch-1.0 --version

At time this guide was written, I got the following version information:

gst-launch-1.0 version 1.22.0
GStreamer 1.22.0
https://tracker.debian.org/pkg/gstreamer1.0

By following these steps, you have successfully installed GStreamer on your Raspberry Pi. This installation lays the foundation for setting up the live video streaming from your drone.

Setup a test stream

Before we begin streaming live video from the camera, it’s important to first establish a test stream to ensure everything is functioning correctly. This process involves creating a simple video stream from the Raspberry Pi to the Ground Control Station (GCS). We’ll use a test pattern generated by GStreamer as our video source. Here’s how to set it up:

1. Generate test stream on Raspberry Pi: Run the following command to start broadcasting a test video stream. This will stream video of a bouncing ball. The stream will be encoded in H.264 format which is a common video compression standard:

gst-launch-1.0 videotestsrc pattern=ball ! video/x-raw,width=640,height=480 ! x264enc ! rtph264pay ! udpsink host=<gcs> port=5000

In this command, videotestsrc pattern=ball generates the test pattern, x264enc compresses the video, and udpsink transmits it to the specified host and port. Replace gcs with the hostname of your gcs.

2. Configure QGroundControl for video streaming:

• Navigate to: Application Settings -> General -> Fly View -> Video Settings.
• Set the Source to UDP h.264 Video Stream and the UDP port to 5000.

3. Verify the Stream:

• Once everything is set up and running, you should see video of a bouncing ball in QGroundControl.
• If the video does not appear, check that the Raspberry Pi and GCS are on the same network and can communicate. Also check that the port on the sending and receiving end are correct. I used 5000 in this guide.

By following these steps, you’ll establish a basic video streaming pipeline from your Raspberry Pi to your GCS, laying the groundwork for live video streaming from your drone’s camera.

Setup stream from camera

After successfully testing with a mock video stream, the next step is to establish a live video stream from the camera attached to the Raspberry Pi. This live stream will capture real-time video, encode it, and then transmit it to your GCS. Here’s how to set it up:

1. Start live stream on Raspberry Pi:

• Run the following command to begin streaming live video from the camera. This pipeline captures the video, encodes it in the H.264 format for efficient bandwidth use, and transmits it via UDP using the RTP (Real-time Transport Protocol). Remember for replace gcs with your GCS hostname or IP address:

gst-launch-1.0 -v libcamerasrc ! video/x-raw, width=640, height=480, framerate=30/1 ! timeoverlay ! videoconvert ! video/x-raw,format=I420 ! x264enc ! rtph264pay config-interval=-1 ! udpsink host=<gcs> port=5000 sync=false async=true

Here’s a breakdown of what each component in the pipeline does:

To learn more about a plugin, use the following command. It will provided detailed information about the allowed parameters and formats of inputs and outputs.

gst-inspect-1.0 <plugin name> 

Documentation of GStreamer plugins can also be found here.

Stream on boot

To ensure that your live video stream starts automatically every time the drone is powered on, we’ll set up the streaming command as a service that runs on boot. This automation saves you from manually starting the stream each time. Follow these steps to set it up on your Raspberry Pi:

1. Create the streaming script:

• On the Raspberry Pi, create a new script named stream-video.sh with the following content. This script encapsulates the commands to start the live stream:

#!/bin/bash
set -e

width=640
height=480
target=127.0.0.1 # replace with your GCS hostname or IP address
port=5000

while [[ "$#" -gt 0 ]]; do
    case $1 in
        -t|--target) target="$2"; shift ;;
        -w|--width) width="$2"; shift ;;
        -h|--height) height="$2"; shift ;;
        -p|--port) port="$2"; shift ;;
        *) echo "Unknown parameter passed: $1"; exit 1 ;;
    esac
    shift
done

echo "Starting ${width}x${height} stream to ${target}:${port}"

gst-launch-1.0 libcamerasrc ! video/x-raw, width=${width}, height=${height}, framerate=30/1 ! timeoverlay ! openh264enc ! rtph264pay config-interval=-1 pt=96 ! udpsink host=${target} port=${port} sync=false async=true

2. Make the script executable:

• Run the following command to make your script executable:

sudo chmod +x /path/to/stream-video.sh

• I stored my script in /usr/local/bin so I could easily use it outside of the service.

3. Create a systemd service:

Create a new systemd service file at /lib/systemd/system/stream-video.service with the following content:

[Unit]
Description=Runs live video stream to GCS
After=multi-user.target

[Service]
User=pi
ExecStart=/path/to/stream-video.sh -t gcs
Restart=always
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=stream_video_service

[Install]
WantedBy=multi-user.target

• Replace /path/to/stream-video.sh with the actual path to your script and gcs with the IP address or hostname of your GCS.

4. Enable and start the service:

• Enable the new service to run on boot:

sudo systemctl enable stream-video.service

• You can start the service immediately without rebooting using:

sudo systemctl start stream-video.service

5. Verify the service:

• Check the status of your service to ensure it’s active and running:

sudo systemctl status stream-video.service 

With these steps, your Raspberry Pi is now configured to automatically start streaming video to your GCS upon booting. This setup enhances the efficiency operating the drone by removing the need for manual stream initiation.

Conclusion

In this guide, we’ve successfully navigated through the steps to live stream video from a drone utilizing a Raspberry Pi as a companion computer. From installing the versatile GStreamer framework to configuring and testing a live video stream, and finally automating the process to initiate the stream upon boot.

Future guides in this series will delve into topics such as transmitting telemetry from the drone to the GCS via the Raspberry Pi and establishing connectivity between the RaspberryPi and GCS across different networks.

Your experiences, insights, and queries are invaluable to this ongoing exploration of building an autonomous drone. I encourage you to share your thoughts and questions in the comments section below.