Build a Low-Latency Raspberry Pi RTSP Viewer with Hardware Decoding

If you need to view an IP camera feed on a Raspberry Pi with sub-100ms latency, standard media players like VLC will bottleneck you with 1-to-3-second delays. The direct answer for a low-latency raspberry pi rtsp viewer is to bypass standard players and use Python 3.11 with OpenCV’s GStreamer backend, explicitly calling the v4l2h264dec hardware decoder on a Raspberry Pi 5.

This build gives you a kiosk-grade, hardware-accelerated video sink while leaving the Python main loop free to handle GPIO triggers—like switching camera feeds via a physical button or firing a relay when a stream drops.

The Verdict: Choosing Your RTSP Rendering Stack

Before wiring anything, you need to pick the right software stack. Most tutorials default to VLC or ffplay, which are fine for casual viewing but fail in embedded kiosk or security applications where latency and GPIO integration matter.

Hardware Spec Sheet & Pin Mapping

This build targets the Raspberry Pi 5 (4GB variant) running Raspberry Pi OS (Bookworm, 64-bit). The Pi 5’s Cortex-A76 cores and updated stateless V4L2 video decoder make it vastly superior to the Pi 4 for multi-stream RTSP decoding.

Parts List

• Compute: Raspberry Pi 5 (4GB) — ~$60 USD
• Enclosure/Cooling: Argon ONE V3 M.2 NVMe Pi 5 Case (includes active fan and clean GPIO routing) — ~$35 USD
• Display Link: Micro-HDMI to HDMI 2.1 cable (1m) — ~$10 USD
• Control/Output: Generic 5V 2-Channel Optocoupler Relay Module — ~$6 USD
• Input: Momentary push-button switch (normally open) — ~$2 USD

GPIO Pin Mapping

We are mapping one physical button to swap between two RTSP camera URLs, and two relay channels to trigger external hardware (like a physical alarm light) if a stream drops.

Step-by-Step Build: GStreamer Pipeline & Python Control

Difficulty Rating: Intermediate (Requires basic Linux CLI and Python familiarity).
Time to Complete: 45 minutes.

1. Flash the OS: Use Raspberry Pi Imager to flash Raspberry Pi OS (64-bit, Bookworm). Do not use the 32-bit version; the V4L2 hardware decoding libraries are optimized for 64-bit memory addressing.
2. Install Dependencies: Open a terminal and install the GStreamer base plugins and OpenCV with GStreamer support.

   sudo apt update
   sudo apt install python3-opencv python3-gpiozero gstreamer1.0-plugins-bad gstreamer1.0-libav gstreamer1.0-tools libgstreamer1.0-dev -y

3. Verify Hardware Decoding: Before writing Python, test the raw GStreamer pipeline in the terminal to ensure your camera's H.264 stream is recognized by the Pi 5's stateless decoder.

   gst-launch-1.0 rtspsrc location=rtsp://admin:password@192.168.1.100:554/stream1 latency=50 ! rtph264depay ! h264parse ! v4l2h264dec ! videoconvert ! autovideosink sync=false

   If a window opens with your camera feed, your hardware pipeline is valid.
4. Write the Python Controller: Create a file named rtsp_viewer.py and paste the complete code block below.

Complete Python Implementation

import cv2
import time
import signal
import sys
from gpiozero import Button, OutputDevice

# --- PIN DEFINITIONS ---
PIN_BTN_SWITCH = 22
PIN_RELAY_1 = 17
PIN_RELAY_2 = 27

# --- RTSP STREAM CONFIGURATION ---
# Replace with your actual camera IPs, credentials, and paths
CAM_1_URL = "rtsp://admin:password123@192.168.1.100:554/cam/realmonitor?channel=1&subtype=0"
CAM_2_URL = "rtsp://admin:password123@192.168.1.101:554/cam/realmonitor?channel=1&subtype=0"

# GStreamer pipeline string for Pi 5 hardware decoding
# v4l2h264dec invokes the stateless V4L2 hardware decoder
GST_PIPELINE = (
    "rtspsrc location={url} latency=50 ! "
    "rtph264depay ! h264parse ! v4l2h264dec ! "
    "videoconvert ! autovideosink sync=false"
)

class RTSPKiosk:
    def __init__(self):
        self.current_cam = 1
        self.cap = None
        
        # Initialize GPIO
        self.btn_switch = Button(PIN_BTN_SWITCH, pull_up=True, bounce_time=0.2)
        self.relay_1 = OutputDevice(PIN_RELAY_1, active_high=True, initial_value=False)
        self.relay_2 = OutputDevice(PIN_RELAY_2, active_high=True, initial_value=False)
        
        self.btn_switch.when_pressed = self.toggle_camera
        
    def build_pipeline(self, url):
        return GST_PIPELINE.format(url=url)

    def start_stream(self):
        url = CAM_1_URL if self.current_cam == 1 else CAM_2_URL
        pipeline = self.build_pipeline(url)
        
        # cv2.CAP_GSTREAMER forces OpenCV to use the GStreamer backend
        self.cap = cv2.VideoCapture(pipeline, cv2.CAP_GSTREAMER)
        
        if not self.cap.isOpened():
            print(f"[ERROR] Failed to open RTSP stream: {url}")
            self.update_relays(False)
            return False
            
        self.update_relays(True)
        return True

    def update_relays(self, stream_active):
        if self.current_cam == 1:
            self.relay_1.value = stream_active
            self.relay_2.value = False
        else:
            self.relay_1.value = False
            self.relay_2.value = stream_active

    def toggle_camera(self):
        print("[INFO] Button pressed. Switching camera feed...")
        self.current_cam = 2 if self.current_cam == 1 else 1
        if self.cap:
            self.cap.release()
        self.start_stream()

    def run(self):
        if not self.start_stream():
            sys.exit(1)
            
        print("[INFO] Kiosk running. Press Ctrl+C to exit.")
        try:
            while True:
                ret, frame = self.cap.read()
                if not ret:
                    print("[WARN] Stream dropped. Attempting reconnect in 2s...")
                    self.update_relays(False)
                    time.sleep(2)
                    self.start_stream()
                    continue
                    
                # cv2.imshow is required to pump the GStreamer window events
                cv2.imshow("RTSP Kiosk", frame)
                
                # 1ms wait to prevent CPU hogging while catching GUI events
                if cv2.waitKey(1) & 0xFF == ord('q'):
                    break
                    
        except KeyboardInterrupt:
            pass
        finally:
            self.cleanup()

    def cleanup(self):
        print("\n[INFO] Shutting down kiosk...")
        if self.cap:
            self.cap.release()
        cv2.destroyAllWindows()
        self.relay_1.off()
        self.relay_2.off()
        self.btn_switch.close()

if __name__ == "__main__":
    kiosk = RTSPKiosk()
    kiosk.run()

Debugging the "Unable to Start Pipeline" Error

When working with OpenCV and GStreamer on embedded Linux, the error messages are notoriously opaque. The most common failure mode when launching the script is this exact string:

[ WARN:0@12.453] global cap_gstreamer.cpp:2859 handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module rtspsrc0 reported: Could not open resource for reading.

This is rarely a hardware failure. It is almost always a pipeline negotiation or network routing issue. Here is the ranked cause list and how to fix them.

The First Three Things to Check

1. URL Authentication and Typos (80% of cases): GStreamer’s rtspsrc does not handle URL encoding gracefully. If your password contains an @ or & symbol, the pipeline will parse the URL incorrectly and fail to connect. Fix: URL-encode special characters in the password string, or change the camera's password to alphanumeric only.
2. H.265 vs H.264 Encoding (15% of cases): Modern 4K IP cameras default to H.265 (HEVC) to save bandwidth. The Pi 5’s v4l2h264dec hardware decoder only decodes H.264. If the camera is pushing H.265, the rtph264depay element will choke. Fix: Log into your camera's web UI and force the video codec to H.264 (Baseline or Main profile).
3. UDP Buffer Exhaustion (5% of cases): By default, rtspsrc uses UDP. On a Pi connected via 2.4GHz WiFi, packet loss will cause the jitterbuffer to stall and drop the resource. Fix: Force TCP transport in the pipeline by appending protocols=tcp to the rtspsrc element: rtspsrc location=... protocols=tcp latency=50 ! ...

Extending and Simplifying the Build

Depending on your final deployment environment, you may need to strip this build down to its bare essentials or scale it up for production.

How to Simplify (The "Dumb Kiosk" Route)

If you do not need GPIO control, stream-switching, or Python-level frame analysis, delete the Python script entirely. You can achieve a highly optimized, auto-restarting kiosk using just mpv and systemd.

1. Install mpv: sudo apt install mpv
2. Create a systemd service that executes:

   mpv --fullscreen --no-cache --untimed --no-demuxer-thread --video-sync=audio --vd-lavc-threads=4 rtsp://admin:pass@192.168.1.100/stream1

This removes the OpenCV overhead and relies purely on the mpv binary's internal FFmpeg/V4L2 hooks, saving roughly 40MB of RAM.

How to Extend (Adding AI or Recording)

If you want to extend this build to record 10-second clips when a physical sensor trips, or run YOLO object detection on the frames:

• For Recording: Do not use OpenCV's cv2.VideoWriter for RTSP; re-encoding H.264 on the CPU will spike temperatures and drop frames. Instead, use Python's subprocess module to spawn a headless ffmpeg instance that copies the stream directly to disk without re-encoding:

  ffmpeg -i rtsp://... -c copy -t 10 -y /home/pi/clips/event.mp4

• For AI Inference: Swap the autovideosink at the end of your GStreamer pipeline with an appsink. This allows OpenCV to pull raw matrices (cv2.Mat) into memory for passing to a TensorFlow Lite or PyTorch inference engine, while GStreamer handles the heavy lifting of network retrieval and hardware decoding in the background.

For deeper reading on GStreamer plugin mechanics and Pi-specific video pipelines, refer to the GStreamer Plugin Writer's Guide and the Raspberry Pi Camera and Video Documentation.