Real-Time Object Detection on Raspberry Pi 5: Hardware, Code, and Debugging

Getting reliable, high-framerate machine vision on a single-board computer used to mean compromising on resolution or settling for 2 FPS on a CPU. In 2026, the hardware landscape has shifted entirely. If you are building a security camera, a robotic rover, or a smart-home occupancy sensor, you need a deterministic pipeline that won't choke when a second person walks into the frame.

This guide cuts through the abstraction layers. We are targeting object detection on Raspberry Pi 5 using the official AI Kit. You will get the exact bill of materials, the physical pin mapping, a production-ready Python inference script with error handling, and a debugging matrix for the specific failure modes that plague this hardware stack.

The Verdict: Which Hardware Stack for Object Detection on Raspberry Pi?

For real-time object detection on Raspberry Pi, the Raspberry Pi 5 (8GB) paired with the official AI Kit (Hailo-8L NPU) is the definitive choice. It delivers 13 TOPS (Trillions of Operations Per Second) for roughly $70 added to the base board cost, pushing YOLOv8n inference past 30 FPS at 720p. CPU-only inference on the Pi 5 maxes out around 4 FPS—usable for time-lapse, but useless for tracking.

Decision Path: Choose Your NPU Stack

Parts List and Pin Mapping

Do not substitute the power supply. The Pi 5 + AI Kit + Camera draws transient spikes up to 24W during NPU initialization. A standard 5V/3A phone charger will cause a brownout and corrupt the PCIe bus.

Exact Bill of Materials

• Compute: Raspberry Pi 5 (8GB) [Part: SC1185] - ~$80
• NPU: Raspberry Pi AI Kit (Hailo-8L M.2 HAT+) [Part: SC1146] - ~$70
• Power: Official 27W USB-C PD Power Supply [Part: SC1099] - ~$12
• Optics: Raspberry Pi Camera Module 3 (IMX708, 12MP) [Part: SC1042] - ~$25
• Storage: 64GB NVMe SSD via M.2 HAT (Optional but recommended over SD for model I/O) - ~$15

Pin and Connector Mapping

Step-by-Step Assembly and Software Setup

1. Prep the HAT+: Peel the thermal pad backing off the Hailo-8L chip. Align the M.2 module into the HAT+ slot and secure it. Warning: The thermal pad is fragile; do not reapply it once peeled.
2. Connect the PCIe Ribbon: Lift the black retaining collar on the Pi 5's PCIe port. Insert the 40-pin ribbon from the HAT+. Ensure the metal traces face the USB ports. Push the collar down firmly.
3. Mount the HAT+: Use the provided 12mm brass standoffs to secure the HAT+ over the Pi 5 GPIO header.
4. Attach the Camera: Route the IMX708 FPC cable through the HAT+ slot. Insert it into the CAM1 port (the port closest to the USB-C power). Pull the collar up, insert, push collar down.
5. Flash and Update OS: Flash Raspberry Pi OS (Bookworm, 64-bit) using Raspberry Pi Imager. Boot, connect to WiFi, and run:

   sudo apt update && sudo apt full-upgrade -y
   sudo apt install rpicam-apps hailo-all -y
   sudo reboot

6. Verify NPU Enumeration: After reboot, run hailortcli fw-control identify. You should see Hailo-8L and the firmware version. If it fails, check the PCIe ribbon orientation.

Complete Python Code for Real-Time Inference

The following script uses picamera2 for zero-copy frame grabbing and tflite_runtime for inference. While the Hailo NPU uses proprietary HEF (Hailo Executable Format) models compiled via the Hailo Dataflow Compiler, this script demonstrates the robust TFLite CPU/XNNPACK pipeline that serves as the foundational fallback and debugging baseline on the Pi 5. It includes explicit error handling for camera and model initialization.

import sys
import time
import cv2
import numpy as np
from picamera2 import Picamera2, MappedArray
from tflite_runtime.interpreter import Interpreter

# --- Configuration & Pin/Path Definitions ---
CAMERA_ID = 0  # 0 for CAM1 on Pi 5 when using picamera2 standard mapping
MODEL_PATH = "yolov8n_full_integer_quant.tflite"  # Download from Ultralytics/TFLite repo
CONFIDENCE_THRESHOLD = 0.45
NMS_THRESHOLD = 0.50
RESOLUTION = (1280, 720)  # 720p for optimal CPU/XNNPACK throughput

def load_labels(path="coco_labels.txt"):
    """Loads COCO class labels. Fails gracefully if missing."""
    try:
        with open(path, 'r') as f:
            return [line.strip() for line in f.readlines()]
    except FileNotFoundError:
        print("[WARN] coco_labels.txt not found. Using generic numeric labels.")
        return [str(i) for i in range(80)]

def main():
    # 1. Initialize Interpreter with XNNPACK delegate for Pi 5 Cortex-A76 optimization
    try:
        interpreter = Interpreter(model_path=MODEL_PATH, num_threads=4)
        interpreter.allocate_tensors()
    except Exception as e:
        print(f"[FATAL] Failed to load TFLite model: {e}")
        sys.exit(1)

    input_details = interpreter.get_input_details()
    output_details = interpreter.get_output_details()
    input_shape = input_details[0]['shape']
    labels = load_labels()

    # 2. Initialize Picamera2
    picam2 = Picamera2(CAMERA_ID)
    config = picam2.create_preview_configuration(main={"size": RESOLUTION, "format": "RGB888"})
    picam2.configure(config)
    
    try:
        picam2.start()
    except RuntimeError as e:
        print(f"[FATAL] Camera acquisition failed: {e}")
        print("Check CAM1 ribbon cable orientation and 'libcamera-hello' status.")
        sys.exit(1)

    print("[INFO] Pipeline initialized. Streaming...")
    
    # 3. Inference Loop
    while True:
        frame = picam2.capture_array()
        frame_rgb = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
        
        # Preprocess for TFLite
        input_tensor = cv2.resize(frame_rgb, (input_shape[1], input_shape[2]))
        input_tensor = np.expand_dims(input_tensor, axis=0).astype(np.uint8)
        
        # Run Inference
        start_time = time.time()
        interpreter.set_tensor(input_details[0]['index'], input_tensor)
        interpreter.invoke()
        inference_time = time.time() - start_time
        
        # Post-process (Extract boxes, classes, scores)
        boxes = interpreter.get_tensor(output_details[0]['index'])[0]
        classes = interpreter.get_tensor(output_details[1]['index'])[0]
        scores = interpreter.get_tensor(output_details[2]['index'])[0]
        
        # Draw bounding boxes
        for i in range(len(scores)):
            if scores[i] > CONFIDENCE_THRESHOLD:
                ymin, xmin, ymax, xmax = boxes[i]
                imH, imW, _ = frame.shape
                xmin, xmax = int(xmin * imW), int(xmax * imW)
                ymin, ymax = int(ymin * imH), int(ymax * imH)
                
                cv2.rectangle(frame, (xmin, ymin), (xmax, ymax), (0, 255, 0), 2)
                label = f"{labels[int(classes[i])]}: {scores[i]:.2f}"
                cv2.putText(frame, label, (xmin, ymin - 10), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 255, 0), 2)
        
        # Overlay FPS
        fps = 1.0 / inference_time
        cv2.putText(frame, f"FPS: {fps:.1f}", (10, 30), cv2.FONT_HERSHEY_SIMPLEX, 1, (0, 0, 255), 2)
        
        cv2.imshow("Object Detection - Pi 5", frame)
        if cv2.waitKey(1) & 0xFF == ord('q'):
            break

    picam2.stop()
    cv2.destroyAllWindows()

if __name__ == "__main__":
    main()

Note: To utilize the Hailo-8L NPU directly in Python, you must compile a standard ONNX model into an .hef file using the Hailo Dataflow Compiler, then swap the tflite_runtime interpreter for the hailo_platform Python API. The TFLite script above is provided for immediate out-of-the-box validation and debugging.

Debugging: First Three Things to Check When It Fails

Embedded vision fails at the intersection of hardware seating and Linux driver states. If your script crashes, follow this ranked decision path before rewriting code.

1. The Camera Fails to Initialize

Exact Error String: RuntimeError: Failed to acquire camera /dev/video0 or picamera2.Picamera2: RuntimeError: Failed to acquire camera

• Cause A (Most Likely): The FPC ribbon cable is inserted backwards into the CAM1 port. The metal traces must face the board (or away, depending on the specific FPC type, but standard Pi camera cables have the blue stiffener facing UP/away from the board on the CAM port).
• Cause B: You plugged the camera into CAM0 instead of CAM1. On the Pi 5, CAM1 is the primary port closest to the USB-C power connector.
• Fix: Power down completely. Reseat the cable. Run libcamera-hello in the terminal. If that fails, the hardware is seated wrong. If it succeeds, the issue is in your Python CAMERA_ID mapping.

2. The NPU is Not Detected on the PCIe Bus

Exact Error String: hailortcli: Failed to find Hailo device or RuntimeError: Failed to load delegate from libhailort.so

• Cause A: The 40-pin PCIe ribbon cable connecting the M.2 HAT+ to the Pi 5 is unseated or flipped.
• Cause B: PCIe Gen 3 is enabled in /boot/firmware/config.txt (dtparam=pciex1_gen=3) but the power supply is sagging, causing the PCIe link to train down to Gen 1 or fail entirely.
• Fix: Run lspci | grep Hailo. If it returns nothing, check the ribbon. If it returns the device but hailortcli fails, revert config.txt to Gen 2 (dtparam=pciex1_gen=2) and reboot. Gen 2 provides 500 MB/s, which is more than enough for the Hailo-8L's 13 TOPS throughput.

3. Picamera2 Buffer/Format Mismatch

Exact Error String: ValueError: Cannot set format: YUV420 or RuntimeError: Request failed

• Cause: TFLite and OpenCV expect RGB888 or BGR888 arrays, but picamera2 defaults to YUV420 for video pipelines to save bandwidth.
• Fix: Explicitly declare the format in the configuration dictionary as shown in the code block above: main={"size": RESOLUTION, "format": "RGB888"}. See the official Picamera2 manual for stream mapping.

Extending or Simplifying the Build

How to Simplify (The "No-HAT" Route)

If you cannot source the AI Kit or want to drop the build cost by $70, you can run the exact Python script above on the Pi 5's CPU. The Broadcom BCM2712's Cortex-A76 cores, combined with the TFLite XNNPACK delegate, will yield roughly 3 to 5 FPS at 720p. This is perfectly adequate for a doorbell camera that only triggers inference on a PIR motion sensor interrupt, saving power and thermal headroom.

How to Extend (Home Assistant Integration)

To turn this from a bench experiment into a smart-home node, add MQTT publishing. Install paho-mqtt via pip. Inside the inference loop, when scores[i] > CONFIDENCE_THRESHOLD, publish a JSON payload to your broker:

import json
import paho.mqtt.client as mqtt

client = mqtt.Client("Pi5_Vision")
client.connect("192.168.1.100", 1883, 60)

# Inside the detection loop:
payload = json.dumps({"class": labels[int(classes[i])], "confidence": float(scores[i])})
client.publish("homeassistant/vision/frontdoor", payload)

This allows Home Assistant to trigger automations (e.g., turning on porch lights when a "person" is detected with >80% confidence) without needing to stream heavy video feeds over your network. For deeper integration with the official Hailo NPU pipeline, refer to the Raspberry Pi AI Kit documentation and the HailoRT GitHub repository for native C++ and Python bindings.