Running OpenCV in Raspberry Pi 5: Build and Debug a Vision Sorter

To run OpenCV in Raspberry Pi 5 reliably today, you must abandon the legacy camera stack. The winning combination for real-time computer vision is the Raspberry Pi 5 (8GB) running Bookworm 64-bit, capturing frames via the picamera2 library (libcamera), and processing them with opencv-python-headless. This guide walks you through building a vision-guided hardware sorter, mapping the exact I2C pins, and debugging the specific pipeline errors that trap most makers on the new OS.

The Hardware Stack: Decision Path for OpenCV in Raspberry Pi

Before buying parts, you need to match your compute requirement to the right board and sensor. OpenCV contour detection is lightweight, but modern object detection (YOLO) requires neural accelerators. Use this decision matrix to lock in your exact BOM.

Hardware BOM and Pin Mapping

This project actuates a physical sorting gate using a PCA9685 16-channel PWM driver. The Pi 5's hardware PWM pins are often occupied by audio or fan headers, so offloading to an I2C PWM driver is the most robust jobsite practice.

Parts List

• Compute: Raspberry Pi 5 (8GB) with Active Cooler and 27W USB-C PSU
• Vision: Raspberry Pi Camera Module 3 (Standard or Wide angle)
• Actuator Driver: Adafruit PCA9685 16-Channel 12-bit PWM/Servo Driver (Product ID: 815)
• Servos: 2x SG90 Micro Servos (for diverter gates)
• Wiring: Female-to-Female jumper wires, Pi 5 mini HDMI cable (for initial setup)

Pin Mapping Table (Pi 5 to PCA9685)

Bulletproof Installation on Bookworm OS

The transition to Debian Bookworm broke thousands of legacy OpenCV tutorials. The picamera library is deprecated. You must use picamera2 and install OpenCV without GUI dependencies to prevent Wayland/X11 conflicts.

1. Flash the OS: Use Raspberry Pi Imager to flash Raspberry Pi OS (64-bit) Bookworm. Enable SSH and set your WiFi in the imager settings.
2. Update System Packages:
   sudo apt update && sudo apt upgrade -y
3. Install libcamera and Picamera2:
   sudo apt install -y python3-picamera2 python3-libcamera
4. Create a Virtual Environment: Bookworm enforces PEP 668, blocking global pip installs.
   python3 -m venv cv_env && source cv_env/bin/activate
5. Install OpenCV and I2C Libraries:
   pip install opencv-python-headless numpy adafruit-circuitpython-pca9685
6. Enable I2C: Run sudo raspi-config, navigate to Interface Options > I2C, and enable it. Reboot.

The Code: Vision-Guided Servo Actuation

This script captures frames via picamera2, converts the numpy array to an OpenCV BGR image, isolates red objects using HSV color masking, and triggers the PCA9685 servo if the object's centroid crosses the center threshold. It includes robust error handling for I2C and camera pipeline failures.

import time
import cv2
import numpy as np
from picamera2 import Picamera2
import board
import busio
from adafruit_pca9685 import PCA9685

# --- PIN & HARDWARE DEFINITIONS ---
I2C_SDA = board.SDA
I2C_SCL = board.SCL
SERVO_CHANNEL = 0
SERVO_MIN_PULSE = 150
SERVO_MAX_PULSE = 600
CENTER_THRESHOLD_X = 320  # Half of 640 width

def init_servo_driver():
    try:
        i2c = busio.I2C(I2C_SCL, I2C_SDA)
        pca = PCA9685(i2c, address=0x40)
        pca.frequency = 50
        return pca
    except ValueError as e:
        print(f'I2C Init Error: {e}. Check wiring and i2cdetect -y 1.')
        raise

def move_servo(pca, channel, angle):
    pulse_width = SERVO_MIN_PULSE + (angle / 180.0) * (SERVO_MAX_PULSE - SERVO_MIN_PULSE)
    pca.channels[channel].duty_cycle = int(pulse_width * 0xFFFF / 20000)

def main():
    print('Initializing Camera and Servos...')
    pca = init_servo_driver()
    move_servo(pca, SERVO_CHANNEL, 90)  # Center position
    
    picam2 = Picamera2()
    config = picam2.create_preview_configuration(main={'size': (640, 480), 'format': 'RGB888'})
    picam2.configure(config)
    
    try:
        picam2.start()
        time.sleep(2)  # Allow camera AGC to settle
        print('Pipeline active. Sorting...')
    except RuntimeError as e:
        print(f'Camera Start Failed: {e}')
        return

    try:
        while True:
            # Capture frame as numpy array (RGB)
            frame_rgb = picam2.capture_array()
            # Convert RGB to BGR for OpenCV processing
            frame_bgr = cv2.cvtColor(frame_rgb, cv2.COLOR_RGB2BGR)
            
            # Convert to HSV for color masking
            hsv = cv2.cvtColor(frame_bgr, cv2.COLOR_BGR2HSV)
            
            # Define range for red color (requires two masks due to HSV wrap-around)
            lower_red1 = np.array([0, 120, 70])
            upper_red1 = np.array([10, 255, 255])
            lower_red2 = np.array([170, 120, 70])
            upper_red2 = np.array([180, 255, 255])
            
            mask1 = cv2.inRange(hsv, lower_red1, upper_red1)
            mask2 = cv2.inRange(hsv, lower_red2, upper_red2)
            mask = cv2.bitwise_or(mask1, mask2)
            
            # Find contours
            contours, _ = cv2.findContours(mask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
            
            if contours:
                largest_contour = max(contours, key=cv2.contourArea)
                area = cv2.contourArea(largest_contour)
                
                if area > 500:  # Filter out noise
                    M = cv2.moments(largest_contour)
                    if M['m00'] != 0:
                        cx = int(M['m10'] / M['m00'])
                        
                        # Actuate servo based on X position
                        if cx > CENTER_THRESHOLD_X + 50:
                            move_servo(pca, SERVO_CHANNEL, 45)  # Push Right
                        elif cx < CENTER_THRESHOLD_X - 50:
                            move_servo(pca, SERVO_CHANNEL, 135) # Push Left
                        else:
                            move_servo(pca, SERVO_CHANNEL, 90)  # Center
    except KeyboardInterrupt:
        print('Stopping...')
    finally:
        picam2.stop()
        move_servo(pca, SERVO_CHANNEL, 90)
        pca.deinit()

if __name__ == '__main__':
    main()

Debugging: First 3 Checks and Exact Error Strings

When your pipeline fails, it usually fails at the intersection of libcamera and I2C. Before rewriting code, run through this diagnostic sequence.

Ranked Causes for Common Error Strings

Error String: cv2.error: OpenCV(4.8.1) /io/opencv/modules/imgproc/src/color.cpp:182: error: (-215:Assertion failed) !_src.empty() in function 'cvtColor'

• Cause 1 (Most Likely): You are trying to use cv2.VideoCapture(0) instead of picamera2. On Bookworm, /dev/video0 is not populated by default without specific V4L2 kernel overlays.
• Cause 2: The picam2.capture_array() returned None because the camera pipeline dropped. Fix: Add a time.sleep(2) after picam2.start() to let the AGC (Auto Gain Control) initialize.

Error String: OSError: [Errno 121] Remote I/O error (Thrown during PCA9685(i2c) initialization)

• Cause 1: The PCA9685 is not receiving 5V power on the green V+ terminal block. The logic chip might power up via VCC, but the I2C handshake fails under load.
• Cause 2: Missing pull-up resistors. The Adafruit board has 10k pull-ups onboard, but if you are using a generic clone, you may need to add 4.7k resistors between SDA/SCL and 3.3V.

Scaling the Build: Simplify or Extend

Once the baseline color sorter is running, you need to decide how to adapt it for your specific bench or production environment. Do not leave the system in a half-finished state; commit to one of these two paths.

Path A: Simplify for Data Logging (No Actuators)

If you only need to count parts or log defect rates, strip out the adafruit-circuitpython-pca9685 dependency entirely. Replace the servo actuation block with an MQTT publish command using paho-mqtt. Send the cx, cy, and area variables as a JSON payload to a local Mosquitto broker. This reduces hardware failure points and drops the CPU load by roughly 4%, allowing you to step up the camera resolution to 1080p without frame drops.

Path B: Extend to Neural Object Detection

HSV color masking fails under shifting ambient light. To make the system robust against shadows and varying part orientations, upgrade to YOLOv8. You will need to install the ultralytics package and export a custom-trained model to ONNX format. Because the Pi 5 CPU will bottleneck at 2-3 FPS running YOLOn, you must add the Raspberry Pi AI Kit (Hailo-8L) to the PCIe M.2 HAT+ slot. This offloads the tensor math to the NPU, pushing your inference back up to a usable 15-20 FPS for real-time sorting.

For deeper integration details on the libcamera pipeline, refer to the official Raspberry Pi Camera Software documentation, and for OpenCV wheel specifics, check the opencv-python GitHub repository. Always verify your I2C wiring against the Adafruit PCA9685 CircuitPython guide before applying power to the servo rail.