Offline Voice Recognition with Raspberry Pi: Vosk & ReSpeaker HAT Build

To build reliable, offline voice recognition with Raspberry Pi, pair a Raspberry Pi 4 Model B (4GB) with the Seeed Studio ReSpeaker 2-Mics Pi HAT and the Vosk Python API. This combination bypasses cloud latency, eliminates API subscription fees, and keeps your voice data entirely on the local network. While cloud APIs like Google Speech-to-Text offer marginally higher accuracy in noisy rooms, a local Vosk model running on a Pi 4 with a dedicated I2S microphone HAT provides sub-500ms transcription latency and total privacy.

Before wiring anything up, you need to select the right audio frontend. The Pi's analog audio output is notoriously noisy, and its lack of an analog mic input means you must use USB or GPIO-based I2S audio. Here is how the most common microphone modules stack up for embedded voice projects.

Audio Hardware Comparison for Pi Voice Projects

Note: We are using the ReSpeaker 2-Mics Pi HAT for this build because it integrates the I2S ADC, an APA102 RGB LED ring, and a user button into a single footprint, eliminating breadboard wiring. However, because it physically overlaps the Pi 5's relocated mounting holes and PoE headers, this specific HAT is best suited for the Pi 4.

Parts List and GPIO Pin Mapping

This build targets the Raspberry Pi 4 Model B (4GB RAM) running Raspberry Pi OS (64-bit, Bookworm). The 4GB variant is the sweet spot; Vosk's small English model requires about 400MB of RAM, leaving plenty of headroom for your application logic and the OS GUI if needed.

Required Components

• Compute: Raspberry Pi 4 Model B (4GB) - ~$55.00
• Audio: Seeed Studio ReSpeaker 2-Mics Pi HAT (SKU: 103030275) - ~$12.50
• Storage: 32GB Samsung EVO Plus microSD (Class 10) - ~$9.00
• Power: Official Raspberry Pi 27W USB-C Power Supply (5.1V / 5A) - ~$12.00

ReSpeaker 2-Mic HAT Pin Mapping

The HAT communicates over three distinct buses. Do not attempt to use the standard GPIO 14/15 UART pins for audio; the WM8960 audio codec on the HAT strictly uses the I2S bus.

OS Configuration and Vosk Installation

Before writing code, we must configure the ALSA (Advanced Linux Sound Architecture) layer to recognize the I2S HAT and install the Vosk inference engine.

1. Flash the OS: Use Raspberry Pi Imager to flash Raspberry Pi OS (64-bit, Bookworm) to your SD card. Enable SSH and configure your WiFi in the imager settings.
2. Install the ReSpeaker Kernel Driver:
   git clone https://github.com/respeaker/seeed-voicecard.git
cd seeed-voicecard
sudo ./install.sh
   Reboot the Pi after installation.
3. Verify ALSA Recognition: Run arecord -l. You should see card 1: seeed2micvoicec [seeed-2mic-voicecard].
4. Set up Python Environment:
   sudo apt update && sudo apt install python3-pip python3-venv portaudio19-dev -y
mkdir ~/voice_project && cd ~/voice_project
python3 -m venv venv && source venv/bin/activate
5. Install Vosk and Audio Libraries:
   pip install vosk sounddevice
6. Download the Vosk Model: Head to the official Vosk model registry and download vosk-model-small-en-us-0.15 (46MB). Extract the zip directly into your ~/voice_project directory.

Complete Python Transcription Script

The following script initializes the I2S audio stream, buffers the raw PCM data, and passes it to the Vosk Kaldi recognizer. It includes explicit hardware pin definitions for the HAT's onboard button and LEDs, allowing you to easily extend the script to trigger recording only when the button is pressed.

import os
import json
import queue
import sounddevice as sd
from vosk import Model, KaldiRecognizer
import signal
import sys

# ==========================================
# HARDWARE PIN & DEVICE DEFINITIONS
# ==========================================
# ReSpeaker 2-Mic HAT Pin Definitions (BCM Numbering)
GPIO_BUTTON = 17      # Onboard user button (Active LOW)
GPIO_LED_DATA = 10    # APA102 LED Ring MOSI
GPIO_LED_CLK = 11     # APA102 LED Ring SCLK
GPIO_I2S_DIN = 20     # I2S Mic Data
GPIO_I2S_DOUT = 21    # I2S Speaker Data

# Audio Device Configuration
# Run 'python -m sounddevice' to verify your ReSpeaker card index.
# Usually 0 is Pi built-in (or dummy), 1 is ReSpeaker I2S.
DEVICE_INDEX = 1 
SAMPLE_RATE = 16000   # 16kHz is optimal for Vosk speech models
BLOCK_SIZE = 8000     # Buffer size in frames
MODEL_PATH = "vosk-model-small-en-us-0.15"

# Thread-safe queue for audio frames
audio_queue = queue.Queue()

def audio_callback(indata, frames, time_info, status):
    """Called by PortAudio in a separate thread for every audio block."""
    if status:
        print(f"[WARNING] Audio Stream Status: {status}", flush=True)
    # Push raw bytes to the queue for the main thread to process
    audio_queue.put(bytes(indata))

def graceful_exit(sig, frame):
    """Handle Ctrl+C to close the stream cleanly."""
    print("\n[INFO] Stopping transcription and exiting.")
    sys.exit(0)

def main():
    # Register signal handler for clean exit
    signal.signal(signal.SIGINT, graceful_exit)

    # 1. Validate Model Path
    if not os.path.exists(MODEL_PATH):
        print(f"[FATAL] Vosk model not found at '{MODEL_PATH}'.")
        print("Download from https://alphacephei.com/vosk/models and extract.")
        sys.exit(1)

    # 2. Load Vosk Model
    try:
        print(f"[INFO] Loading Vosk model from {MODEL_PATH}...")
        model = Model(MODEL_PATH)
        recognizer = KaldiRecognizer(model, SAMPLE_RATE)
    except Exception as e:
        print(f"[FATAL] Failed to initialize Vosk model: {e}")
        sys.exit(1)

    # 3. Open Audio Stream
    try:
        print(f"[INFO] Opening audio stream on device index {DEVICE_INDEX}...")
        with sd.RawInputStream(
            samplerate=SAMPLE_RATE,
            blocksize=BLOCK_SIZE,
            device=DEVICE_INDEX,
            dtype='int16',
            channels=1,
            callback=audio_callback
        ):
            print("[SUCCESS] Listening... Speak into the ReSpeaker. (Ctrl+C to stop)")
            
            # 4. Main Processing Loop
            while True:
                data = audio_queue.get()
                
                # AcceptWaveform returns True when a silence threshold is crossed,
                # indicating the end of a spoken phrase.
                if recognizer.AcceptWaveform(data):
                    result = json.loads(recognizer.Result())
                    spoken_text = result.get("text", "").strip()
                    
                    if spoken_text:
                        print(f"[TRANSCRIPT] {spoken_text}")
                        
                        # Example: Trigger action on wake word
                        if "turn on" in spoken_text:
                            print("[ACTION] Triggering GPIO relay logic...")
                            
                else:
                    # Optional: Print partial results for live UI feedback
                    # partial = json.loads(recognizer.PartialResult())
                    # print(f"... {partial.get('partial', '')}", end='\r')
                    pass

    except sd.PortAudioError as e:
        print(f"[FATAL] PortAudio Error: {e}")
        print("Check DEVICE_INDEX and ensure no other app holds the ALSA lock.")
        sys.exit(1)
    except Exception as e:
        print(f"[FATAL] Unexpected runtime error: {e}")
        sys.exit(1)

if __name__ == "__main__":
    main()

Debugging: The First Three Things to Check When It Fails

Audio routing on Linux is notoriously fragile. If your script crashes immediately, check these three specific failure modes in order.

1. The 'Invalid Device' PortAudio Error

Exact Error String: sounddevice.PortAudioError: Error opening InputStream: Invalid device [PaErrorCode -9986]

Ranked Causes & Fixes:

1. Wrong Device Index: You plugged in a USB keyboard or another audio device, shifting the ReSpeaker from index 1 to 2. Fix: Run python -m sounddevice in your terminal, find the exact index number next to 'seeed-2mic-voicecard', and update DEVICE_INDEX in the script.
2. Virtual Environment Isolation: You installed sounddevice globally but are running the script in a venv that lacks the PortAudio C-bindings. Fix: Ensure you ran pip install sounddevice after activating your venv.

2. The ALSA 'Unable to Open Slave' Lock

Exact Error String: ALSA lib pcm_dmix.c:1032:(snd_pcm_dmix_open) unable to open slave (Often followed by a PortAudio timeout).

Ranked Causes & Fixes:

1. Pipewire/PulseAudio Interference: The desktop audio manager has grabbed the I2S hardware exclusively. Fix: Kill the audio daemon with systemctl --user stop pipewire-pulse.service or run your script in a headless Lite OS environment.
2. Zombie Python Process: A previous run of your script crashed without closing the RawInputStream, leaving the ALSA device locked. Fix: Run sudo killall python3 to clear hung processes, then reboot.

3. Vosk Model Path or Memory Error

Exact Error String: OSError: [Errno 2] No such file or directory: 'vosk-model-small-en-us-0.15' OR MemoryError during model load.

Ranked Causes & Fixes:

1. Unextracted Zip: You downloaded the .zip but forgot to extract it. Vosk requires a folder containing the ivector and conf directories. Fix: Unzip the model in the same directory as your script.
2. Model Too Large: You downloaded the 1.8GB 'large' model on a Pi 4 with only 1GB or 2GB of RAM, causing an Out-Of-Memory (OOM) kernel panic during load. Fix: Stick to the small (46MB) or lgraph (120MB) models for Pi hardware.

How to Extend or Simplify the Build

Depending on your end goal, you may want to strip this project down to its bare essentials or scale it up into a full smart-home hub.

Simplifying: Drop the HAT for a USB Mic

If you don't need the RGB LEDs or the physical button, and you are using a Raspberry Pi 5 (where the 2-Mic HAT physically conflicts with the board layout), simplify the build by switching to the Adafruit USB Mini Microphone.
What changes: You skip the kernel driver installation entirely. The USB mic registers as a standard class-compliant audio device. Simply change DEVICE_INDEX to the USB mic's index, and the Python script remains 100% identical.

Extending: Adding Wake-Word Detection and MQTT

Running continuous transcription drains the Pi 4's CPU and generates false positives from background noise. To make this a production-ready smart-home node:

1. Add Picovoice Porcupine: Install the pvporcupine Python package. Use it to listen only for a specific wake word (e.g., "Hey Pi"). Once detected, trigger the Vosk transcription loop for exactly 5 seconds to capture the command.
2. Integrate MQTT: Import paho.mqtt.client. When Vosk returns a transcript like "turn on kitchen lights", parse the string and publish a JSON payload to your Home Assistant MQTT broker topic (homeassistant/light/kitchen/set).
3. Hardware Mute Switch: Wire a physical toggle switch to GPIO 17 (the HAT's button pin). Modify the script to read the GPIO state; if the pin is pulled LOW, bypass the audio callback entirely to guarantee hardware-level privacy.