Offline Speech to Text on Raspberry Pi 5: Vosk vs Whisper Build Guide

To run reliable offline speech to text on Raspberry Pi, use the Vosk engine paired with a Seeed ReSpeaker 2-Mic I2S HAT on a Raspberry Pi 5 8GB. This combination delivers sub-200ms latency without relying on cloud APIs or saturating the CPU, which is a common failure point when attempting to run Whisper.cpp on ARM architectures without dedicated NPU acceleration.

The Decision Tree: Which Speech-to-Text Engine?

Before wiring the HAT, you must select the right inference engine. The choice dictates your RAM requirements, latency, and network dependency. Here is the decision matrix for embedded Raspberry Pi deployments in 2026:

Decision Path: If your project requires real-time command parsing (e.g., home automation relays, robot navigation) and must function during network outages, choose Vosk. If you are building a post-processing dictation tool where a 3-second delay is acceptable and accuracy on heavy accents is paramount, choose Whisper.cpp. For this build, we terminate on Vosk.

Hardware Spec Sheet & Pin Mapping

The Seeed ReSpeaker 2-Mic HAT uses the WM8960 stereo codec, communicating via I2S for audio data and I2C for LED/button control. Because the Pi 5 routes peripherals through the RP1 chip, the physical 40-pin header maintains backward compatibility, but the device tree overlays must be Pi 5 specific.

I2S and I2C Pin Mapping

The HAT uses the following BCM pins. Do not use these pins for GPIO in your Python script, or you will crash the audio bus.

Step-by-Step Build: Vosk Offline STT on Pi 5

1. Flash the OS: Use Raspberry Pi Imager to flash Raspberry Pi OS Bookworm (64-bit). Enable SSH and configure WiFi in the Imager settings.
2. Mount the HAT: Align the ReSpeaker 2-Mic HAT over the 40-pin header and press down firmly. Secure with the included standoffs.
3. Install the Voicecard Driver: Bookworm uses PipeWire by default, which can conflict with raw ALSA I2S access. We must install the Seeed DKMS driver to properly register the WM8960 codec.

   git clone https://github.com/HinTak/seeed-voicecard.git
   cd seeed-voicecard
   sudo ./install.sh
   sudo reboot

4. Verify ALSA Registration: After reboot, run aplay -l. You must see card 1: seeed2micvoicecard. If it shows as card 0, that is fine, but note the card number.
5. Set Up Python Environment: Never install audio libraries globally in Bookworm. Use a virtual environment.

   python3 -m venv ~/stt_env
   source ~/stt_env/bin/activate
   pip install vosk sounddevice

6. Download the Vosk Model: Download the small US English model (~50MB). For higher accuracy, download the large 1.8GB model, but ensure your Pi 5 has active cooling.

   wget https://alphacephei.com/vosk/models/vosk-model-small-en-us-0.15.zip
   unzip vosk-model-small-en-us-0.15.zip
   rm vosk-model-small-en-us-0.15.zip

The Python Implementation

This script uses a non-blocking audio callback via sounddevice to push raw PCM data into a thread-safe queue. The main thread reads the queue and feeds it to the Vosk Kaldi recognizer. This prevents audio buffer overruns during garbage collection pauses.

import sounddevice as sd
from vosk import Model, KaldiRecognizer
import queue
import json
import sys
import os

# Hardware mapping note: I2S pins (BCM 18, 19, 20, 21) are handled by the OS overlay.
# We address the HAT via its ALSA string name rather than hardcoded index.
DEVICE_NAME = 'seeed-2mic-voicecard'
MODEL_PATH = 'vosk-model-small-en-us-0.15'
SAMPLE_RATE = 16000

def find_audio_device(name_part):
    """Finds the ALSA device index for the ReSpeaker HAT."""
    devices = sd.query_devices()
    for i, dev in enumerate(devices):
        if name_part.lower() in dev['name'].lower() and dev['max_input_channels'] > 0:
            return i
    return None

def main():
    # 1. Verify Model Directory
    if not os.path.exists(MODEL_PATH):
        print(f'FATAL: Model directory "{MODEL_PATH}" not found. Did you extract the zip?')
        sys.exit(1)

    # 2. Load Vosk Model
    try:
        print('Loading Vosk model into RAM...')
        model = Model(MODEL_PATH)
        recognizer = KaldiRecognizer(model, SAMPLE_RATE)
    except Exception as e:
        print(f'FATAL: Failed to load model. Error: {e}')
        sys.exit(1)

    # 3. Locate Audio Device
    dev_index = find_audio_device(DEVICE_NAME)
    if dev_index is None:
        print(f'FATAL: Could not find input device containing "{DEVICE_NAME}".')
        print('Run "aplay -l" and "arecord -l" to check ALSA status.')
        sys.exit(1)

    # 4. Setup Thread-Safe Queue
    audio_queue = queue.Queue()

    def audio_callback(indata, frames, time, status):
        """Called by PortAudio in a separate thread."""
        if status:
            print(f'Audio Status Warning: {status}', file=sys.stderr)
        audio_queue.put(bytes(indata))

    # 5. Stream Audio
    print(f'Starting STT stream on device {dev_index}... Speak now.')
    try:
        with sd.RawInputStream(
            samplerate=SAMPLE_RATE,
            blocksize=8000,
            device=dev_index,
            dtype='int16',
            channels=1,
            callback=audio_callback
        ):
            while True:
                data = audio_queue.get()
                if recognizer.AcceptWaveform(data):
                    result = json.loads(recognizer.Result())
                    text = result.get('text', '').strip()
                    if text:
                        print(f'COMMAND RECOGNIZED: {text}')
                        # Add your GPIO/MQTT logic here
                        if 'shutdown' in text:
                            print('Shutdown command received. Exiting.')
                            break
                else:
                    # Optional: print partial results for UI feedback
                    partial = json.loads(recognizer.PartialResult())
                    if partial.get('partial'):
                        pass # print(f"Listening: {partial['partial']}", end='\r')

    except KeyboardInterrupt:
        print('\nStream interrupted by user.')
    except sd.PortAudioError as e:
        print(f'FATAL: PortAudio stream failed to open.\nError: {e}')
        sys.exit(1)

if __name__ == '__main__':
    main()

Debugging: Exact Errors and Ranked Causes

Audio pipelines on Linux embedded systems are notoriously fragile. When the script fails, it will almost always throw one of two specific errors. Here is how to resolve them.

Error 1: PortAudio Device Unavailable

Exact Error String: sounddevice.PortAudioError: Error opening InputStream: Invalid device [PaErrorCode -9986] or Device unavailable [PaErrorCode -9985]

Ranked Causes & Fixes:

1. PipeWire is hogging the ALSA device. Bookworm runs PipeWire by default. If PipeWire claims the WM8960 codec, raw ALSA access via PortAudio will fail. Fix: Run systemctl --user stop pipewire before executing your script, or configure PipeWire to release the device.
2. I2S Overlay Missing. The RP1 chip didn't initialize the I2S bus. Fix: Check /boot/firmware/config.txt for dtparam=i2s=on and dtoverlay=seeed-2mic-voicecard. If missing, the Seeed install script failed; re-run it.
3. Wrong Device Index. The script hardcoded an index that shifted after a USB device was plugged in. Fix: The provided code uses string matching (find_audio_device) to prevent this. Ensure DEVICE_NAME matches the output of arecord -l.

Error 2: Vosk Model Load Failure

Exact Error String: OSError: Cannot load model from 'vosk-model-small-en-us-0.15'

Ranked Causes & Fixes:

1. Incorrect Working Directory. You ran the script from ~/stt_env instead of the directory containing the extracted model folder. Fix: cd into the directory containing both your .py file and the vosk-model-... folder.
2. Corrupt Extraction. The unzip command created a nested folder (e.g., vosk-model-small-en-us-0.15/vosk-model-small-en-us-0.15). Fix: Run ls -l inside the model folder. You must see am, conf, graph, and ivector directories at the root of the path specified in MODEL_PATH.

Extending and Simplifying the Pipeline

Depending on your project requirements, you may need to alter the architecture of the STT pipeline.

How to Simplify (For Basic Trigger Scripts)

If you do not need continuous, real-time streaming and only want to record 3-second audio clips to parse after the fact, drop the queue and callback architecture entirely. Use blocking recording:

audio_data = sd.rec(int(3 * SAMPLE_RATE), samplerate=SAMPLE_RATE, channels=1, dtype='int16')
sd.wait()
if recognizer.AcceptWaveform(audio_data.tobytes()):
    print(json.loads(recognizer.Result())['text'])

How to Extend (For Smart Home Integration)

To turn this into a practical home automation node, extend the if text: block with an MQTT publisher. Install paho-mqtt in your virtual environment and publish the parsed string to a broker (e.g., Home Assistant Mosquitto). Furthermore, to prevent the Pi from processing ambient noise continuously, integrate Porcupine Wake Word Engine (by Picovoice) upstream of Vosk. Porcupine uses less than 2% CPU to listen for a trigger phrase (like 'Computer'), and only activates the Vosk inference pipeline when triggered, saving thermal headroom and reducing false positives.

References: Vosk Model Registry (Alphacephei), Seeed Studio ReSpeaker Wiki, Raspberry Pi Device Tree Configuration Docs.