How to Build a Speech to Text Arduino Project Using ESP32-S3 and I2S

If you are searching for a speech to text Arduino project, the hard truth is that a standard 8-bit Arduino Uno or Nano cannot process human speech into text. They lack the RAM (2KB) and clock speed (16MHz) to buffer audio, let alone run acoustic models. To build a functional, real-world speech-to-text (STT) system in the Arduino IDE ecosystem in 2026, you must step up to a 32-bit dual-core microcontroller with native I2S hardware and WiFi.

The gold standard for this build is the ESP32-S3 (N8R8 variant) paired with an INMP441 I2S MEMS microphone. The ESP32-S3 handles the high-speed digital audio sampling via its I2S peripheral, buffers the data in its 8MB of PSRAM, and streams the WAV payload over WiFi to a cloud API like Google Cloud Speech-to-Text.

Hardware Reality Check: Choosing the Right Board

Before ordering parts, it is critical to understand why certain popular boards fail at audio tasks. The table below compares common microcontrollers for STT viability based on their SRAM, clock speed, and I2S peripheral support.

Source: Espressif ESP32-S3 Datasheet

Parts List and Pin Mapping

This build specifically targets the ESP32-S3-DevKitC-1 (N8R8). The "N8R8" designation is mandatory: it means 8MB Flash and 8MB Octal PSRAM. Do not use the N8 (no PSRAM) variant, or your DMA buffers will exhaust the internal SRAM and crash the board.

Bill of Materials

• MCU: ESP32-S3-DevKitC-1 (N8R8) - ~$9.00
• Microphone: INMP441 Omnidirectional I2S MEMS Module - ~$4.00
• Wiring: 24 AWG silicone stranded jumper wires
• Power: High-quality USB-C data cable (capable of 5V/2A to prevent brownouts)

Pin Mapping Table

The INMP441 uses the I2S protocol, which requires a Word Select (WS), Bit Clock (BCLK/SCK), and Serial Data (SD/DOUT) line. Note: The L/R pin determines the I2S channel. Tying it to GND selects the Left channel, which our code expects.

Wiring and I2S Configuration Steps

1. Prep the Mic: Solder a 6-pin breakaway header to the INMP441 breakout board. Ensure the solder joints are clean; the I2S clock is highly sensitive to capacitive loading from solder blobs.
2. Channel Strapping: Connect the L/R pad to the GND pad on the microphone module itself, or wire it directly to the ESP32 GND. If left floating, the mic will output garbage data on alternating clock cycles.
3. Connect I2S Lines: Wire WS to GPIO 4, SCK to GPIO 5, and SD to GPIO 6. Double-check these against the pin mapping table above.
4. Power Verification: Before plugging in the USB cable, use a multimeter in continuity mode to verify there is no short between the 3V3 and GND rails on your breadboard.
5. IDE Setup: In the Arduino IDE Boards Manager, install the esp32 board package by Espressif (v2.0.14 or newer). Select ESP32S3 Dev Module. Under the Tools menu, set PSRAM to OPI PSRAM and USB CDC On Boot to Enabled.

The Code: Capturing Audio and Streaming to Cloud STT

The following code initializes the I2S peripheral, records 3 seconds of 16kHz 16-bit mono audio into PSRAM, constructs a valid WAV header, and POSTs the payload to a cloud endpoint. It includes robust error handling for DMA allocation and WiFi connectivity.

#include <driver/i2s.h>
#include <WiFi.h>
#include <HTTPClient.h>

// --- USER CONFIGURATION ---
const char* ssid = "YOUR_WIFI_SSID";
const char* password = "YOUR_WIFI_PASSWORD";
// Replace with your actual Cloud STT API endpoint (e.g., Google Cloud, AWS Transcribe, or local Whisper server)
const char* apiEndpoint = "https://your-stt-webhook-url.com/transcribe"; 

// --- PIN DEFINITIONS ---
#define I2S_WS 4
#define I2S_SCK 5
#define I2S_SD 6
#define I2S_PORT I2S_NUM_0

// --- AUDIO PARAMETERS ---
#define SAMPLE_RATE 16000
#define SAMPLE_BITS 16
#define RECORD_SECONDS 3
#define BUFFER_LEN (SAMPLE_RATE * RECORD_SECONDS * (SAMPLE_BITS / 8))

// Allocate audio buffer in PSRAM
int16_t* audioBuffer = nullptr;

void setup() {
  Serial.begin(115200);
  delay(1000);
  Serial.println("Booting ESP32-S3 Speech-to-Text Node...");

  // 1. Initialize PSRAM
  if (!psramInit()) {
    Serial.println("FATAL: PSRAM initialization failed. Check Tools -> PSRAM setting.");
    while (1) delay(1000);
  }
  
  // Allocate buffer in PSRAM
  audioBuffer = (int16_t*) ps_malloc(BUFFER_LEN);
  if (!audioBuffer) {
    Serial.println("FATAL: Failed to allocate PSRAM audio buffer.");
    while (1) delay(1000);
  }

  // 2. Connect to WiFi
  WiFi.begin(ssid, password);
  Serial.print("Connecting to WiFi");
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  Serial.println("\nWiFi Connected.");

  // 3. Configure I2S
  i2s_config_t i2s_config = {
    .mode = (i2s_mode_t)(I2S_MODE_MASTER | I2S_MODE_RX),
    .sample_rate = SAMPLE_RATE,
    .bits_per_sample = I2S_BITS_PER_SAMPLE_16BIT,
    .channel_format = I2S_CHANNEL_FMT_ONLY_LEFT,
    .communication_format = I2S_COMM_FORMAT_STAND_I2S,
    .intr_alloc_flags = ESP_INTR_FLAG_LEVEL1,
    .dma_buf_count = 8,
    .dma_buf_len = 1024,
    .use_apll = false,
    .tx_desc_auto_clear = false,
    .fixed_mclk = 0
  };

  i2s_pin_config_t pin_config = {
    .bck_io_num = I2S_SCK,
    .ws_io_num = I2S_WS,
    .data_out_num = I2S_PIN_NO_CHANGE,
    .data_in_num = I2S_SD
  };

  esp_err_t err = i2s_driver_install(I2S_PORT, &i2s_config, 0, NULL);
  if (err != ESP_OK) {
    Serial.printf("FATAL: Failed installing I2S driver: %d\n", err);
    while (1) delay(1000);
  }
  i2s_set_pin(I2S_PORT, &pin_config);
  i2s_zero_dma_buffer(I2S_PORT);
  
  Serial.println("System Ready. Send any character via Serial to record.");
}

void loop() {
  if (Serial.available()) {
    Serial.read(); // Clear buffer
    recordAndTranscribe();
  }
}

void recordAndTranscribe() {
  Serial.println("Recording 3 seconds...");
  size_t bytes_read = 0;
  
  // Read I2S data into PSRAM buffer
  i2s_read(I2S_PORT, audioBuffer, BUFFER_LEN, &bytes_read, portMAX_DELAY);
  Serial.printf("Captured %d bytes of audio.\n", bytes_read);

  // Construct WAV Header (44 bytes)
  uint8_t wavHeader[44];
  buildWavHeader(wavHeader, bytes_read);

  // Combine Header and Audio Data for HTTP POST
  size_t totalPayloadSize = 44 + bytes_read;
  uint8_t* payload = (uint8_t*) ps_malloc(totalPayloadSize);
  if (!payload) {
    Serial.println("Error: Failed to allocate payload buffer.");
    return;
  }
  
  memcpy(payload, wavHeader, 44);
  memcpy(payload + 44, audioBuffer, bytes_read);

  // Send to Cloud API
  if (WiFi.status() == WL_CONNECTED) {
    HTTPClient http;
    http.begin(apiEndpoint);
    http.addHeader("Content-Type", "audio/wav");
    
    int httpCode = http.POST(payload, totalPayloadSize);
    if (httpCode > 0) {
      Serial.printf("HTTP Response Code: %d\n", httpCode);
      if (httpCode == HTTP_CODE_OK) {
        String response = http.getString();
        Serial.println("Transcription Result:");
        Serial.println(response);
      }
    } else {
      Serial.printf("HTTP POST failed, error: %s\n", http.errorToString(httpCode).c_str());
    }
    http.end();
  }
  
  free(payload);
  Serial.println("Ready for next recording.");
}

void buildWavHeader(uint8_t* header, size_t dataSize) {
  uint32_t sampleRate = SAMPLE_RATE;
  uint16_t numChannels = 1;
  uint16_t bitsPerSample = SAMPLE_BITS;
  uint32_t byteRate = sampleRate * numChannels * (bitsPerSample / 8);
  uint16_t blockAlign = numChannels * (bitsPerSample / 8);
  uint32_t chunkSize = 36 + dataSize;

  memcpy(header, "RIFF", 4);
  memcpy(header + 4, &chunkSize, 4);
  memcpy(header + 8, "WAVE", 4);
  memcpy(header + 12, "fmt ", 4);
  uint32_t subchunk1Size = 16;
  memcpy(header + 16, &subchunk1Size, 4);
  uint16_t audioFormat = 1; // PCM
  memcpy(header + 20, &audioFormat, 2);
  memcpy(header + 22, &numChannels, 2);
  memcpy(header + 24, &sampleRate, 4);
  memcpy(header + 28, &byteRate, 4);
  memcpy(header + 32, &blockAlign, 2);
  memcpy(header + 34, &bitsPerSample, 2);
  memcpy(header + 36, "data", 4);
  memcpy(header + 40, &dataSize, 4);
}

Debugging: Fixing the "I2S DMA malloc failed" Crash

When working with I2S audio on the ESP32, memory allocation is the most common point of failure. If your board enters a bootloop and you see the following exact error string in the Serial Monitor:

E (512) I2S: i2s_dma_malloc(142): I2S DMA malloc failed
Guru Meditation Error: Core 1 panic'ed (LoadProhibited). Exception was unhandled.

This means the ESP32's internal SRAM was exhausted while trying to create the I2S DMA (Direct Memory Access) buffers. The I2S driver requires DMA buffers to be placed in internal SRAM, not external PSRAM, for timing reasons. If internal SRAM is fragmented or full, the allocation fails, returning a null pointer that triggers the LoadProhibited panic.

The First Three Things to Check When It Fails

1. Verify PSRAM is Enabled in IDE: Go to Tools > PSRAM and ensure OPI PSRAM is selected. If it is set to "Disabled", the ps_malloc() function in our code will fail, and subsequent heap allocations will cannibalize the internal SRAM needed by the I2S driver.
2. Reduce DMA Buffer Count: In the i2s_config_t struct, look at .dma_buf_count = 8. If you have other libraries consuming internal RAM (like heavy TLS stacks for WiFi), drop this to 4 and increase .dma_buf_len to 2048 to maintain the same total buffer size while using fewer allocation blocks.
3. Check the L/R Pin Strapping: While a floating L/R pin won't cause a DMA crash, it will cause the I2S clock to desync, resulting in a buffer filled with zeros or deafening static, which the cloud API will reject with a 400 Bad Request error. Ensure L/R is firmly tied to GND.

Extending and Simplifying the Build

Depending on your project goals, you may want to pivot away from cloud-dependent STT or add two-way communication.

How to Simplify: Go Offline with UART

If you do not need full dictation and only want to recognize 10 to 50 specific voice commands (e.g., "Turn on lights", "Open garage"), drop the ESP32 and INMP441 entirely. Instead, use the DFRobot Gravity: Offline Voice Recognition Sensor (SEN0539).

• Why it wins: It requires zero WiFi, zero cloud APIs, and handles all acoustic modeling on its onboard ASIC.
• How it works: You train wake words and commands via a USB serial interface. In your Arduino code, you simply read a single byte over Hardware Serial (UART) representing the command ID.
• Trade-off: It cannot transcribe arbitrary speech; it is strictly a command-matching engine.

How to Extend: Build a Two-Way Voice Assistant

To turn this STT node into a full voice assistant, you need audio output. Because the ESP32-S3 I2S peripheral supports full-duplex operation (or you can use the second I2S port, I2S_NUM_1), you can add a MAX98357A I2S DAC/Amplifier module.

1. Wire the MAX98357A to a separate set of I2S pins (e.g., BCLK to GPIO 15, LRC to GPIO 16, DIN to GPIO 17).
2. Use the ESP8266Audio library (which supports ESP32) to stream MP3 or WAV responses from a Text-to-Speech (TTS) API directly to the amplifier.
3. Implement a push-to-talk button on GPIO 0 to prevent the microphone from picking up the speaker's output (acoustic echo cancellation is computationally heavy and best avoided in simple DIY builds).