ESP32 Unzip Guide: Extracting ZIP Files to SD Card Without Memory Panics

The most reliable way to handle an esp32 unzip operation is using the native miniz.h library built directly into the ESP32 Arduino core, streaming the decompressed chunks to an SD card via a callback function. This bypasses the ESP32's strict contiguous RAM limits and prevents the heap fragmentation panics that plague most third-party ZIP libraries.

Most tutorials instruct you to load the entire unzipped file into memory using mz_zip_reader_extract_to_heap. On an ESP32-WROOM-32 with only 520KB of SRAM, this guarantees a crash on any file larger than ~150KB. By targeting the ESP32-WROOM-32 DevKit V1 and streaming directly to storage, we can extract multi-megabyte archives safely.

Hardware Spec Sheet and Pin Mapping

Before writing code, we need to establish the physical layer. SPI communication to SD cards is highly sensitive to wire length and voltage levels. The ESP32 operates at 3.3V logic, while many cheap SD breakout boards are designed for 5V Arduino Unos.

SPI Pin Mapping Table

Why Native Miniz Beats Third-Party Libraries

When searching for an ESP32 unzip solution, you will find libraries like ArduinoZip or ESP32-targz. While useful, they add flash overhead and often abstract away the memory management that causes crashes. The miniz library is already compiled into the ESP-IDF (Espressif IoT Development Framework) that underlies the Arduino core. By simply including #include "miniz.h", you access a highly optimized, zero-overhead C library without downloading a single third-party package.

The critical trick is using mz_zip_reader_extract_to_callback. Instead of asking the ESP32 to allocate a massive contiguous block of RAM for the unzipped file, this function decompresses the archive in small chunks (usually 4KB to 64KB) and passes each chunk to a callback function. Our callback simply writes that chunk directly to the SD card, keeping RAM usage flat and predictable.

Complete Streaming Extraction Code

This code targets the ESP32-WROOM-32 DevKit V1. It mounts the SD card, opens a ZIP file named payload.zip located in the root directory, and extracts the first file inside it to extracted.txt using the streaming callback method.

#include <Arduino.h>
#include <SD.h>
#include <SPI.h>
#include "miniz.h"

// --- PIN DEFINITIONS ---
#define SD_CS_PIN   5
#define SD_MOSI_PIN 23
#define SD_MISO_PIN 19
#define SD_SCK_PIN  18

// --- CALLBACK FUNCTION ---
// Streams decompressed chunks directly to the SD card File object
size_t sd_write_callback(void *pOpaque, mz_uint64 file_ofs, const void *pBuf, size_t n) {
  File* out_file = (File*)pOpaque;
  
  // Ensure we are at the correct position (handles sparse files)
  if (file_ofs != out_file->position()) {
    out_file->seek(file_ofs);
  }
  
  size_t bytes_written = out_file->write((const uint8_t*)pBuf, n);
  if (bytes_written != n) {
    Serial.println("[ERROR] SD write failed or disk full!");
  }
  return bytes_written;
}

void setup() {
  Serial.begin(115200);
  delay(1000); // Allow serial monitor to connect
  Serial.println("\n--- ESP32 Miniz Streaming Unzip ---");

  // 1. Initialize SPI and SD Card
  SPI.begin(SD_SCK_PIN, SD_MISO_PIN, SD_MOSI_PIN, SD_CS_PIN);
  if (!SD.begin(SD_CS_PIN)) {
    Serial.println("[FATAL] SD Card mount failed. Check wiring and FAT32 format.");
    return;
  }
  Serial.println("SD Card mounted successfully.");

  // 2. Initialize Miniz Archive Reader
  mz_zip_archive zip_archive;
  memset(&zip_archive, 0, sizeof(zip_archive));
  
  const char* zip_filename = "/payload.zip";
  if (!mz_zip_reader_init_file(&zip_archive, zip_filename, 0)) {
    Serial.printf("[FATAL] Failed to open %s. File missing or corrupt.\n", zip_filename);
    return;
  }

  // 3. Iterate and Extract Files
  mz_uint num_files = mz_zip_reader_get_num_files(&zip_archive);
  Serial.printf("Found %d files in archive.\n", num_files);

  for (mz_uint i = 0; i < num_files; i++) {
    mz_zip_archive_file_stat file_stat;
    if (!mz_zip_reader_file_stat(&zip_archive, i, &file_stat)) {
      Serial.println("[ERROR] Could not read file stat.");
      continue;
    }

    // Skip directories
    if (mz_zip_reader_is_file_a_directory(&zip_archive, i)) continue;

    Serial.printf("Extracting: %s (%lu bytes)\n", file_stat.m_filename, (unsigned long)file_stat.m_uncomp_size);

    // Open destination file on SD card
    File out_file = SD.open("/extracted_" + String(i) + ".bin", FILE_WRITE);
    if (!out_file) {
      Serial.println("[ERROR] Failed to create destination file on SD.");
      continue;
    }

    // 4. STREAM EXTRACTION (The Magic Step)
    // Extracts in chunks, passing data to sd_write_callback
    mz_bool status = mz_zip_reader_extract_to_callback(
      &zip_archive, 
      i, 
      sd_write_callback, 
      &out_file, 
      0
    );

    out_file.close();

    if (!status) {
      Serial.println("[ERROR] Extraction failed! Archive may be corrupt.");
    } else {
      Serial.println(" -> Success.");
    }
  }

  // 5. Cleanup
  mz_zip_reader_end(&zip_archive);
  Serial.println("Unzip process complete.");
}

void loop() {
  // Nothing to do here
}

Troubleshooting: Ranked Causes for Unzip Failures

When working with file systems and decompression on microcontrollers, failures are rarely random. They follow specific hardware and memory patterns. Here are the exact error strings you will encounter and how to fix them.

1. Guru Meditation Error: Core 1 panic'ed (StoreProhibited)

Exact Error String: Guru Meditation Error: Core 1 panic'ed (StoreProhibited). Exception was unhandled.

Cause: You used mz_zip_reader_extract_to_heap or mz_zip_reader_extract_to_mem on a file larger than your available contiguous RAM. The ESP32 tried to allocate a massive block, failed, and dereferenced a null pointer.

Fix: Switch to the mz_zip_reader_extract_to_callback streaming method provided in the code above. If you absolutely must use heap extraction, ensure your target file is under 100KB and call heap_caps_malloc(size, MALLOC_CAP_8BIT) to check for available contiguous blocks first.

2. mz_zip_reader_init_file failed

Exact Error String: [FATAL] Failed to open /payload.zip. File missing or corrupt. (Generated by our custom serial print, triggered when the miniz function returns MZ_FALSE).

Cause: Miniz cannot parse the ZIP header. This happens if the ZIP was created using a non-standard compression method (like LZMA or AES encryption) which miniz does not support, or if the file on the SD card is 0 bytes due to a failed download.

Fix: Re-create your ZIP archive using standard Deflate compression (the default in Windows/macOS and 7-Zip). Verify the file size on the SD card via SD.open() before passing it to miniz.

3. sdmmc_send_cmd returned 0x107

Exact Error String: E (412) sdmmc_cmd: sdmmc_read_sectors: sdmmc_send_cmd returned 0x107

Cause: SPI bus timeout. The ESP32 sent a clock signal but the SD card controller did not respond in time. This is almost always a physical wiring issue or a brownout on the SD card's VCC line during heavy write operations.

Fix: Add a 100µF decoupling capacitor across the VCC and GND pins of the SD card module. Ensure your USB power supply can deliver at least 1A; SD card write spikes can pull 300mA+ momentarily.

Extending and Simplifying the Build

How to Simplify: If you are building a product and want to eliminate the SD card entirely, upgrade your hardware to an ESP32-S3-WROOM-1 (N8R8). The S3 variant includes 8MB of octal PSRAM. By initializing PSRAM (psramInit()) and configuring the Arduino IDE to use "OPI PSRAM", you can safely use mz_zip_reader_extract_to_heap for files up to 4MB, as the heap allocator will pull from the massive external RAM pool instead of the constrained internal SRAM.

How to Extend: The ultimate IoT pattern is downloading a ZIP over Wi-Fi and unzipping it without ever saving the ZIP to storage. You can achieve this by combining the HTTPClient library with miniz's memory-reading functions. Instead of mz_zip_reader_init_file, you use mz_zip_reader_init_mem. You stream the HTTP response into a circular PSRAM buffer, and miniz reads directly from that buffer, streaming the decompressed output to LittleFS or an external SPI flash chip. This reduces flash wear and cuts the total update time in half.

ESP32 Unzip FAQ

Can I unzip files directly to ESP32 LittleFS without an SD card?

Yes, but with strict size limitations. LittleFS on a standard 4MB ESP32 usually has a maximum partition size of 1.5MB to 2MB. Because LittleFS wears down flash memory with every write, unzipping large, frequently updated archives directly to LittleFS will degrade the flash chip over time. Use the SD card method for large payloads, and reserve LittleFS for small configuration files or web assets (HTML/CSS/JS) extracted from a single, infrequently updated ZIP.

Why does my ESP32 crash when unzipping a 500KB file?

A 500KB compressed file might decompress to 2MB or more. The ESP32-WROOM-32 only has ~520KB of internal SRAM, and much of that is consumed by the Wi-Fi stack, Bluetooth stack, and FreeRTOS overhead. If your code attempts to hold the decompressed file in RAM, it exceeds physical memory limits, causing a StoreProhibited panic. You must use the streaming callback method shown above to process the file in 4KB chunks.

How do I download and unzip a file via HTTP without saving the ZIP first?

You need an ESP32 with PSRAM (like the ESP32-S3 or ESP32-WROVER). You allocate a buffer in PSRAM using heap_caps_malloc(size, MALLOC_CAP_SPIRAM), stream the HTTPClient response into that buffer, and then pass the buffer pointer to mz_zip_reader_init_mem(). From there, you can use the same callback function to write the extracted chunks to LittleFS or an SD card.

Does the ESP32 miniz library support password-protected ZIP files?

No. The version of miniz bundled with the ESP-IDF and Arduino core is a lightweight, public-domain implementation that strictly supports standard Deflate and uncompressed ZIP formats. It does not include the cryptographic modules required for ZipCrypto or AES-encrypted archives. If you require encrypted archives, you must implement a custom decryption layer or use a heavier library like mbedTLS alongside a full zlib port, which will consume significantly more flash memory.