How to Create Firmware for ESP32: Partitioning and Flash Limits

Firmware is the low-level software programmed directly into a microcontroller’s non-volatile flash memory that dictates how its hardware peripherals, logic gates, and pins behave. It is the permanent code that bridges the gap between raw silicon and functional electronics. When you create firmware, you are fundamentally changing a dead $4 ESP32-WROOM-32E module into a device capable of reading I2C sensors, driving PWM motor controllers, and transmitting MQTT telemetry over Wi-Fi. Beginners frequently confuse firmware with high-level application software (which runs on top of an operating system like Linux or Windows) or with the bootloader (which is merely a tiny, temporary stub program used to load the main firmware into memory).

Where You Meet Firmware in Practice

You interact with firmware creation every time you hit the 'Upload' button in the Arduino IDE or PlatformIO. However, you are not simply copying files to a drive like you would with a USB stick. You are invoking a toolchain (like Xtensa GCC for the ESP32) that compiles your C++ code into raw machine instructions, links it against the ESP-IDF or Arduino core libraries, and uses a utility like esptool.py to write those bytes to specific hexadecimal addresses in the SPI flash chip.

In practice, creating firmware means managing three distinct layers: the hardware abstraction layer (HAL) provided by the silicon vendor, the core framework (like Arduino or ESP-IDF), and your application logic. Understanding how these layers consume your limited flash memory is the difference between a reliable IoT node and a bricked prototype.

The Math of Flash: A Worked Partition Example

Unlike a PC with a massive 1TB NVMe drive, a standard ESP32-WROOM-32E module typically ships with just 4MB (4,194,304 bytes) of external SPI flash. When you create firmware, you must divide this space into partitions. If you miscalculate, your code will overwrite critical system data.

Let us look at a real-world numeric breakdown of a standard 4MB flash layout using the default.csv partition scheme provided by Espressif. According to the official Espressif Partition Table Documentation, the memory is sliced as follows:

Partition Name	Type	Hex Offset	Size (Bytes)	Size (MB)	Purpose
nvs	data	0x9000	20,480	0.019	Stores Wi-Fi credentials and calibration data
otadata	data	0xe000	8,192	0.007	Tracks which OTA partition is currently active
app0 (OTA_0)	app	0x10000	1,966,080	1.875	Primary firmware binary
app1 (OTA_1)	app	0x1F0000	1,966,080	1.875	Secondary firmware binary for updates
spiffs	data	0x3D0000	196,608	0.187	File system for HTML, JSON, or logs

Notice that out of 4MB, you only get 1.87MB for your actual application code when Over-The-Air (OTA) updates are enabled. This is because the device needs a second, equally sized partition (OTA_1) to download the new firmware safely before rebooting and swapping the active boot flag. If you are building a simple sensor node without OTA, you can switch to a no_ota.csv scheme, which merges those two partitions and gives your firmware a massive 3.1MB of contiguous space.

Scenario Walkthrough: The OTA Bootloop Trap

To understand why partition math matters, let us walk through a real-world failure that happens frequently on the workbench when engineers create firmware for commercial IoT deployments.

The Setup: A developer is building a custom air quality monitor using an ESP32-S3-WROOM-1 (4MB flash). The device features a 2.4-inch TFT display and needs to push data to AWS IoT Core. They start with a basic blink sketch, verify OTA works, and then begin adding heavy libraries: TFT_eSPI for the screen, LVGL for the graphical interface, and AWSIoT for secure MQTT.

The Numbers: The base firmware with Wi-Fi and basic sensors compiles to 1.1MB. Adding the LVGL graphics library and custom TrueType fonts pushes the compiled binary size to 2.05MB. The active OTA partition limit on their 4MB board is 1.87MB.

The Outcome: The developer hits 'Upload' via the Arduino IDE over a USB cable. Because USB uploads bypass the OTA partition limits and write directly to the absolute flash addresses, the IDE forces the 2.05MB binary onto the chip, silently overwriting the beginning of the OTA_1 and SPIFFS partitions. The device boots and runs fine. Two weeks later, the developer pushes a minor bug fix via the web interface (OTA). The OTA process downloads the new 2.05MB binary into the 1.87MB OTA_1 partition. It runs out of space, truncates the final 180KB of the binary, marks the partition as 'valid', and reboots.

What Went Wrong: Upon reboot, the ESP32 attempts to execute the firmware in OTA_1. When the CPU's instruction fetcher hits the truncated boundary at 1.87MB, it reads garbage data (likely the SPIFFS file system header) as machine code. The serial monitor immediately outputs:

Guru Meditation Error: Core 1 panic'ed (InstrFetchProhibited). Exception was unhandled.

The device enters an infinite bootloop. The fix requires physically connecting the board via USB and flashing a stripped-down firmware, or switching to an ESP32-S3 module with 8MB or 16MB of flash to accommodate the heavy GUI libraries.

Step-by-Step: Configuring Your Build Environment

To prevent partition mismatches when you create firmware, you must explicitly define your memory layout in your build configuration. Here is how to do it using PlatformIO, which offers far more control than the Arduino IDE.

1. Define the Board and Framework: Open your platformio.ini file. Set your board (e.g., esp32dev) and framework (arduino). This tells the compiler which hardware abstraction layer to link against.
2. Select a Partition Scheme: Add the line board_build.partitions = min_spiffs.csv. This specific scheme sacrifices the SPIFFS file system size to maximize your OTA app partitions, giving you roughly 1.9MB per app slot on a 4MB board. For the exact definitions of built-in CSV files, refer to the PlatformIO Espressif 32 Documentation.
3. Monitor Binary Size on Compile: Run pio run. Look at the terminal output for the 'RAM' and 'Flash' usage bars. If Flash usage exceeds 100% of the defined app partition, the build will halt with an error before it bricks your device.
4. Enable Verbose Uploads for Debugging: If an upload fails, add upload_flags = -v to your environment. This forces esptool.py to print the exact hex addresses it is writing to, allowing you to verify it is targeting the correct partition offset.
5. Erase Flash Before Major Changes: If you switch from a 4MB to an 8MB partition scheme, the old partition table remains in memory. Always run pio run --target erase_flash before uploading the new firmware to wipe the slate clean and prevent ghost partition conflicts.

Frequently Asked Questions