Dual-WAN Raspberry Pi Routing: Build a Failover Gateway with CM4

Turning a Raspberry Pi into a dedicated network router is a rite of passage for advanced makers. While a standard Pi 4 or 5 can handle basic packet forwarding via USB adapters, serious raspberry pi routing demands hardware-level reliability, dedicated PHYs, and deterministic failover. By stepping up to the Compute Module 4 (CM4) ecosystem, you gain access to native PCIe lanes and dual high-density board-to-board connectors, allowing you to build a true dual-WAN edge router capable of Gigabit throughput and automated ISP failover.

Project Spec Sheet & Difficulty Rating

Hardware BOM & CM4 Pin Mapping

The CM4 routes its high-speed interfaces through two 40-pin high-density connectors (J1 and J2). The Waveshare CM4-DUAL-ETH-BASE utilizes the native PCIe Gen 2.0 single-lane interface on J2 for the primary Gigabit Ethernet port (RTL8111G PHY), while the secondary port relies on a USB 3.0 hub controller. Understanding the physical pin mapping is critical if you need to debug link-state issues or design a custom carrier board later.

Note: Pinout references the official Raspberry Pi CM4 Datasheet. Always verify against your specific carrier board schematic.

Base OS & IP Forwarding Configuration

Before writing automation scripts, the Linux kernel must be instructed to forward packets between interfaces and perform Network Address Translation (NAT). Raspberry Pi OS Bookworm uses nftables as the backend for firewall rules, deprecating legacy iptables commands.

1. Enable IP Forwarding: Edit /etc/sysctl.conf and uncomment or add net.ipv4.ip_forward=1. Apply immediately with sudo sysctl -p.
2. Identify Interfaces: Run ip link. Identify your primary WAN (e.g., eth0 via PCIe) and secondary WAN (e.g., eth1 via USB), plus your LAN interface (e.g., wlan0 or a dedicated VLAN).
3. Configure NAT via nftables: Create a base NAT rule so LAN clients can route out through whichever WAN is currently active.

   sudo nft add table nat
   sudo nft add chain nat postrouting { type nat hook postrouting priority 100 \; }
   sudo nft add rule nat postrouting oifname "eth0" masquerade
   sudo nft add rule nat postrouting oifname "eth1" masquerade

4. Persist Rules: Install the persistence daemon via sudo apt install nftables and ensure the service is enabled. Save your ruleset with sudo nft list ruleset > /etc/nftables.conf.

Python WAN Failover Routing Script

This script targets the Raspberry Pi Compute Module 4 (CM4) running Bookworm. It monitors the primary gateway via ICMP. If the primary fails, it dynamically alters the kernel routing table to shift the default route to the secondary interface. It includes robust error handling for subprocess calls.

#!/usr/bin/env python3
"""
Dual-WAN Failover Router for Raspberry Pi CM4
Targets: Raspberry Pi OS Bookworm 64-bit
Dependencies: None (uses standard library subprocess and socket)
"""

import subprocess
import time
import logging
import socket

logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(levelname)s - %(message)s')

# --- INTERFACE & GATEWAY DEFINITIONS ---
WAN_PRIMARY = "eth0"      # PCIe RTL8111G
WAN_SECONDARY = "eth1"    # USB 3.0 RTL8153
GW_PRIMARY = "192.168.1.1"   # ISP 1 Gateway
GW_SECONDARY = "192.168.2.1" # ISP 2 / 5G Backup Gateway
PING_TARGET = "1.1.1.1"      # Cloudflare DNS for reachability test
CHECK_INTERVAL = 10          # Seconds between health checks

def is_reachable(interface: str, target: str) -> bool:
    """Pings a target IP strictly bound to a specific source interface."""
    try:
        result = subprocess.run(
            ["ping", "-c", "1", "-W", "2", "-I", interface, target],
            capture_output=True, text=True, timeout=5
        )
        return result.returncode == 0
    except subprocess.TimeoutExpired:
        return False

def get_current_default_route():
    """Parses 'ip route' to find the current default gateway interface."""
    result = subprocess.run(["ip", "route", "show", "default"], capture_output=True, text=True)
    for line in result.stdout.splitlines():
        if line.startswith("default"):
            parts = line.split()
            if "dev" in parts:
                return parts[parts.index("dev") + 1]
    return None

def set_default_route(gateway: str, interface: str):
    """Adds a default route, handling specific RTNETLINK errors."""
    # First, flush existing default routes to prevent duplicates
    subprocess.run(["ip", "route", "flush", "exact", "0.0.0.0/0"], capture_output=True)
    
    cmd = ["ip", "route", "add", "default", "via", gateway, "dev", interface]
    result = subprocess.run(cmd, capture_output=True, text=True)
    
    if result.returncode != 0:
        err_msg = result.stderr.strip()
        logging.error(f"Failed to set route via {interface}: {err_msg}")
        if "RTNETLINK answers: Network is unreachable" in err_msg:
            logging.critical("Gateway is not on the local subnet or interface is DOWN.")
        return False
    
    logging.info(f"Successfully routed default traffic via {interface} ({gateway})")
    return True

def main():
    logging.info(f"Starting Dual-WAN Monitor on CM4. Primary: {WAN_PRIMARY}, Secondary: {WAN_SECONDARY}")
    active_wan = WAN_PRIMARY
    set_default_route(GW_PRIMARY, WAN_PRIMARY)

    while True:
        if not is_reachable(active_wan, PING_TARGET):
            logging.warning(f"Primary WAN ({active_wan}) lost connectivity.")
            if active_wan == WAN_PRIMARY and is_reachable(WAN_SECONDARY, PING_TARGET):
                logging.info("Failing over to Secondary WAN...")
                if set_default_route(GW_SECONDARY, WAN_SECONDARY):
                    active_wan = WAN_SECONDARY
            elif active_wan == WAN_SECONDARY:
                logging.error("Both WAN links are DOWN. Retrying primary...")
                if is_reachable(WAN_PRIMARY, PING_TARGET):
                    if set_default_route(GW_PRIMARY, WAN_PRIMARY):
                        active_wan = WAN_PRIMARY
        else:
            # If we are on secondary, but primary comes back, failback
            if active_wan == WAN_SECONDARY and is_reachable(WAN_PRIMARY, PING_TARGET):
                logging.info("Primary WAN restored. Failing back...")
                if set_default_route(GW_PRIMARY, WAN_PRIMARY):
                    active_wan = WAN_PRIMARY

        time.sleep(CHECK_INTERVAL)

if __name__ == "__main__":
    main()

Debugging: "RTNETLINK answers: Network is unreachable"

When manipulating the Linux routing table via ip route or Python's subprocess, the most common fatal error you will encounter is:

RTNETLINK answers: Network is unreachable

This is not a generic "internet is down" message. It is a strict kernel-level rejection of your routing command. Here are the ranked causes and the first three things to check when it fails:

1. The Interface Link State is DOWN (Most Common): The kernel will refuse to route traffic out of an interface that hasn't negotiated a physical link.
   Fix: Run ip link show eth1. If it says state DOWN, check your Cat6 cable, the switch port, and ensure the RTL8153 USB driver hasn't crashed (check dmesg | grep rtl8152).
2. Gateway IP is Outside the Interface Subnet: You cannot set a default gateway via an interface if the gateway's IP address does not fall within the IP subnet assigned to that interface.
   Fix: Run ip addr show eth1. If your IP is 10.0.0.5/24 but you are trying to route via 192.168.2.1, the kernel throws this error. Ensure DHCP has properly assigned an IP before the script attempts to route.
3. Missing ARP Resolution: Even if the subnet matches, if the kernel cannot resolve the MAC address of the gateway via ARP (e.g., the gateway is blocking ICMP/ARP or is completely powered off), the route addition may fail or immediately drop packets.

Extending and Simplifying the Build

Not every project requires a Compute Module 4. Here is how to scale this architecture based on your actual throughput needs and budget.

How to Simplify (The Pi 4 / Pi 5 Route)

If you don't need the PCIe pin-mapping complexity of the CM4, you can simplify the build by using a standard Raspberry Pi 4 Model B (4GB) or Raspberry Pi 5 paired with a Waveshare USB 3.0 to Dual Gigabit Ethernet Adapter.

• Pros: No high-density connectors to seat; standard microSD boot; lower total cost (~$85).
• Cons: Both NICs share the single USB 3.0 host controller bus. Maximum aggregate throughput is capped around 700-800 Mbps, meaning you cannot pull a full Gigabit on WAN while pushing a full Gigabit on LAN simultaneously.

How to Extend (Enterprise Features)

To push this CM4 router into production-grade territory:

• Add Intrusion Detection: Install Suricata to inspect mirrored traffic on the LAN bridge.
• Implement WireGuard: Use the secondary WAN exclusively for an encrypted WireGuard tunnel to a remote VPS, bypassing local ISP CGNAT restrictions.
• Add NVMe Storage: If using a baseboard with an M.2 slot (sharing the PCIe bus), you can boot the OS from an NVMe drive for extreme logging durability, though this requires a baseboard with a PCIe switch chip since the CM4 only has one native PCIe lane.

Raspberry Pi Routing FAQ

Can I use Raspberry Pi routing for a full home network replacing my ISP router?

Yes, but with caveats. A CM4-based router can easily handle NAT and routing for a typical 500 Mbps home connection. However, you lose the hardware-accelerated Wi-Fi routing found in commercial mesh systems. You will need to configure a separate Managed Access Point (like a Ubiquiti U6 or TP-Link Omada) and bridge it to the Pi's LAN interface. The Pi handles the DHCP, DNS (via Pi-hole), and WAN routing, while the AP handles the RF layer.

How do I force Raspberry Pi routing to prefer a 5G cellular backup over Ethernet?

Linux routing relies on the "metric" value; lower metrics are preferred. If your 5G router (connected to eth1) should be the primary, assign it a lower metric in your DHCP client config or via static IP setup. In the Python script provided above, you would simply swap the logic: set WAN_PRIMARY = "eth1" (5G) and WAN_SECONDARY = "eth0" (Cable/Fiber). Alternatively, use ip route add default via [5G_GW] dev eth1 metric 100 and metric 200 for the Ethernet link.

Why is my Raspberry Pi routing throughput capped at 300 Mbps instead of Gigabit?

If you are using a Pi 4 or Pi 5 with a USB-based dual-NIC adapter, you are likely hitting USB overhead or interrupt bottlenecks. First, ensure the adapter is plugged into a true USB 3.0 port (the blue ones). Second, check your CPU governor; if the Pi is idling at 600MHz, it cannot process software NAT fast enough. Install cpufrequtils and set the governor to performance. Finally, verify your nftables rules aren't forcing excessive logging, which stalls the network stack.

Does the CM4 support VLAN tagging (802.1Q) for network segmentation?

Absolutely. The RTL8111G PHY on the PCIe bus and the RTL8153 on the USB bus both support hardware VLAN offloading. You can create virtual interfaces (e.g., eth0.10 for IoT, eth0.20 for Cameras) using the vconfig or ip link add link eth0 name eth0.10 type vlan id 10 commands. This allows a single physical CM4 Ethernet port to route traffic for multiple isolated subnets, mimicking enterprise firewall behavior.