Debugging Raspberry Pi sshd: Headless Setup, GPIO Monitoring, and Connection Fixes

Deploying a Raspberry Pi as a headless embedded node—whether for remote environmental logging or an off-grid MQTT gateway—lives and dies by the reliability of its SSH daemon (sshd). When you strip away the monitor and keyboard, sshd is your only lifeline. Yet, default Raspberry Pi OS configurations prioritize desktop convenience over embedded resilience, leading to dropped connections, boot-time lockouts, and security vulnerabilities.

The direct answer for a production-ready headless Pi: disable password authentication entirely, enforce Ed25519 public key authentication, disable Wi-Fi power management to prevent sleep-state dropouts, and use a GPIO-driven Python watchdog to visually confirm daemon health on the bench. This guide walks through the exact hardware mapping, systemd hardening, and the specific error strings that trip up embedded developers.

Hardware Spec Sheet and GPIO Pin Mapping

Before touching the software, we need a physical feedback mechanism. When a headless Pi drops off the network, you need to know if the board is frozen, if the network is down, or if sshd specifically has crashed. We will wire a status LED and an I2C sensor to verify both daemon health and bus integrity.

Physical Pin Mapping (40-Pin Header)

The Raspberry Pi 5 maintains backward compatibility with the standard I2C bus on the primary header. Ensure your I2C lines have pull-up resistors (the Adafruit BME280 breakout includes these onboard).

The Authentication Decision Path

Leaving password authentication enabled on an internet-facing or IoT-network Pi is a guaranteed way to get your SSH port hammered by brute-force bots. You must choose an authentication method. Here is the decision matrix for embedded deployments.

Hardening sshd_config for Embedded Nodes

Once your Ed25519 key is installed, lock down the daemon. On Raspberry Pi OS, the configuration file is located at /etc/ssh/sshd_config. Open it with sudo nano /etc/ssh/sshd_config and enforce these exact directives:

1. Disable Passwords: Set PasswordAuthentication no and ChallengeResponseAuthentication no.
2. Enforce Key Auth: Set PubkeyAuthentication yes.
3. Disable Root Login: Set PermitRootLogin no. Always log in as the standard user and sudo when needed.
4. Keep-Alive Configuration: Embedded nodes often sit behind NAT routers that drop idle TCP connections. Add these lines to force the server to send keep-alive packets:

   ClientAliveInterval 60
   ClientAliveCountMax 3

After saving, restart the daemon and verify it binds correctly:

sudo systemctl restart ssh
sudo systemctl status ssh

Debugging sshd: Exact Error Strings and Ranked Causes

When your headless Pi refuses connections, the terminal output tells you exactly what failed. Here are the three most common exact error strings and how to fix them, ranked by frequency in embedded deployments.

1. "ssh: connect to host 10.0.0.5 port 22: Connection refused"

What it means: The Pi is on the network and responding to ICMP (ping), but nothing is listening on TCP port 22.

• Cause A (Most Likely): The ssh service is disabled. On headless first-boot, Raspberry Pi OS disables SSH by default. Fix: Place an empty file named ssh (no extension) in the /boot/firmware/ partition of the SD card, or run sudo raspi-config to enable it.
• Cause B: The Pi is still booting. The network stack comes up before sshd finishes generating host keys on first boot. Fix: Wait 45 seconds and retry.
• Cause C: iptables or ufw is blocking port 22. Fix: Run sudo ufw allow 22/tcp.

2. "Permission denied (publickey)"

What it means: The daemon is running, but it rejected your cryptographic handshake.

• Cause A (Most Likely): Directory permissions are too open. OpenSSH strictly enforces permissions. Fix: Run chmod 700 ~/.ssh and chmod 600 ~/.ssh/authorized_keys on the Pi.
• Cause B: You are offering the wrong key. Fix: Run SSH with verbose output (ssh -vvv pi@10.0.0.5) to see which keys your host machine is attempting to present.
• Cause C: The authorized_keys file is owned by root instead of the pi user (common if you used sudo to copy the file). Fix: sudo chown pi:pi ~/.ssh/authorized_keys.

3. "Connection timed out" or Intermittent Drops

What it means: The TCP handshake never completed, or an established session was silently killed.

• Cause A (Most Likely): Wi-Fi power management is putting the radio to sleep. Fix: Disable power save via sudo iwconfig wlan0 power off. To make it persistent, add it to a systemd service or /etc/network/interfaces.d/ script.
• Cause B: The Pi browned out under load, resetting the Wi-Fi chip. Fix: Verify you are using the official 27W USB-C PD supply, not a generic phone charger.

Python Health Monitor: Automating sshd and GPIO Status

To close the loop on our hardware setup, we need a script that runs in the background. This Python script uses the gpiozero library (the modern standard for Pi 5, replacing the deprecated RPi.GPIO) to monitor the ssh systemd service. If the daemon is active, the LED on GPIO 17 breathes (pulses). If it crashes, the LED turns off. If the script encounters an I2C bus error, it blinks rapidly.

Target Board: Raspberry Pi 5 (4GB) running Raspberry Pi OS Lite (64-bit).

#!/usr/bin/env python3
"""
sshd_gpio_monitor.py
Monitors Raspberry Pi SSH daemon status and I2C bus health.
Flashes GPIO 17 LED based on system state.
"""

import subprocess
import time
import sys
import math
from gpiozero import PWMLED

# --- PIN DEFINITIONS ---
STATUS_LED_PIN = 17  # Physical Pin 11

# Initialize LED with PWM for breathing effect
led = PWMLED(STATUS_LED_PIN)

def check_sshd_active():
    """Checks if the ssh systemd service is active."""
    try:
        # On modern Pi OS, the service is named 'ssh', not 'sshd'
        result = subprocess.run(
            ['systemctl', 'is-active', 'ssh'],
            capture_output=True, text=True, check=False
        )
        return result.stdout.strip() == 'active'
    except Exception as e:
        print(f"[ERROR] Failed to query systemctl: {e}", file=sys.stderr)
        return False

def check_i2c_bus():
    """Pings the I2C bus to ensure hardware connectivity."""
    try:
        result = subprocess.run(
            ['i2cdetect', '-y', '1'],
            capture_output=True, text=True, check=False
        )
        # BME280 typically shows up at 0x77 or 0x76
        return '77' in result.stdout or '76' in result.stdout
    except Exception:
        return False

def breathe_led(duration=2.0, steps=50):
    """Creates a smooth breathing effect on the PWM LED."""
    for i in range(steps):
        # Sine wave mapped to 0.0 - 1.0 for smooth PWM transition
        brightness = (math.sin(math.pi * i / steps - math.pi / 2) + 1) / 2
        led.value = brightness
        time.sleep(duration / steps)

def panic_blink(count=5):
    """Rapid blink for hardware/I2C failure."""
    for _ in range(count):
        led.on()
        time.sleep(0.1)
        led.off()
        time.sleep(0.1)

def main():
    print("Starting sshd and I2C health monitor...")
    try:
        while True:
            ssh_ok = check_sshd_active()
            i2c_ok = check_i2c_bus()

            if not i2c_ok:
                # Hardware fault takes priority
                print("[WARN] I2C Bus fault detected.")
                panic_blink()
            elif ssh_ok:
                # SSH is healthy, breathe the LED
                breathe_led(duration=2.0)
            else:
                # SSH is down, LED off
                led.off()
                time.sleep(2.0)
                
    except KeyboardInterrupt:
        print("\nMonitor stopped by user.")
    finally:
        led.off()
        print("GPIO cleaned up. Exiting.")

if __name__ == "__main__":
    main()

Extending the Build: Remote Telemetry and Socket Activation

Once your baseline sshd connection is hardened and monitored via GPIO, you have two distinct paths for scaling this embedded node.

To Simplify (Reduce Attack Surface): Instead of running sshd continuously in the background, configure systemd socket activation. This keeps port 22 closed and the daemon entirely unloaded from RAM until a connection request actually hits the network interface. To do this, disable the main service (sudo systemctl disable ssh) and enable the socket (sudo systemctl enable ssh.socket). This reduces idle RAM usage and hides the service from casual port scanners.

To Extend (Add Sensor Telemetry): Expand the Python script to read the BME280 sensor data using the adafruit-circuitpython-bme280 library. Instead of just monitoring SSH, wrap the script in a FastAPI web server or push the telemetry to an MQTT broker (like Mosquitto) over port 8883. You can then use your hardened SSH connection strictly for out-of-band management and firmware updates, while the sensor data flows over a separate, isolated protocol.

For deep-dive configuration parameters, always refer to the official OpenSSH manual and the Raspberry Pi OS configuration documentation to ensure your daemon settings align with the latest kernel networking stacks.