# Boot Partition Full Fix Lesson

Source: https://chatgpt.com/share/69f366fd-4fe8-83eb-adf7-b497e33c0b47

## Core Lesson

A small `/tmp` can break a kernel install even when `/` has free disk space.

In this incident, `/tmp` was a small loop-mounted filesystem, about 468 MB. During kernel installation, `initramfs` needed temporary working space and failed with:

```text
cpio: write error: No space left on device
```

The problem was not normal disk capacity. The problem was temporary workspace exhaustion.

## Why `/tmp` Matters

Many system tools use `/tmp` heavily:

- `initramfs`
- `apt` and `dpkg`
- package extraction
- compilers
- cPanel Elevate
- other upgrade tooling

Rule: on production servers, keep `/tmp` at least 1-2 GB, especially before OS upgrades, kernel updates, and cPanel Elevate runs.

## cPanel Trap

cPanel often uses `/usr/tmpDSK` mounted as `/tmp`.

That setup can be a small loop file. If it is too small, package operations can fail even though the main disk looks healthy.

Recommended permanent fixes:

```bash
/scripts/securetmp
```

Or use a larger `tmpfs` mount:

```bash
mount -t tmpfs -o size=2G tmpfs /tmp
```

For a permanent `tmpfs`, configure `/etc/fstab`.

## Upgrade Checklist

Before a major upgrade, check space and inodes:

```bash
df -h
df -h /tmp
df -i
```

Confirm:

- `/tmp` has at least 1 GB free.
- root disk has enough free space.
- inode usage is healthy.
- console access is available.

SSH is not enough. Network configuration can break or pause after an OS upgrade.

## Kernel Cleanup Rule

Do not manually delete kernel files from `/boot` unless there is no safer option.

Risky pattern:

```bash
rm -f /boot/initrd...
```

Safer pattern:

```bash
apt remove linux-image-...
```

Package-aware removal keeps `dpkg` state consistent and avoids leaving the system half-broken.

## dpkg Recovery Pattern

When kernel installation or `initramfs` generation fails:

1. Fix the root cause first, such as `/tmp` size.
2. Rebuild or regenerate the failed artifacts if needed.
3. Resume package configuration:

```bash
dpkg --configure -a
```

Do not repeatedly rerun package commands before fixing the real cause.

## After Upgrade Checks

Major upgrades can leave service or permission drift. Verify from system-level tools before trusting a control panel status page.

Useful checks:

```bash
systemctl --failed
systemctl status apache2
systemctl status mysql
systemctl status redis
```

Check permissions if services complain about paths like:

- `/var/run`
- `/etc/apache2`
- `/var/lib/mysql`

WHM or another dashboard can show stale or pending status while services are actually running.

## Do Not Interrupt Package Operations

Avoid pressing `Ctrl+C` during `apt` or `dpkg` unless you are certain it is safe. Interrupting package operations can leave the system in a broken or partially configured state.

## One-Line Takeaway

A small `/tmp` broke kernel installation, which broke `dpkg`, which broke the upgrade flow.

If you see `No space left on device`, check `/tmp`, not only `/`.