Boot partition full

[DonQuishot]

Hi All,

Problem Description
After the last software update my boot partition, vfat - mounted as /boot/firmware, size 270 MB, is full.

That last update included linux-image-6.1.0-22-arm64. I’ve had problems in the past before, where I had to do a clean re-install, say every 6 months, because of not being able to run any further software updates, or crashes of my FreedomBox. I fear of running into that again.

Is there anything I can/should do now to prevent this from happening?

My setup is:

Raspberry Pi 3 Model B Plus Rev 1.3, installed from downloaded image
running from sd card, 32 Gb,
second partition, btrfs, mounted as / 8 GB/ 32 GB in use
FreedomBox version 24.13, system is up to date
installed software Radical

[Avron]

If you can log in via ssh, try sudo apt autoremove to remove old kernels.

[njoseph]

@DonQuishot Is this an automatic update or a manual update?

[DonQuishot]

Tried doing that via the terminal, which resulted in

Reading package lists… Done
Building dependency tree… Done
Reading state information… Done
The following packages will be REMOVED:
linux-image-6.1.0-20-arm64
0 upgraded, 0 newly installed, 1 to remove and 0 not upgraded.
3 not fully installed or removed.
After this operation, 356 MB disk space will be freed.
Do you want to continue? [Y/n]
quota not working (qgroup not set)
(Reading database … 67142 files and directories currently installed.)
Removing linux-image-6.1.0-20-arm64 (6.1.85-1) …
/etc/kernel/postrm.d/initramfs-tools:
update-initramfs: Deleting /boot/initrd.img-6.1.0-20-arm64
/etc/kernel/postrm.d/z50-raspi-firmware:
cp: error writing ‘/boot/firmware/initrd.img-6.1.0-22-arm64’: No space left on device
run-parts: /etc/kernel/postrm.d/z50-raspi-firmware exited with return code 1
dpkg: error processing package linux-image-6.1.0-20-arm64 (–remove):
installed linux-image-6.1.0-20-arm64 package post-removal script subprocess returned error exit status 1
dpkg: too many errors, stopping
Errors were encountered while processing:
linux-image-6.1.0-20-arm64
Processing was halted because there were too many errors.
quota not working (qgroup not set)
E: Sub-process /usr/bin/dpkg returned an error code (1)

So sadly, that didn’t work.

[DonQuishot]

@njoseph : it is a manual update.

[Avron]

What is the output of ls -la /boot and of df -h?

[DonQuishot]

@Avron

ls -la /boot

ls -la /boot

total 206224
drwxr-xr-x 1 root root      574 Jul  1 06:09 .
drwxr-xr-x 1 root root      226 Jun 30 06:46 ..
-rw-r--r-- 1 root root   291002 Feb  1 09:05 config-6.1.0-18-arm64
-rw-r--r-- 1 root root   291030 May  3 14:36 config-6.1.0-21-arm64
-rw-r--r-- 1 root root   291030 Jun 21 05:59 config-6.1.0-22-arm64
drwxr-xr-x 2 root root    16384 Jan  1  1970 firmware
-rw-r--r-- 1 root root 37406638 Feb 27 01:08 initrd.img-6.1.0-18-arm64
-rw-r--r-- 1 root root 37420889 Jun 30 06:36 initrd.img-6.1.0-21-arm64
-rw-r--r-- 1 root root 37407690 Jul  1 00:03 initrd.img-6.1.0-22-arm64
-rw-r--r-- 1 root root       83 Feb  1 09:05 System.map-6.1.0-18-arm64
-rw-r--r-- 1 root root       83 May  3 14:36 System.map-6.1.0-21-arm64
-rw-r--r-- 1 root root       83 Jun 21 05:59 System.map-6.1.0-22-arm64
-rw-r--r-- 1 root root 32622528 Feb  1 09:05 vmlinuz-6.1.0-18-arm64
-rw-r--r-- 1 root root 32688064 May  3 14:36 vmlinuz-6.1.0-21-arm64
-rw-r--r-- 1 root root 32688064 Jun 21 05:59 vmlinuz-6.1.0-22-arm64

@Avron

df -h

df -h

Filesystem      Size  Used Avail Use% Mounted on
udev            399M     0  399M   0% /dev
tmpfs            91M  2.5M   88M   3% /run
/dev/mmcblk0p2   30G  7.4G   22G  26% /
tmpfs           452M     0  452M   0% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
/dev/mmcblk0p1  256M  256M     0 100% /boot/firmware
/dev/mmcblk0p2   30G  7.4G   22G  26% /.snapshots
tmpfs            91M     0   91M   0% /run/user/10000

[Avron]

I was expecting /boot to be a partition and to be full, like on the Pioneer, but apparently on your device it is not so, the partition that is full is /boot/firmware.

What is the output of ls -lRa /boot/firmware?

[DonQuishot]

@Avron

ls -lRa /boot/firmware

ls -lRa /boot/firmware
/boot/firmware:
total 261884
drwxr-xr-x 2 root root    16384 Jan  1  1970 .
drwxr-xr-x 1 root root      574 Jul  1 06:09 ..
-rwxr-xr-x 1 root root    27406 Jul  1 17:58 bcm2711-rpi-400.dtb
-rwxr-xr-x 1 root root    27434 Jul  1 17:58 bcm2711-rpi-4-b.dtb
-rwxr-xr-x 1 root root    27323 Jul  1 17:58 bcm2711-rpi-cm4-io.dtb
-rwxr-xr-x 1 root root    14816 Jul  1 17:58 bcm2837-rpi-3-a-plus.dtb
-rwxr-xr-x 1 root root    14993 Jul  1 17:58 bcm2837-rpi-3-b.dtb
-rwxr-xr-x 1 root root    15349 Jul  1 17:58 bcm2837-rpi-3-b-plus.dtb
-rwxr-xr-x 1 root root    14355 Jul  1 17:58 bcm2837-rpi-cm3-io3.dtb
-rwxr-xr-x 1 root root    14667 Jul  1 17:58 bcm2837-rpi-zero-2-w.dtb
-rwxr-xr-x 1 root root    52460 Feb 27 01:02 bootcode.bin
-rwxr-xr-x 1 root root      132 Jun 30 06:37 cmdline.txt
-rwxr-xr-x 1 root root      577 Jun 30 06:37 config.txt
-rwxr-xr-x 1 root root     3170 Feb 27 01:02 fixup4cd.dat
-rwxr-xr-x 1 root root     5399 Feb 27 01:02 fixup4.dat
-rwxr-xr-x 1 root root     8379 Feb 27 01:02 fixup4db.dat
-rwxr-xr-x 1 root root     8379 Feb 27 01:02 fixup4x.dat
-rwxr-xr-x 1 root root     3170 Feb 27 01:02 fixup_cd.dat
-rwxr-xr-x 1 root root     7262 Feb 27 01:02 fixup.dat
-rwxr-xr-x 1 root root    10228 Feb 27 01:02 fixup_db.dat
-rwxr-xr-x 1 root root    10226 Feb 27 01:02 fixup_x.dat
-rwxr-xr-x 1 root root 37406638 Feb 27 01:08 initrd.img-6.1.0-18-arm64
-rwxr-xr-x 1 root root 37419781 Apr 24 17:49 initrd.img-6.1.0-20-arm64
-rwxr-xr-x 1 root root 37420889 Jun 30 06:37 initrd.img-6.1.0-21-arm64
-rwxr-xr-x 1 root root  2777088 Jul  1 17:58 initrd.img-6.1.0-22-arm64
-rwxr-xr-x 1 root root   803964 Feb 27 01:02 start4cd.elf
-rwxr-xr-x 1 root root  3744808 Feb 27 01:02 start4db.elf
-rwxr-xr-x 1 root root  2249280 Feb 27 01:02 start4.elf
-rwxr-xr-x 1 root root  2996680 Feb 27 01:02 start4x.elf
-rwxr-xr-x 1 root root   803964 Feb 27 01:02 start_cd.elf
-rwxr-xr-x 1 root root  4816712 Feb 27 01:02 start_db.elf
-rwxr-xr-x 1 root root  2973536 Feb 27 01:02 start.elf
-rwxr-xr-x 1 root root  3720360 Feb 27 01:02 start_x.elf
-rwxr-xr-x 1 root root 32622528 Feb 27 01:08 vmlinuz-6.1.0-18-arm64
-rwxr-xr-x 1 root root 32688064 Apr 24 17:49 vmlinuz-6.1.0-20-arm64
-rwxr-xr-x 1 root root 32688064 Jun 30 06:37 vmlinuz-6.1.0-21-arm64
-rwxr-xr-x 1 root root 32688064 Jul  1 17:58 vmlinuz-6.1.0-22-arm64

[Avron]

Apparently, the kernel (vmlinux) and init ramdisk (initrd) files are duplicated in /boot and /boot/firmware, the “sudo apt autoremove” command removed the ones for 6.1.0-20-arm64 from /boot but not from /boot/firmware.

What I would try (but this could mess up things):

• delete /boot/firmware/initrd.img-6.1.0-20-arm64 and /boot/firmware/vmlinuz-6.1.0-20-arm64 (sudo rm /boot/firmware/initrd.img-6.1.0-20-arm64 /boot/firmware/vmlinuz-6.1.0-20-arm64)
• run sudo apt autoremove again

Because this could mess up things, before doing that, I would backup /boot/firmware by putting the micro SD card into another GNU/Linux computer and running dd if=/dev/sdX1 of=boot-firmware-partition-backup.img (X needs to be replaced with the correct letter, you can find it by running lsblk --fs and see the partition that is 256 MB and ext2).

If the computer is running GNU/Linux with systemd (like Debian, Ubuntu or similar), before inserting the micro SD card in it, run sudo systemctl stop udisks2.service so that the computer does not try to mount it automatically when inserted.

That said, I don’t know why you are running into that issue. Obviously, your /boot/firmware partition can contain at most 3 (kernel + initram disk), not sure why you have 4.

[lifeform]

Is raspi-firmware installed? See this thread that discusses pruning of /boot/firmware.

[DonQuishot]

dpkg -l raspi-firmware
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name           Version         Architecture Description
+++-==============-===============-============-================================================
ii  raspi-firmware 1.20220830+ds-1 all          Raspberry Pi family GPU firmware and bootloaders

So, that seems to be installed.

[DonQuishot]

@Avron : as much as I appreciate your solution, and I do, I’m afraid it is just a temporary solution. A few kernel updates down the road, and I’m back where I started.

[jvalleroy]

I believe this is the bug in raspi-firmware: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1032186

(For anyone interested, the code change to fix it is here: Don't copy files to /boot/firmware on remove (Closes: #1032186) (7ebd5e57) · Commits · Debian / raspi-firmware · GitLab)

This was fixed in raspi-firmware 1.20230405+ds-1, so it has been fixed in unstable and testing. However, Debian stable has raspi-firmware 1.20220830+ds-1, and I don’t see any bugfix updates for it.

So I would recommend that we request to the raspi-firmware maintainers to see if the fix can be applied to raspi-firmware in Debian stable. I see they have a mailing list here: Pkg-raspi-maintainers Info Page

[DonQuishot]

Now that we know that it is a bug in raspi-firmware, and that the fix is not yet available in Debian stable, is there a fail save temporary solution that an inexperienced user like me could apply right now? New updates of other packages have arrived but I can’t install these now, as the process is blocked by the unfinished update of the kernel.