How to change the boot partition?

[tobiasBora]

In order to have a bit of fun I wanted to try ZFS on the pi5 to get snapshots, incremental backups etc. so I got the kernel working, but now I need to update the configuration to make it work, and my main question is on how to setup the /boot partition. Of course, it cannot be a ZFS partition, so I need a separate FAT32 partitition, but I don't know, can I just assume that the FIRMWARE partition contains the /boot partition and don't do anything else beside adding a bunch of

  fileSystems."/home" =
    { device = "zfs_pool/home";
      fsType = "zfs";
    };

in my configuration.nix? Or do I need more care here?

[tstat]

I don't think you need more care than that, but don't know for certain. If you try it out please report back how it goes.

[tobiasBora]

Ok thanks… but I just tried as a first step to shrink the ext4 partition and put in the free space a ZFS one… and my raspberry pi 5 couldn't boot anymore. As I had no mini-HDMI cable, and couldn't get UART to work (while it was working on the rpi4), I can't tell why exactly, hopefully I can try again in a few days. So I have 2 questions:

1. is there a code that tries to expand the ext4 partition that may fail if there is a partition after it? Or maybe the issue was that I also added at the same time a ZFS module in the kernel and that it broke the kernel?
2. can I somehow revert back to an older generation (e.g. the one without ZFS in the kernel)?

[tstat]

Oh dear, sorry I never saw this response.

is there a code that tries to expand the ext4 partition that may fail if there is a partition after it? Or maybe the issue was that I also added at the same time a ZFS module in the kernel and that it broke the kernel?

Hmm, what are the kernel params now (stored at /boot/firmware/cmdline.txt)? I wonder if the root param is no longer pointing to the correct partition.

can I somehow revert back to an older generation (e.g. the one without ZFS in the kernel)?

If you are using the default boot process then the kernel is in the boot partition, and there isn't a way to roll back the kernel. You can, of course, replace it with another kernel.

Rolling back everything else can be done with some manual effort. It is a standard initScript bootloader. /sbin/init is ran after the kernel is loaded. That script will just exec <some-nix-path>, so replacing it with a previous configurations exec <some-other-nix-path> will do the trick. The previous init scripts are recorded in /boot/init-other-configurations-contents.txt.

If you are using the uboot bootloader then rolling back can be done easily over uart at boot time. Generation selection is presented over uart.

[tstat]

Ok, I split off the sd card related stuff into a separate module in #79. Using the traditional nixos-install steps I have an rpi5 with a guid partition table and btrfs. I think step one is making this ergonomic, then explore building those images directly.

[tobiasBora]

Thanks a lot for your answer and work, and sorry for my late answer, I had no time yet to try this change but this is planned. But I don't understand how to use nix-install in this setting (do you mean I should do this on my nixos host system, with binfmt correctly configured?) and manually create a boot partition (ext4) and a zfs one? Considering the discussion in #79 I'm not the only one interested by this info ^^ I also see in the PR an option firmware-partition-label. Should I also fill this line?

[tobiasBora]

So I'm trying to reinstall the raspberry pi with these options, but I'm a bit confused about the code in the now external sd-card module: I can see stuff like:

35-        populate-kernel =
36:          if cfg.uboot.enable
37-          then ''
38:            cp ${cfg.uboot.package}/u-boot.bin firmware/u-boot-rpi-arm64.bin
39-          ''
40-          else ''
41-            cp "${kernel}" firmware/kernel.img
42-            cp "${initrd}" firmware/initrd
43-            cp "${kernel-params}" firmware/cmdline.txt
44-          '';
45-      in

but my understanding is that without enabling the sd-card module, this will not be ran. So if I manually run the traditional inxos-install steps, should I manually copy this file to the fat partition? What about all the hardware files, root commands etc?

[tstat]

Oh right, it's been a while since I did this, but I do think I ran that script manually to populate the boot partition. I think I ought to move that population script into system.build.installBootLoader, but haven't looked into that yet.