Re: ZFS on root - almost there

[Roy Marples <roy%marples.name@localhost>]

On 14/02/2020 12:58, Roy Marples wrote:
> So I thought I would have a go at setting up ZFS on root.

I've now comitted enough to manually build a ramdisk to set this all up.
Quick instruction steps which I'll document on web page later:

Compile the ramdisk
cd src/distrib/amd64/ramdisks/ramdisk-zfsroot
nbmake-amd64

Ensure you are using GPT and not MBR. If you need to change, dd the disk
using /dev/zero as source for about 32k and then the installer will ask you if 
you want MBR or GPT. Once set, it will not prompt to change it again.

Use the installer to do a normal installation, extracting base, modules and 
rescue sets to a small FFS parition (I chose 2G). Do not allow the installer to 
use the rest of the disk.
Drop to the prompt and copy the ramdisk you made earlier to /
Edit /boot.cfg and add this menu item:
menu=Boot ZFS root:fs /ramdisk-zfsroot.fs;boot

Create a ZFS pool on another partition called rpool.
Create the ZFS root filesytem called rpool/ROOT.
zfs set mountpoint=legacy rpool/ROOT
This step is important - the only downside is if you want to create any ZFS 
datasets in rpool/ROOT you need to either set mountpoints in /etc/fstab or 
specify them as they will automatically inherit legacy from ROOT.
Extract the sets you want rpool/ROOT.
Create dev on rpool/ROOT, copy MAKEDEV from /dev to it, cd to it and run 
./MAKEDEV all
Copy your /etc/fstab to rpool/ROOT/etc, but remove the / entry.
Ensure that rc.conf is setup in rpool/ROOT/etc and it has zfs=YES

You should now be good to go!

WARNING: There seems to be a bug that once booted into a ZFS root and mount any 
device and write to it the system will hang trying to unmount it. This is not a 
fault with the ramdisk, but rather with how ZFS works with device nodes on ZFS. 
So to update the kernel, boot into the FFS partition and copy from the ZFS 
partition rather from doing it within the ZFS root.

Once that is fixed I might look into trying to automate some of this in our 
installer.

Good luck!

Roy