This is Gentoo's testing wiki. It is a non-operational environment and its textual content is outdated.
Please visit our production wiki at https://wiki.gentoo.org
User:Fearedbliss/Installing Gentoo Linux On ZFS
Install Gentoo Linux on ZFS
Author: Jonathan Vasquez (fearedbliss)
Contact: jon@xyinn.org
Preface
This guide will show you how to install Gentoo Linux on AMD64 with:
* UEFI-GPT (EFI System Partition) - This will be on a FAT32 unencrypted partition as per UEFI Spec. * Everything on ZFS (/, /home, /boot, swap) * ZFS 0.7.8 * GRUB 2.02 (Bootloader) * OpenRC * Gentoo Stable (amd64)
Notes on other stuff you can do in the ZFS install can be found here.
Required Tools
Download my modified System Rescue CD that includes ZFS from one of the mirrors on the front page.
Linux
If you have a Linux system, you can create a bootable usb from it by doing the following:
root #mkdir /tmp/iso
root #mount -o loop,ro <system rescue iso> /tmp/iso
root #/tmp/iso/usb_inst.sh
That should open up a screen where you can select your plugged in flash drive. If script show dialog: loadlocale[...] try set your locales to en_US.UTF-8
Windows
You can download "Rufus" and select the ISO by clicking the little image next to the "Create a bootable disk using <>". After that, you can use the default settings and press "Start".
Mac OS
If you have a Mac OS system, you can create a bootable usb by doing the following:
- Insert the USB device.
- Using Disk Utility, format the USB device. Make sure to set the file system to FAT.
- Mount the ISO. Both the freshly formatted USB and the ISO should appear mounted under /Volumes
- From a terminal, run: cp -R /Volumes/sysresccd_zfs /Volumes/<USB name> (Chances are your USB drive name is UNTITLED.)
- Unmount the USB using Finder or the command line diskutil application.
- Reboot, holding down Left-Option when you hear the chime. Release after a few seconds.
- You should see a removable device you can now boot from. After selecting, it should load GRUB.
Assumptions
- Only installing Gentoo on one drive called /dev/sda.
- You are using my modified System Rescue CD that includes ZFS.
- bliss-initramfs and bliss-kernel are used. You can also use genkernel and your own custom compiled kernel if desired.
Also, this guide is the way I install Gentoo, not exactly the way the handbook has it.
Confirm that you booted in UEFI Mode
After you booted into the Live CD, make sure that you booted into UEFI mode by typing the following:
root #ls /sys/firmware/efiIf the above directory is empty or doesn't exist, you are not in UEFI mode. Reboot and boot into UEFI mode.
Continuing the installation without being in UEFI mode will most likely yield an unbootable system. If you want to install in BIOS mode, you will need a different setup.
Partition
We will now partition the drive and aim to create the following layout:
/dev/sda1 | 512 MiB | EFI System Partition | /boot/efi /dev/sda2 | 500 MiB | Boot Partition (No Feature Flags) | /boot /dev/sda3 | Rest of Disk | ZFS | /, /home, swap, etc
There are many UEFI motherboard firmwares that are extremely buggy. We will attempt to use a 512 MiB FAT32 partition configuration to increase success.
Open up your drive in GNU parted and tell it to use optimal alignment:
root #parted -a optimal /dev/sdaKeep in mind that all of the following operations will affect the disk immediately. GNU parted does not stage changes like fdisk or gdisk.
Set the units to Mebibytes
(parted)unit mibCreate GPT partition layout
This will delete all partitions and create a new GPT table.
(parted)mklabel gptCreate and label your partitions
(parted)mkpart primary 1 513
(parted)mkpart primary 513 1013
(parted)mkpart primary 1013 -1
(parted)name 1 esp
(parted)name 2 boot
(parted)name 3 rootfs
Set the bootable flag on the ESP partition
(parted)set 1 boot onFinal View
(parted)printModel: ATA VBOX HARDDISK (scsi) Disk /dev/sda: 102400MiB Sector size (logical/physical): 512B/512B Partition Table: gpt Number Start End Size File system Name Flags 1 1.00MiB 513MiB 512MiB esp boot, esp 2 513MiB 1013MiB 500MiB boot 3 1013MiB 102399MiB 101386MiB rootfs
Exit the application
(parted)quitFormat your drives
Create your boot partition
root #mkfs.fat -F32 /dev/sda1This partition needs to be FAT32 due to it being an UEFI requirement. If it isn't, your system will not boot!
Create your zpool
Create your zpool which will contain your drives and datasets:
root #zpool create -f -o ashift=12 -o cachefile= -O compression=lz4 -m none -R /mnt/gentoo rpool /dev/sda3Create your zfs datasets
We will keep it simple and just create a few datasets for /, and /home. ZFS is extremely flexible and you can easily add or remove datasets in the future.
root #zfs create rpool/ROOT
root #zfs create -o mountpoint=/ rpool/ROOT/gentoo
root #zfs create -o mountpoint=/home rpool/HOME
Create your boot pool
It is safer for us to create a separate zpool that has all feature flags disabled. This is because even though grub currently supports the latest feature flags for the 0.6.5 release of ZFS, new feature flags added to ZFS without proper bootloader support can make your system unbootable. Since GRUB 2 has solid support for a zpool with no feature flags, we will create a separate featureless boot pool. However, our main system will still have all features enabled.
root #zpool create -f -d -o ashift=12 -o cachefile= -m /boot -R /mnt/gentoo boot /dev/sda2Do not do a 'zpool upgrade' on your boot pool. Doing this will mostly likely leave your system unbootable.
Create your swap zvol
This creates the zfs block device that we will use for swap.
root #zfs create -o sync=always -o primarycache=metadata -o secondarycache=none -b 4k -V 4G -o logbias=throughput rpool/swap
root #mkswap -f /dev/zvol/rpool/swap
root #swapon /dev/zvol/rpool/swap
- Currently, the performance of the swap zvol isn't that great. This should improve over time.
- You should have a swap even if it is small. Your system could lock up under high load, especially since ZFS uses your physical RAM for its ARC.
Verify everything looks good
You can verify that all of these things worked by running the following:
root #zpool status pool: boot
state: ONLINE
status: The pool is formatted using a legacy on-disk format. The pool can
still be used, but some features are unavailable.
action: Upgrade the pool using 'zpool upgrade'. Once this is done, the
pool will no longer be accessible on software that does not support
feature flags.
scan: none requested
config:
NAME STATE READ WRITE CKSUM
boot ONLINE 0 0 0
sda2 ONLINE 0 0 0
errors: No known data errors
pool: rpool
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
rpool ONLINE 0 0 0
sda3 ONLINE 0 0 0
errors: No known data errors
root #zfs listNAME USED AVAIL REFER MOUNTPOINT boot 484K 448M 136K /mnt/gentoo/boot rpool 4.25G 91.7G 96K none rpool/HOME 192K 91.7G 96K /mnt/gentoo/home rpool/ROOT 192K 91.7G 96K none rpool/ROOT/gentoo 96K 91.7G 96K /mnt/gentoo rpool/swap 4.25G 95.9G 64K -
Now we are ready to install Gentoo!
Installing Gentoo
Check and set your time
root #dateIf it's wrong then fix it. Let's say it's August 24, 2014 @ 2:48 PM (will be 14:48 in 24 hour time), we would do the following:
root #date 08241448201408 24 1448 2014 (Month, Date, 24 Hour Time, Year). It will be UTC at the moment but we will fix it once we are inside our environment.
Preparing to chroot
First let's mount our efi boot partition in our chroot directory:
root #cd /mnt/gentoo
root #mkdir boot/efi
root #mount /dev/sda1 boot/efi
and now download the amd64 image here and extract it:
root #wget <file>
root #tar xpf <file>
Copy zpool cache
root #mkdir etc/zfs
root #cp /etc/zfs/zpool.cache etc/zfs
Copy network settings
root #cp /etc/resolv.conf etc/Mount required devices
root #mount --rbind /dev dev && mount --make-rslave dev
root #mount --rbind /proc proc && mount --make-rslave proc
root #mount --rbind /sys sys && mount --make-rslave sys
Chroot into your environment
root #env -i HOME=/root TERM=$TERM $(which chroot) . bash -lInside the chroot
Generate locales
I'm using US English w/ UTF-8, so I will enable that locale on my system:
root #nano /etc/locale.gen
root #locale-gen
root #eselect locale set en_US.utf8
root #env-update
root #. /etc/profile
Fix your date
I'm in New York and the time currently is August 24, 2014 @ 2:48 PM (will be 14:48 in 24 hour time)
root #ln -sf /usr/share/zoneinfo/America/New_York /etc/localtime
root #date 082414482014
Set hostname
Open up /etc/conf.d/hostname with your text editor and add your desired hostname in there.
root #nano /etc/conf.d/hostnameFor example, my /etc/conf.d/hostname file could be:
hostname="bliss"
Create mtab symlink
root #ln -sf /proc/self/mounts /etc/mtabEdit fstab
Everything is on zfs so we don't need anything in here except for the boot and swap zvol entries. My fstab looks as follows:
root #nano /etc/fstab/dev/sda1 /boot/efi vfat noauto 1 2 /dev/zvol/rpool/swap none swap sw 0 0
Do not auto-mount the /boot/efi partition. If you do, the init system will create the /boot/efi folder if it doesn't exist, and that will prevent the 'boot' zpool from mounting.
Modify make.conf
Let's modify our /etc/portage/make.conf so we can start installing stuff with a good base (Change it to what you need):
root #nano /etc/portage/make.confCPU_FLAGS_X86="mmx sse sse2" USE="-branding" # This should be your number of processors + 1 MAKEOPTS="-j5" EMERGE_DEFAULT_OPTS="--with-bdeps=y --keep-going=y" FEATURES="buildpkg" LINGUAS="en en_US" # This is required so that when we compile GRUB later, EFI support is built. GRUB_PLATFORMS="efi-64"
Get the portage tree
Copy the default example portage config
root #cp /usr/share/portage/config/repos.conf /etc/portage/repos.confroot #emerge --syncCompiling your own kernel
Using the 'bliss-kernel' configuration as a base
If you don't want to use the pre-compiled "bliss-kernel" in "bliss-overlay" below, I've made available the 'bliss-kernel' configs so that you can compile the kernel yourself using that configuration as a base. The configuration is ready to go for general desktop, laptop, and server use, and it is also configured to work with ZFS.
You can download the 'bliss-kernel' configuration here.
Using your own configuration
If you are not going to use the above configuration, then you will need to make sure your kernel has the following options enabled or ZFS will not work:
This list is still being updated, and is using the options listed in the sys-kernel/spl ebuild as a base.
Disable anything marked [ ], Compile in [*], Options marked [M] may either be separate modules or compiled into the kernel:
[ ] DEBUG_LOCK_ALLOC [*] EFI_PARTITION [*] IOSCHED_NOOP [*] MODULES [*] BLK_DEV_INITRD [*] KALLSYMS [ ] PAX_KERNEXEC_PLUGIN_METHOD_OR [ ] PAX_SIZE_OVERFLOW [M] ZLIB_DEFLATE [M] ZLIB_INFLATE
(Optional) Add bliss-overlay and install a pre-compiled/configured ZFS-ready kernel
bliss-kernel is a vanilla kernel (directly from kernel.org, no patches) that fearedbliss maintains. It uses a slightly tweaked fedora config. The kernel is pre-configured and tested for ZFS. Use this if you don't want to spend time tweaking/debugging the Linux Kernel.
Open repos.conf and add bliss-overlay
root #nano /etc/portage/repos.conf[bliss-overlay] location = /var/lib/overlays/bliss-overlay sync-type = git sync-uri = https://github.com/fearedbliss/bliss-overlay.git auto-sync = yes
Compile git
root #emerge dev-vcs/gitSync once more to get bliss-overlay
root #emerge --syncInstall the kernel
root #emerge bliss-kernelConsider pinning the kernel version that was installed so that when we do a future kernel upgrade, an emerge --depclean won't delete our kernel files from the /boot directory. If the bliss-kernel ebuild that emerge pulled to install was: bliss-kernel-4.1.8, we would do:
root #emerge --noreplace =bliss-kernel-4.1.8Install required applications
Enable ZFS support in GRUB
root #echo "sys-boot/grub libzfs" >> /etc/portage/package.useNow install the apps:
root #emerge bliss-initramfs grubUpdate your system
root #emerge -uDNav @worldInstalling the bootloader onto your drive
We will need to install the bootloader onto the drive. Before we do that however, let's see if GRUB can detect our /boot and /boot/efi filesystem types:
root #grub-probe /bootThis should say 'zfs'. If it doesn't, then something is wrong and your system will not boot!
root #grub-probe /boot/efiThis should say 'fat'. If it doesn't, then something is wrong and your system will not boot!
Before we install the bootloader, we will need to have read/write access to the efi nvram variables. Let's remount our efivars now:
root #mount -o remount,rw /sys/firmware/efi/efivars/Now run the following to install the bootloader to the drive:
root #grub-install --efi-directory=/boot/efiThe above command will install the grub bootloader files into /boot and the efi files into /boot/efi. It should return a "Installation finished. No error reported." message. If it doesn't, then something is wrong and your system will not boot!
If after a reboot your UEFI motherboard doesn't detect your bootloader, it might be because of a buggy motherboard firmware. Try to pass the "--removable" flag to grub install and try again. This will install the efi file to the EFI specification's fallback directory.
Make the GRUB 2 configuration file
You can use the following configuration file as a basis for your system:
root #nano /boot/grub/grub.cfg
set default=0
set timeout=1
insmod part_gpt
insmod fat
insmod efi_gop
insmod efi_uga
menuentry "Gentoo - 4.1.8-FC.01" {
linux /@/kernels/4.1.8-FC.01/vmlinuz root=rpool/ROOT/gentoo by=id elevator=noop quiet logo.nologo triggers=zfs refresh
initrd /@/kernels/4.1.8-FC.01/initrd
}
Generating new zpool.cache file before/after reboot
ZFS is very sensitive about the data that is contained in the zpool.cache file and at this point, when we reboot, the information in it might not be completely accurate. To ensure we have a good cache file, we have instructed bliss-initramfs in the bootloader config above, to ignore the current cachefile on the system, and make a new one that is up-to-date. We only need to do this once.
Generate the initramfs and move the file to its correct location
root #bliss-initramfs zfs 4.1.8-FC.01
root #mv initrd-4.1.8-FC.01 /boot/kernels/4.1.8-FC.01/initrd
Final steps before reboot
root #rc-update add zfs-import boot
root #rc-update add zfs-mount boot
root #rc-update add zfs-share default
root #rc-update add zfs-zed default
root #passwd
root #exit
root #reboot
After you reboot
Remove zpool.cache refresh flag from bootloader configuration
Open up your grub.cfg and remove the 'refresh' flag from the kernel line.
root #nano /boot/grub/grub.cfgTake a snapshot of your new system
Since we now have a working system, we will snapshot it in case we ever want to go back or recover files:
root #zfs snapshot boot@2016-03-25-0000-01-INSTALL
root #zfs snapshot rpool/ROOT/gentoo@2016-03-25-0000-01-INSTALL
root #zfs snapshot rpool/HOME@2016-03-25-0000-01-INSTALL
You can view the contents of these snapshots by checking their respective and hidden .zfs directories:
root #ls /boot/.zfs/snapshots
root #ls /.zfs/snapshots
root #ls /home/.zfs/snapshots
We are naming the snapshots in this format because it's the format that my 'bliss-zfs-scripts' use, and it is also very readable.
Other stuff you might want to install
Tools in Portage
root #emerge sudo htop gentoolkit eix app-misc/screen app-text/treeBliss ZFS Scripts
The following scripts allow you to automatically:
- Take snapshots of your pool
- Replicate the pool to another pool (Full and Incremental Backups)
- Clean the old snapshots on your pools.
You can download, customize, and install the scripts into your /usr/local/sbin directory. Github.
And that's it. Enjoy!