blog.lazkani.io/content/posts/nixos-on-encrypted-zfs.md

+++
title = "NixOS on encrypted ZFS"
author = ["Elia el Lazkani"]
date = 2021-10-17
lastmod = 2021-10-17
tags = ["zfs", "encryption"]
categories = ["nix", "nixos"]
draft = false
+++

I wouldn't call myself a distro hopper. The decision of distribution is solely
based on requirements. I have requirements and I want the distribution to
fulfill them as much as possible. After 15 years, I know what I want and I go
and find it.

In this case, an unexpected project caught my eye. The idea is so radically
different that I wasn't actually searching for it this time. It is one of those
times where it found me first.

After looking into **Nix** and **NixOS**, I decided it is going to be my
distribution of choice on the desktop. I will use that as my test bed before
migrating all the serious work there. That's how I got my first taste of **NixOS**
outside of the deterministic virtualization layer and into the wild.

<!--more-->


## Requirements {#requirements}

Before installing any new system, I draftdown a list of requirements I would
need this system to run. These are things that are very hard to change on the
fly in the future without some serious work. Also, things that simply need to be
there in this day and age.


### Filesystem {#filesystem}

I'm a big fan of `zfs`. I've been running it on Linux, since the `openzfs`
project successfully ported it, with no issues. It's a solid choice for a
filesystem and I don't see a reason not to choose it.

Is it really a requirement ?

Well, yes. By now, `openzfs` should be accessible to all distributions but my
choice of distribution is not usually for the beginner user. I need to know
it's supported and documented. I can figure out the rest from there.


### Encryption {#encryption}

This is the first requirement, I always want encryption. The reason why I put it
second in the list is that I needed to talk about `zfs` first.

The `zfs` filesystem offers encryption. Unfortunately, my research have shown
that `zfs` might not encrypt some metadata. This might not be a big deal but the
choice of using Luks is there as well.

With Luks, we can encrypt the whole drive. So let's do that; Luks with `zfs` on top.


## NixOS {#nixos}

**NixOS** utilizes **Nix** to build you an OS from a configuration file. This
configuration file is written in the `nix` language. It is very analogous to
written an `Ansible` playbook but it builds an OS instead.

The idea sounded appealing to me. A good friend of mine, [setkeh](https://setkeh.com/), gave me a quick and
dirty overview, at first. That pushed me into doing more research of my own
where I found out that I can spawn off a `nix-shell` with whatever dependencies
I want without having them installed on my system. What a great concept for
development or even running applications you don't really want to run. `Java`
stuff for example.

Anyway, for all of these different reasons I have chosen **NixOS** to be the OS of
choice to go on the desktop.


## Installation {#installation}

After testing [**NixOS**](https://nixos.org/) in a VM a few times, I got `setkeh` on a conference
session and we dug into this.


### Filesystem partitioning {#filesystem-partitioning}

For the filesystem, we're going to create two partitions. We need one, `vfat`,
for the boot and another, `zfs`, for the rest of the filesystem.

<div class="admonition note">
<p class="admonition-title"><b>Note</b></p>

The assumption is that we're installing **NixOS** on an `EFI` enabled system.

</div>

We can start by creating the first partition of `1GB`.

```shell
sgdisk -n3:1M:+1024M -t3:EF00 /dev/disk/by-id/VENDOR-ID
```

Followed by the rest of the filesystem.

```shell
sgdisk -n1:0:0 -t1:BF01 /dev/disk/by-id/VENDOR-ID
```

<div class="admonition note">
<p class="admonition-title"><b>Note</b></p>

It is usually easier to do the partitioning using `GParted`. Make sure that the
partitions are unformatted, if you do so.

</div>

<div class="admonition warning">
<p class="admonition-title"><b>Warning</b></p>

Do **NOT** forget to enable the boot flag on the first partition or your system
will not boot.

</div>


### Filesystem formatting {#filesystem-formatting}

Now that we got our partitions creates, let's go ahead and format them properly.

Starting with the `boot` partition first.

```shell
mkfs.vfat /dev/disk/by-id/VENDOR-ID-part1
```

<div class="admonition warning">
<p class="admonition-title"><b>Warning</b></p>

At this sage, you're formatting a partition. Make sure you're pointing to the
partition and not your whole disk as in the previous section.

</div>

Then our `zfs` partition, but we need to encrypt it first. So, we create the
Luks partition.

```shell
cryptsetup luksFormat /dev/disk/by-id/VENDOR-ID-part2
```

At this stage, stage we are done with the filesystem formatting and we need to
create the `zfs` pool. To do so, we need to mount the encrypted `root`
filesystem; Luks.

```shell
cryptsetup open --type luks /dev/disk/by-id/VENDOR-ID-part2 crypt
```

This mounts the filesystem in `/dev/mapper/crypt`. We'll use that to create the pool.

```shell
zpool create -O mountpoint=none rpool /dev/mapper/crypt
zfs create -o mountpoint=legacy rpool/root
zfs create -o mountpoint=legacy rpool/root/nixos
zfs create -o mountpoint=legacy rpool/home
```


### Filesystem mounting {#filesystem-mounting}

After creating the filesystem, let's mount everything.

```shell
# Mounting filesystem
mount -t zfs rpool/root/nixos /mnt
mkdir /mnt/home
mkdir /mnt/boot
# Mounting home directory
mount -t zfs rpool/home /mnt/home
# Mounting boot partition
mount /dev/disk/by-id/VENDOR-ID-part1 /mnt/boot
```


### Generating NixOS configuration {#generating-nixos-configuration}

At this stage, we need a `nix` configuration to build our system from. I didn't
have any configuration to start from so I generated one.

```shell
nixos-generate-config --root /mnt
```


### NixOS configuration {#nixos-configuration}

Due to the weird configuration we've had, we need to make a few adjustements to
the suggested configuration layed out in the docs.

The required configuration bits to be added to
`/mnt/etc/nixos/configuration.nix` are:

```nix
boot.supportedFilesystems = [ "zfs" ];
# Make sure you set the networking.hostId option, which ZFS requires:
networking.hostId = "<random 8-digit hex string>";
# See https://nixos.org/nixos/manual/options.html#opt-networking.hostId for more.

# Use the GRUB 2 boot loader.
boot.loader.grub = {
  enable = true;
  version =2;
  device = "nodev";
  efiSupport = true;
  enableCryptodisk = true;
};

boot.initrd.luks.devices = {
 root = {
   device = "/dev/disk/by-uuid/VENDOR-UUID-part2"; ## Use blkid to find this UUID
   # Required even if we're not using LVM
   preLVM = true;
 };
};
```


### NixOS installation {#nixos-installation}

If we're done with all of the configuration as described above, we should be
able to build a bootable system. Let's try that out by installing **NixOS**.

```shell
nixos-install
```


## Conclusion {#conclusion}

It took a bit of trial and error, and a loooooooot of mounting over and over
again. At the end, though, it wasn't as bad as I thought it would be. I'm still
trying to get myself familiarised with **NixOS** and the new way of doing things.
All in all, I would recommend trying **NixOS**, or the very least **Nix**.