NixOS on Apple silicon with UTM

Rationale

I am not even sure where I heard first about Nix and NixOS, but I liked the concept right away.

I think you can get addicted to automation (it’s a nice blog post idea!), and implementing everything in a declarative way. There might be some ramp-up period that varies person to person, but once you get its benefits (or better, once you are affected by the lack of them) you will try to do everything in a reproducible way. If this applies to you, be cautious with Nix(OS)!

A few weeks ago I decided to give it a try, and installed it on a RPI4. I played with it for about and hour, and I liked it. Since a few weeks before that my Manjaro Linux broke, I decided to use NixOS there as well.

A few days later I installed it on Virtualbox on my Intel MBP, and finally it made its way to on my M1 machine too.

The reason behind nixing with VMs on macOS’ was to get rid of the chaos of Homebrew. While you can use Nix on bare software macOS, I thought it’s better to do it the native way, and provision declarative VMs, that can even share certain parts of the configs. This will keep the host OS clean, you can run multiple versions of the same packages, and you can keep all of your config in a safe git repository. Additionally, I will finally get a better docker experience compared to the one I have on macOS.

In the following sections, I will go trough my setup, and share a few tips I learned during installing NixOS to 4 different machines.

UTM

Since you cannot use Virtualbox on Apple silicon (this might be the future-proof reference instead of M1, as we are already at M2), I settled with UTM to run my VM.

Note: UTM is basically a QEMU frontend designed for macOS.

I heard good (and some bad) things about Parallels and VMware Fusion, but I decided to try all this the simplest way possible.

This is the spec of my M1:

-- MacBook Pro (16-inch, 2021)
-- Apple M1 Max
-- Memory 64 GB

This is my UTM setup:

System
-- Architecture: ARM64 (aarch64)
-- System: QEMU 7.0 ARM Virtual Machine (alias of virt-7.0) (virt)
-- Memory: 16GB
-- CPU: 4 cores

QEMU
-- Tweaks
---- UEFI Boot [x]
---- RNG Device [x]
---- Use Hypervisor [x]

Display
-- Display: Full Graphics
-- Emulated Display Card: virtio-ramfb-gl (GPU Supported)
-- Upscaling: Linear
-- Downscaling: Linear

Network
-- Network Mode: Shared Network

Be aware, there’s no ACPI shutdown support yet, so never restart the VM from the host machine!

If you skipped this warning and you are in trouble booting your system, follow these steps:

  1. switch to Console Only graphics type
  2. attach the installer image, and boot into it live
  3. sudo fsck /dev/disk/by-label/nixos
  4. repair everything

then, shutdown, enable graphics, boot the VM, it works.

See the following issue for further details: https://github.com/utmapp/UTM/issues/2682

Installation

First, download an iso image.

I wanted to run an ARM build, so I used the latest build from here: https://hydra.nixos.org/job/nixos/release-22.05-aarch64/nixos.iso_minimal.aarch64-linux

You can follow the official guide, except you don’t need to care about setting up network if you are installing to a VM.

Just to persist it, I went with the following partitions:

Note: This should probably be a Makefile, a shell script executing all these steps over ssh, or a nix flake even, while also potentially handling the various platform it is running on. Maybe later.

parted /dev/vda -- mklabel gpt
parted /dev/vda -- mkpart primary 1GiB -8GiB
parted /dev/vda -- mkpart primary linux-swap -8GiB 100%

parted /dev/vda -- mkpart ESP fat32 1MiB 512MiB
parted /dev/vda -- set 3 esp on

mkfs.ext4 -L nixos /dev/vda1
mkswap -L swap /dev/vda2
mkfs.fat -F 32 -n boot /dev/vda3

mount /dev/disk/by-label/nixos /mnt
mkdir -p /mnt/boot
mount /dev/disk/by-label/boot /mnt/boot

Then, we can generate the initial configuration for our system:

nixos-generate-config --root /mnt

At this point, you can alter this configuration to you liking and set up the first iteration that will be installed.

As vim is not included by default, now is a good time to enable it (via nano) at environment.systemPackages.

Apart from vim, you don’t really have to add anything else at this point.

Note: If you are installing this to bare metal, and you don’t have an Ethernet cable at hand, make sure to set networking.wireless.enable to true, because otherwise you won’t be able to rebuild NixOS after the installation due to not having Internet connection. Alternatively, networking.networkmanager.enable could also work, but only if your initial configuration supports it, e.g. if you are starting with xfce+i3 right away.

You can install the OS now:

nixos-install

First steps

Now, you should be able to login with the root password you just set up.

This is a good time to create your own user, and set its password. You can manage all its config with better tools introduced later.

After the initial installation, you can make incremental changes to your configuration.nix, but from this point, you will use nixos-rebuild.

Note: nixos-rebuild might become simply nixos at some point in the future depending on how #122715 moves forward.

Shortcut, if you had enough

You followed along but you got tired of all this already? Miss the time when you didn’t even had to partition your system via the terminal, and when your Manjaro was up & running in minutes?

You can jump ship here!

If you just want to hack together a simple GUI setup and add some packages the easiest possible way, then start adding this to your configuration.nix and come back to the rest of the sections when you are ready:

{ config, pkgs, ... }:

{
...
  # Enable the X11 windowing system.
  # We are configuring xfce as the desktop manager, and setting i3 as the window manager.
  # If you truly had enough, you can just use xfce for everything, reducing simplicity even further.
  services.xserver = {
    enable = true;
    desktopManager = {
      xterm.enable = false;
      xfce = {
        enable = true;
        noDesktop = true;
        enableXfwm = false;
      };
    };
    displayManager.defaultSession = "xfce+i3";
    windowManager.i3 = {
      enable = true;
      extraPackages = with pkgs; [
        dmenu
        i3status
        i3lock
        i3blocks
      ];
    };
  };
...
  # Define a user account. Don't forget to set a password with ‘passwd’.
  users.users.kfekete = {
    isNormalUser = true;
    extraGroups = [ "wheel" ]; # Enable ‘sudo’ for the user.
      packages = with pkgs; [
        firefox
  #     thunderbird
  #     your-favourite-app
      ];
  };
...
}

That’s it.

nixos-rebuild switch \
reboot

If you want to use the VM on e.g. a full HD screen with proper scaling, you can add this to your displayManager block:

      sessionCommands = ''
         ${pkgs.xorg.xrandr}/bin/xrandr -s '1920x1080'
       '';

No, I want to do this better

A better way to do manage a Nix system is using flakes and Home Manager.

Flakes enable you to create hermetic Nix projects where you can define dependencies and handle its output from other flakes.

Home Manager

Home Manager is a tool to manage user environments and it also supports flakes.

First, enable nix-command and flakes in your configuration.nix by adding this to you configuration:

nix.settings.experimental-features = [ "nix-command" "flakes" ];

Then build it:

nixos-rebuild test #to avoid a reboot

Add Home Manager to nix-channels:

nix-channel --add https://github.com/nix-community/home-manager/archive/release-22.05.tar.gz home-manager
nix-channel --update

Logout/login to populate $NIX_PATH with /.nix-defexpr/channels.

Add the channel to the imports section of your configuration:

imports = [ <home-manager/nixos> ];

Now, you can create your flake.nix, e.g.:

{
  description = "Home-Manager config flake thing";

  inputs = {
    nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable";
    home-manager = {
      url = "github:nix-community/home-manager";
      inputs.nixpkgs.follows = "nixpkgs";
    };
  };

  outputs = { self, nixpkgs, home-manager, ... }:
    let
      system = "aarch64-linux";
      pkgs = nixpkgs.legacyPackages.${system};
      hostname = "virtan1x";
    in
    {
      formatter.${system} = pkgs.nixpkgs-fmt;
      defaultPackage.aarch64-linux = home-manager.defaultPackage.aarch64-linux;
      nixosConfigurations.${hostname} = nixpkgs.lib.nixosSystem {
        inherit system;
        modules = [ ./nixos/configuration.nix ];
      };

      homeConfigurations.${hostname} = home-manager.lib.homeManagerConfiguration {
        inherit pkgs;

        modules = [
          ./home.nix
        ];
      };
    };
}

This is quite similar to what helmfile provides. You declare your inputs (e.g. repositories), and helmfile/Nix both can lock the state of your choice with a .lock file.

Note: Handling multiple architectures and machines in the config above in a DRY way is quite straightforward. I am planning to do follow-up pieces covering a setup like that.

Now the following command will build & activate your Home Manager config:

nix run . switch

Note: this is working because we set the defaultPackages to the packages of Home Manager. This basically makes running a home-manager switch with a soon to be installed Home Manager possible. Thanks to Ole Kristian Pedersen!

If it’s successful, you should see something like this:

Starting Home Manager activation
Activating checkFilesChanged
Activating checkLinkTargets
Activating writeBoundary
Activating installPackages
installing 'home-manager-path'
building '/nix/store/y0rjnqiriq2x5sxn1wqkip8sgdiljs2p-user-environment.drv'...
Activating linkGeneration
Creating profile generation 1
Creating home file links in /home/kfekete
Activating onFilesChange
Activating reloadSystemd

Notice the homeConfigurations block at the end of my last snippet. I am importing modules there from a file called home.nix.

When you first install home-manager, it will create a file like this:

{ config, pkgs, ... }:

{
  # Home Manager needs a bit of information about you and the
  # paths it should manage.
  home.username = "jdoe";
  home.homeDirectory = "/home/jdoe";

  # This value determines the Home Manager release that your
  # configuration is compatible with. This helps avoid breakage
  # when a new Home Manager release introduces backwards
  # incompatible changes.
  #
  # You can update Home Manager without changing this value. See
  # the Home Manager release notes for a list of state version
  # changes in each release.
  home.stateVersion = "22.05";

  # Let Home Manager install and manage itself.
  programs.home-manager.enable = true;
}

My home.nix is a slightly modified version of this template. Currently this is the diff:

diff myhome.nix defaulthome.nix 
6,7c6,7
<   home.username = "kfekete";
<   home.homeDirectory = "/home/kfekete";
---
>   home.username = "jdoe";
>   home.homeDirectory = "/home/jdoe";
21,27d20
< 
<   # https://github.com/nix-community/home-manager/issues/2942
<   nixpkgs.config.allowUnfreePredicate = (pkg: true);
< 
<   imports = [
<     ./modules
<   ];

Notice the modules/ import!

That’s an initial attempt to organize my applications.

I have these files there:

/home/kfekete/.config/nixpkgs/modules/
├── cli.nix
├── default.nix
├── neofetch.nix
├── git.nix
└── zsh.nix

If you import a directory, default.nix will be evaluated first with the following content:

{ ... }:

{
  imports = [
    # apps
    # ./app.nix # this can contain e.g. Firefox, Thunderbird, etc.

    # cli tools
    ./cli.nix
    ./git.nix
    ./neofetch.nix

    # shell
    # ./zsh.nix
    
    # dev tools
    # ./dev.nix
    
    # projects
    # ./project-a.nix
    # ./project-b.nix
  ];
}

Then this will import all my tools organized into multiple categories.

As I mentioned, this is just an initial setup, I am sure there is a more nixian way to do some of this, but this should be enough to get someone started with Nix(OS).

You can find a followup piece here, on how my configuration and tooling have evolved.