Preface

When I first started to experiment with NixOS, I did not expect that I will have it installed on 2 VMs, on my old laptop, two VPS servers running a self hosted Mastodon (well, this is actually not running anymore) and a Kali/ParrotOS alternative, plus on my Raspberry Pi. But here I am.

I also did not expected it to power most of the dev and regular, home-computer workflows, but currently this is exactly what it does.

My initial goal was to run a little self-hosted music server on my RPi4 on top of NixOS, so I can get some hand-son experience with it, but after trying it out, the idea of using NixOS as my daily-driver hit me.

The funny thing is that I only got to set up the music server as one of the last thing, but now I cannot imagine having it sitting on my desk and providing me all the features it has (it’s also an adblocker and a VPN).

UTM, Parallels, then VMware Fusion

Most of the time, I am running NixOS as a virtual machine, on top of macOS systems as I mentioned previously.

If you clicked the previous link (or, had some vague recollection of my post from the end of last year), you know that I started out with using UTM for this purpose.

UTM works great, but I was not able to make hardware acceleration work and that had some performance limitations. I am not doing anything crazy with these VMs, mostly just spinning up ephemeral Kubernetes clusters, and work with eBPF and Go. However, having the snappines of using a bare metal machine was something I missed greatly.

To give you some context, I had 7-10 FPS when running WebGL Aquarium on UTM with its default settings (1024x1024 canvas, 500 fish).

First, I tried out Parallels, which was working out of the box (I didn’t even have to install any guest tools, it was only needed if I wanted better copy-paste support), and was performing with stable 60 FPS in the Aquarium.

This was quite impressive. I haven’t seen VMs performing this well before this.

During their trial, I also tried out VMware Fusion as recently they also ironed out a few things regarding hw accelleration. By the time my trials were ending I’ve found that I had more control when running on VMware Fusion, and I also didn’t experience any storage corruption under my machine, which I unfortunately did with Parallels. They also annoyed me with ad popups so I am using Fusion ever since and enjoying the native-like performance (60 FPS here as well among the fishes).

dotfiles

My nixos configs have also evolved since I introduced them last time.

This is a structure very similar to the one I have for my config:

.
├── flake.nix
├── home.nix
├── machines
│   ├── virtan1x
│   │   ├── configuration.nix
│   │   └── hardware-configuration.nix
│   └── virtanix
│       ├── configuration.nix
│       └── hardware-configuration.nix
└── modules
    ├── cli.nix
    ├── default.nix
    ├── git.nix
    ├── neofetch.nix
    ├── neovim.nix
    ├── vmware-guest.nix
    └── firefox.nix

In reality I have more machines, and lots of extra modules.

The source of truth is my flake.nix file. It looks something like this:

{
  description = "Home-Manager config flake thing";

  inputs = {
    nixpkgs.url = "github:nixos/nixpkgs/nixos-23.11";
    nixpkgsUnstable.url = "github:NixOS/nixpkgs/master";
    home-manager = {
      url = "github:nix-community/home-manager/release-23.11";
      inputs.nixpkgs.follows = "nixpkgs";
    };
    flake-utils.url = "github:numtide/flake-utils";
  };

  outputs = inputs @ { self, nixpkgs, nixpkgsUnstable, flake-utils,  home-manager, ... }:

    flake-utils.lib.eachDefaultSystem
      (system:
        let
          pkgs = import nixpkgs { inherit system; };
        in
        {
          devShell = with pkgs; pkgs.mkShell {
            buildInputs = [
            ];
          };

        })
    // # <- concatenates Nix attribute sets
    {
      homeConfigurations = {
        # You can update home-manager for these targets like this:
        # home-manager --flake switch .#virtanix

        virtanix = inputs.home-manager.lib.homeManagerConfiguration {
          pkgs = inputs.nixpkgs.legacyPackages.x86_64-linux;
          modules = [ ./home.nix ];
          extraSpecialArgs = { pkgsUnstable = inputs.nixpkgsUnstable.legacyPackages.x86_64-linux; };
        };

        virtan1x = inputs.home-manager.lib.homeManagerConfiguration {
          pkgs = inputs.nixpkgs.legacyPackages.aarch64-linux;
          modules = [ ./home.nix ];
          extraSpecialArgs = { pkgsUnstable = inputs.nixpkgsUnstable.legacyPackages.aarch64-linux; };
        };
      };

      nixosConfigurations = {
        # You can rebuild these targets like this:
        ## sudo nixos-rebuild --flake .#virtanix switch

        virtanix = nixpkgs.lib.nixosSystem {
          system = "x86_64-linux";
          specialArgs = { common = self.common; inherit inputs; };
          modules = [ ./machines/virtanix/configuration.nix ];
        };

        virtan1x = nixpkgs.lib.nixosSystem {
          system = "aarch64-linux";
          specialArgs = { common = self.common; inherit inputs; };
          modules = [ ./machines/virtan1x/configuration.nix ];
        };
      };
  };
}

Here, I define all the various homeConfigurations (user and application specific configs via home-manager), and nixosConfigurations (system and hardware specific configs) that I am using across my fleet of machines.

My workflow is mostly using the commands that you can see as comments to update/upgrade my systems.

Currently I use the same home-manager config for all my machines, hence referencing the same home.nix in all homeConfigurations, but this setup would enable me to differentiate easily via that single file (if I want to keep my config simple).

The home.nix file can look like something like the example I used in my previous article.

Let’s say I want to install a new application or apply some changes for my ARM based machine, virtan1x.

After adding the additional config to my files in modules/, I can sync the changes like this:

home-manager --flake switch .#virtan1x

This is quite nice.

Bounty hunting, self-hosting Spotify, and surfing safe and ad-free while travelling

I always had issues with Kali Linux and ParrotOS when doing anything pentesting, CTFs, or hunting for bug bounties.

Apart from minor inconveniences such as using the default display or window managers, I frequently broke my system due to continuosly jumping between a wide range of Python versions and doing dirty fixes to ensure that the PoC at hand works fine.

I have spent countless hours (and I am not even doing this full-time) fixing my system after I broke it, and this was especially time consuming when I am jumping between infosec projects.

In hindsight, it’s trivial that NixOS can solve all of these issues by providing first class support for ad-hoc dev environments.

Yes, you can pyenv and Dockerfile all you want, but once you are over the core concepts (and starting out with nixos-shell is not even that hard), you will realize that you can adopt NixOS gradually, without learning every single bit of the language and the best practices right away.

Take a look at the following snippet:

# shell.nix
{ pkgs ? import <nixpkgs> { } }:
pkgs.mkShell {
  name = "dev-shell-for-someones-arcane-xss-thing";

  buildInputs = [
    (pkgs.python3.withPackages (ps: [
      ps.tld
      ps.fuzzywuzzy
      ps.requests
    ]))
  ];
}

That’s all you need to have all the packages available in you shell from requirements.txt.

You can activate the shell as simply as nix-shell.

Doing the very same thing with flakes is not that hard neither.

As I mentioned I am also self-hosting my music.

I use navidrome for this. Navidrome can function as a lightweight Subsonic-API compatible server, so I can even use any Subsonic clients.

As of now, I am satisfied with the default web UI, since I can even use it from my mobile browser (trough Tailscale).

Tailscale is what I use as a WireGuard based VPN, and I combine it with AdGuard Home to block tracking and ads at home and while travelling.

Conclusion

Leveraging the power of NixOS on machines other than my main workstations proved to be a game changer for me. Now I can quickly iterate between different projects, program versions without paying extra attention to not avoid breaking anything.

Another huge benefit is the ability to re-use modules across systems. It was trivial to take my monitoring modules from my Mastodon PoC, and import them into my Raspberry Pi config, and have metrics/logs collection in about two minutes.