Merge pull request 'move breakpoint checks out of default build' (#440) from Mic92-main into main

docs/machines.md

@@ -0,0 +1,115 @@
+# Managing NixOS Machines
+
+## Add Your First Machine
+
+To start managing a new machine, use the following commands to create and then list your machines:
+
+```shellSession
+$ clan machines create my-machine
+$ clan machines list
+my-machine
+```
+
+## Configure Your Machine
+
+In the example below, we demonstrate how to add a new user named `my-user` and set a password. This user will be configured to log in to the machine `my-machine`.
+
+### Creating a New User
+
+```shellSession
+# Add a new user
+$ clan config --machine my-machine users.users.my-user.isNormalUser true
+
+# Set a password for the user
+$ clan config --machine my-machine users.users.my-user.hashedPassword $(mkpasswd)
+```
+
+_Note: The `$(mkpasswd)` command generates a hashed password. Ensure you have the `mkpasswd` utility installed or use an alternative method to generate a secure hashed password._
+
+## Test Your Machine Configuration Inside a VM
+
+Before deploying your configuration to a live environment, you can create a virtual machine (VM) to test the settings:
+
+```shellSession
+$ clan vms create my-machine
+```
+
+This command creates a VM based on the configuration of `my-machine`, allowing you to verify changes in a controlled environment.
+
+## Installing a New Machine
+
+Clan CLI, in conjunction with [nixos-anywhere](https://github.com/nix-community/nixos-anywhere), provides a seamless method for installing NixOS on various machines.
+This process involves preparing a suitable hardware and disk partitioning configuration and ensuring the target machine is accessible via SSH.
+
+### Prerequisites
+
+- A running Linux system with SSH on the target machine is required. This is typically pre-configured for many server providers.
+- For installations on physical hardware, create a NixOS installer image and transfer it to a bootable USB drive as described below.
+
+## Creating a Bootable USB Drive on Linux
+
+To create a bootable USB flash drive with the NixOS installer:
+
+1. **Build the Installer Image**:
+
+   ```shellSession
+   $ nix build git+https://git.clan.lol/clan/clan-core.git#install-iso
+   ```
+
+2. **Prepare the USB Flash Drive**:
+
+   - Insert your USB flash drive into your computer.
+   - Identify your flash drive with `lsblk`. Look for the device with a matching size.
+   - Ensure all partitions on the drive are unmounted. Replace `sdX` in the command below with your device identifier (like `sdb`, etc.):
+
+     ```shellSession
+     sudo umount /dev/sdX*
+     ```
+
+3. **Write the Image to the USB Drive**:
+
+   - Use the `dd` utility to write the NixOS installer image to your USB drive:
+
+     ```shellSession
+     sudo dd bs=4M conv=fsync oflag=direct status=progress if=./result/stick.raw of=/dev/sdX
+     ```
+
+4. **Boot and Connect**:
+   - After writing the installer to the USB drive, use it to boot the target machine.
+   - The installer will display an IP address and a root password, which you can use to connect via SSH.
+
+### Finishing the installation
+
+With the target machine running Linux and accessible via SSH, execute the following command to install NixOS on the target machine, replacing `<target_host>` with the machine's hostname or IP address:
+
+```shellSession
+$ clan machines install my-machine <target_host>
+```
+
+## Update Your Machines
+
+Clan CLI enables you to remotely update your machines over SSH. This requires setting up a deployment address for each target machine.
+
+### Setting the Deployment Address
+
+Replace `host_or_ip` with the actual hostname or IP address of your target machine:
+
+```shellSession
+$ clan config --machine my-machine clan.networking.deploymentAddress root@host_or_ip
+```
+
+_Note: The use of `root@` in the deployment address implies SSH access as the root user. Ensure that the root login is secured and only used when necessary._
+
+### Updating Machine Configurations
+
+Execute the following command to update the specified machine:
+
+```shellSession
+$ clan machines update my-machine
+```
+
+You can also update all configured machines simultaneously by omitting the machine name:
+
+```shellSession
+$ clan machines update
+```

docs/quickstart.md

@@ -1,6 +1,6 @@
 # Initializing a New Clan Project
 
-## Create a new Clan flake
+## Create a new flake
 
 1. To start a new project, execute the following command to add the clan cli to your shell:
 
@@ -32,33 +32,9 @@ drwxrwxrwt root  root  139 B  12 seconds ago ../
 The `.clan-flake` marker file serves an optional purpose: it helps the `clan-cli` utility locate the project's root directory.
 If `.clan-flake` is missing, `clan-cli` will instead search for other indicators like `.git`, `.hg`, `.svn`, or `flake.nix` to identify the project root.
 
-## Add your first machine
+## What's next
 
-```shellSession
-$ clan machines create my-machine
-$ clan machines list
-my-machine
-```
-
-## Configure your machine
-
-In this example we crate a user named `my-user` that is allowed to login to the machine
-
-```shellSession
-# create a new user
-$ clan config --machine my-machine users.users.my-user.isNormalUser true
-
-# set some password
-$ clan config --machine my-machine users.users.my-user.hashedPassword $(mkpasswd)
-```
-
-## Test your machine config inside a VM
-
-```shellSession
-$ nix build .#nixosConfigurations.my-machine.config.system.build.vm
-...
-$ ./result/bin/run-nixos-vm
-```
+After creating your flake, you can check out how to add [new machines](./machines.md)
 
 ---
 
@@ -153,3 +129,7 @@ Absolutely, let's break down the migration step by step, explaining each action
 7. **Verify**: After the reboot, confirm that your system is running with the new configuration, and all services and applications are functioning as expected.
 
 By following these steps, you've successfully migrated your NixOS Flake configuration to include the `clan-core` input and adapted the `outputs` section to work with Clan Core's new machine provisioning method.
+
+## What's next
+
+After creating your flake, you can check out how to add [new machines](./machines.md)

nixosModules/hidden-ssh-announce.nix

@@ -37,7 +37,7 @@
       wantedBy = [ "multi-user.target" ];
       serviceConfig = {
         # ${pkgs.tor}/bin/torify
-        ExecStart = pkgs.writers.writeDash "announce-hidden-service" ''
+        ExecStart = pkgs.writeShellScript "announce-hidden-service" ''
           set -efu
           until test -e ${config.services.tor.settings.DataDirectory}/onion/hidden-ssh/hostname; do
             echo "still waiting for ${config.services.tor.settings.DataDirectory}/onion/hidden-ssh/hostname"

nixosModules/installer/default.nix

@@ -19,26 +19,32 @@
   '';
   hidden-ssh-announce = {
     enable = true;
-    script = pkgs.writers.writeDash "write-hostname" ''
+    script = pkgs.writeShellScript "write-hostname" ''
       set -efu
+      export PATH=${lib.makeBinPath (with pkgs; [ iproute2 coreutils jq qrencode ])}
+
       mkdir -p /var/shared
       echo "$1" > /var/shared/onion-hostname
-      ${pkgs.jq}/bin/jq -nc \
+      local_addrs=$(ip -json addr | jq '[map(.addr_info) | flatten | .[] | select(.scope == "global") | .local]')
+      jq -nc \
         --arg password "$(cat /var/shared/root-password)" \
-        --arg address "$(cat /var/shared/onion-hostname)" '{
-          password: $password, address: $address
-        }' > /var/shared/login.info
-      cat /var/shared/login.info |
-        ${pkgs.qrencode}/bin/qrencode -t utf8 -o /var/shared/qrcode.utf8
-      cat /var/shared/login.info |
-        ${pkgs.qrencode}/bin/qrencode -t png -o /var/shared/qrcode.png
+        --arg onion_address "$(cat /var/shared/onion-hostname)" \
+        --argjson local_addrs "$local_addrs" \
+        '{ password: $password, onion_address: $onion_address, local_addresses: $local_addrs }' \
+        > /var/shared/login.json
+      cat /var/shared/login.json | qrencode -t utf8 -o /var/shared/qrcode.utf8
     '';
   };
   services.getty.autologinUser = lib.mkForce "root";
   programs.bash.interactiveShellInit = ''
     if [ "$(tty)" = "/dev/tty1" ]; then
-      echo 'waiting for tor to generate the hidden service'
-      until test -e /var/shared/qrcode.utf8; do echo .; sleep 1; done
+      echo -n 'waiting for tor to generate the hidden service'
+      until test -e /var/shared/qrcode.utf8; do echo -n .; sleep 1; done
+      echo
+      echo "Root password: $(cat /var/shared/root-password)"
+      echo "Onion address: $(cat /var/shared/onion-hostname)"
+      echo "Local network addresses:"
+      ${pkgs.iproute}/bin/ip -brief -color addr | grep -v 127.0.0.1
       cat /var/shared/qrcode.utf8
     fi
   '';

pkgs/clan-cli/clan_cli/machines/update.py

@@ -87,7 +87,7 @@ def get_all_machines(clan_dir: Path) -> HostGroup:
         text=True,
     ).stdout
 
-    machines = json.loads(Path(machines_json).read_text())
+    machines = json.loads(Path(machines_json.rstrip()).read_text())
 
     hosts = []
     for name, machine_data in machines.items():

pkgs/clan-cli/default.nix

@@ -132,6 +132,14 @@ python3.pkgs.buildPythonApplication {
       ${checkPython}/bin/python -m pytest -m "not impure" -s ./tests
       touch $out
     '';
+    check-for-breakpoints = runCommand "breakpoints" { } ''
+      if grep --include \*.py -Rq "breakpoint()" ${source}; then
+        echo "breakpoint() found in ${source}:"
+        grep --include \*.py -Rn "breakpoint()" ${source}
+        exit 1
+      fi
+      touch $out
+    '';
   };
   passthru.clan-openapi = runCommand "clan-openapi" { } ''
     cp -r ${source} ./src
@@ -167,11 +175,6 @@ python3.pkgs.buildPythonApplication {
   '';
   checkPhase = ''
     PYTHONPATH= $out/bin/clan --help
-    if grep --include \*.py -Rq "breakpoint()" $out; then
-      echo "breakpoint() found in $out:"
-      grep --include \*.py -Rn "breakpoint()" $out
-      exit 1
-    fi
   '';
   meta.mainProgram = "clan";
   desktopItems = [