|
| 1 | +# Builder Host Design |
| 2 | + |
| 3 | +**Date:** 2026-03-07 |
| 4 | +**Status:** Approved |
| 5 | + |
| 6 | +## Goal |
| 7 | + |
| 8 | +Enable `./bloud start --build --install` to build the ISO locally on a |
| 9 | +pre-existing builder VM, then run the normal install cycle — eliminating the |
| 10 | +wait for GitHub Actions on every iteration. |
| 11 | + |
| 12 | +## Approach: `BLOUD_BUILDER_HOST` |
| 13 | + |
| 14 | +Add a `BLOUD_BUILDER_HOST` env var (e.g. `builder@192.168.0.105`) that points |
| 15 | +to any SSH-accessible host with Nix installed. When set, `doBuild()` SSHes |
| 16 | +directly to that host using the default SSH key — no Proxmox VM lifecycle, no |
| 17 | +dedicated keypair, no passwords. |
| 18 | + |
| 19 | +## What Gets Removed |
| 20 | + |
| 21 | +The Proxmox-managed builder VM (VMID 9998, Ubuntu cloud image) never worked |
| 22 | +reliably and is replaced entirely: |
| 23 | + |
| 24 | +- `createBuilderVM` — Ubuntu cloud image download + VM creation |
| 25 | +- `ensureBuilderKey` / `builderKeyPaths` — dedicated `~/.bloud/builder_rsa` keypair |
| 26 | +- `builderCfg` / `waitForBuilderSSH` / `builderExec` / `builderExecStream` |
| 27 | +- All `pveBuild*` constants (VMID, name, memory, cores, disk, image URL/file, key path) |
| 28 | +- `destroy-builder` command (makes no sense for externally managed VMs) |
| 29 | + |
| 30 | +## New Design |
| 31 | + |
| 32 | +### Configuration |
| 33 | + |
| 34 | +``` |
| 35 | +# .env |
| 36 | +BLOUD_BUILDER_HOST=builder@192.168.0.105 |
| 37 | +``` |
| 38 | + |
| 39 | +### `setup-builder` (repurposed) |
| 40 | + |
| 41 | +When `BLOUD_BUILDER_HOST` is set, provisions that host: |
| 42 | +1. SSH to the host |
| 43 | +2. Verify Nix is installed (error if not) |
| 44 | +3. Install Go and Node via `nix profile install nixpkgs#go nixpkgs#nodejs` |
| 45 | +4. Install git and rsync via apt if not present |
| 46 | +5. Touch `~/.bloud-provisioned` sentinel file (idempotent) |
| 47 | + |
| 48 | +### `doBuild()` |
| 49 | + |
| 50 | +1. Read `BLOUD_BUILDER_HOST` — error if not set |
| 51 | +2. SSH directly to host (default key, no sshpass) |
| 52 | +3. Rsync source from project root (excluding `build/`, `node_modules/`, `.direnv/`) |
| 53 | +4. Run build script on remote: |
| 54 | + - `export PATH="$HOME/.nix-profile/bin:$PATH"` (Go + Node from Nix profile) |
| 55 | + - Build host-agent binary (CGO_ENABLED=0, linux/amd64) |
| 56 | + - Build installer binary |
| 57 | + - Build frontend and installer-web |
| 58 | + - `git add -f build/` to make artifacts visible to Nix flake |
| 59 | + - `nix build .#packages.x86_64-linux.iso --no-link` |
| 60 | +5. Get ISO store path, SCP ISO to local `/tmp/bloud-built.iso` |
| 61 | +6. SCP ISO to Proxmox ISO storage |
| 62 | +7. Return — normal `cmdStartPVE` flow continues |
| 63 | + |
| 64 | +### SSH Helpers |
| 65 | + |
| 66 | +Two new helpers for direct-SSH (no dedicated key): |
| 67 | +- `builderSSHExec(host, cmd string) (string, error)` |
| 68 | +- `builderSSHExecStream(host, cmd string) error` |
| 69 | + |
| 70 | +Both use `ssh -o StrictHostKeyChecking=no -o ConnectTimeout=10 host cmd`. |
| 71 | + |
| 72 | +### Build Directory |
| 73 | + |
| 74 | +Remote build dir: `~/bloud` (relative to the builder user's home). |
| 75 | + |
| 76 | +## Security |
| 77 | + |
| 78 | +- All SSH uses key auth — keys were pre-installed via `ssh-copy-id` |
| 79 | +- No passwords stored or passed anywhere |
| 80 | +- `BLOUD_BUILDER_HOST` is the only credential needed |
| 81 | + |
| 82 | +## Usage |
| 83 | + |
| 84 | +```bash |
| 85 | +# .env |
| 86 | +BLOUD_BUILDER_HOST=builder@192.168.0.105 |
| 87 | + |
| 88 | +# First time: provision the builder |
| 89 | +./bloud setup-builder |
| 90 | + |
| 91 | +# Build ISO locally and run full install cycle |
| 92 | +./bloud start --build --install |
| 93 | + |
| 94 | +# Build ISO only (no install) |
| 95 | +./bloud start --build |
| 96 | +``` |
0 commit comments