Deploying anvil on hagrid

anvil runs as a single Docker container at anvil.richardscollin.com, fronted by hagrid's Caddy reverse proxy.

Deploying: compose.yaml + hag

compose.yaml at the repo root describes the deployment — the image, the network, the volume, the published SSH port. Running it on hagrid is hag, the shared deploy tool for every project on that host: it builds the image, pushes it to registry.vibe.richardscollin.com, copies compose.yaml up, then pulls and recreates the container there.

compose.override.yaml sits next to it for local development — loopback ports, the dev config, the Docker socket agent sessions need. Compose merges it automatically when you run docker compose in the repo root, and hag always passes -f compose.yaml explicitly, so none of it ever reaches hagrid. See the README for the local loop.

hag deploy is the whole deploy:

hag deploy          # build, push, pull and recreate on hagrid
hag deploy -n       # dry run: print every step, change nothing

docker/Dockerfile compiles anvild itself, from the build context, in a builder stage that cross-links to a fully static x86_64-unknown-linux-musl binary with cargo-zigbuild. The builder runs on your architecture (--platform=$BUILDPLATFORM) and only the linker targets amd64, so an arm64 Mac produces hagrid's image at native speed rather than emulating rustc. The runtime stage then COPY --from=builders the binary onto ubuntu:26.04.

That the build context is the only input is the point: there is no staged artifact in the working tree, so hag deploy cannot ship a binary older than the commit it tags. It replaces the deploy/build.sh + deploy/deploy.sh pair that used to sit in front of it.

Do not build on the VPS — a release build needs ~2–4 GB peak and OOMs a cheap, swap-less droplet. The VPS never compiles anything; it only pulls.

One-time setup is now just the registry:

docker login registry.vibe.richardscollin.com    # on this machine and hagrid

No host Rust toolchain, no zig, no cargo-zigbuild — the builder stage carries all three, with zig pinned by version and sha256 and cargo-zigbuild pinned by version. The Rust version still comes from rust-toolchain.toml and nowhere else: the Dockerfile copies that file in ahead of the sources and lets rustup materialize the pinned channel and the musl target, exactly as docker/runner/build.sh reads it to bake the same toolchain into anvil-runner:rust.

Cargo.lock is committed and the builder passes --locked, so the image is built from the resolved dependency set the commit records — a build that would need to change the lockfile fails instead of silently drifting.

Build caching. The builder mounts BuildKit caches over the cargo registry and target/, so only what you changed recompiles. They are not part of any layer, which is why the binaries are copied out to /out inside the same RUN. docker builder prune throws these away and the next build is cold.

Every build is tagged twice: with the short git sha of the checkout (plus a -dirty suffix when the working tree has uncommitted changes) and with latest. The deploy pins the host to the exact sha it just pushed, so hag status always names the running build. To roll back, run an older tag on the host:

ssh hagrid 'cd compose/anvil && IMAGE_TAG=<sha> docker compose up -d --no-build'

Managing it afterwards needs no ssh and no remembering where it lives:

hag status      # what the host is running
hag logs -f     # its logs
hag restart     # recreate its containers
hag down        # stop it
hag config      # what hag resolved for this project
hag ls          # every compose project on the host

4. What compose.yaml runs

image: registry.vibe.richardscollin.com/anvil:${IMAGE_TAG:-latest}
container_name: anvil
restart: unless-stopped
ports: ["165.232.162.167:22:2222"]
volumes: [anvil-data:/data]
networks: [hagrid]

• networks: [hagrid] — an external network, so Caddy can reach anvil:3000. The web port is deliberately not published.
• ports — git-over-SSH, published on the droplet's default public IPv4 only. The reserved IP (137.184.249.48) arrives on the anchor address 10.15.0.6, where the host's own sshd listens, so binding one specific IP keeps the two off each other. ANVIL_SSH_BIND_IP / ANVIL_SSH_PORT in the host's .env override it.
• anvil-data:/data — a named volume holding the SQLite DB, the bare repos and the persistent SSH host key. Pinned with name: to that exact string, so compose adopts the volume the pre-compose deploys created instead of deriving a fresh empty anvil_anvil-data from the project name. A named volume (not a host bind mount) keeps it owned by the in-container anvil user.
• No Docker socket. anvil does not execute CI — runners dial in and run jobs on their own daemons (see docs/remote-runners.md). The container has no reason to reach Docker, so the mount is gone, and with it the root-equivalent hold the internet-facing process used to have on the host.

Agent sessions are the exception. They still drive Docker locally, so turning them on means putting the socket back — a volumes: entry for /var/run/docker.sock and a matching group_add: — and accepting that the container again has root-equivalent control of the host. They are off by default and absent from deploy/anvil.toml; read docs/untrusted-mode.md §7 before changing that.

The baked config lives at /etc/anvil/anvil.toml (see deploy/anvil.toml). Override it by bind-mounting your own file over that path.

Host-side config. compose.yaml loads .env from the project directory on the host (~/compose/anvil) if it exists, and skips it if not — hag never copies a local .env up, so the workstation and the host keep separate config. Today that file carries the SSO client secret (§ docs/oidc.md):

ssh hagrid 'printf "ANVIL_OIDC_CLIENT_SECRET=%s\n" \
    "$(tr -d "[:space:]" < ~/.config/anvil/oidc-client-secret)" \
    > ~/compose/anvil/.env && chmod 600 ~/compose/anvil/.env'

Leave the variable out entirely rather than setting it empty: empty would override the baked config and turn a confidential OIDC client into a public one.

Migrating from the docker run deploys. Done on 2026-08-27, kept here because the failure mode is not obvious. The old container was created by docker run, so it carried no compose labels; compose read container_name: anvil as taken by a foreign container and refused to adopt it. Clearing it once (ssh hagrid 'docker rm -f anvil') left the anvil-data volume untouched, which is what the compose container picked up. hag also moved projects from ~/<name> to ~/compose/<name> and refuses to deploy while state sits at the old path — it prints the mv to run.

5. First run: create your account, key, and the repo

# admin user
docker exec anvil anvild -c /etc/anvil/anvil.toml \
    user create collin --email you@example.com --password '<password>' --admin

# your SSH public key (so you can push over SSH)
docker exec -i anvil anvild -c /etc/anvil/anvil.toml \
    user add-key collin --title laptop --key "$(cat ~/.ssh/id_ed25519.pub)"

# the anvil repo itself
docker exec anvil anvild -c /etc/anvil/anvil.toml repo create collin/anvil \
    --description "a minimal git forge in Rust"

(You can also create the user, add keys, and create repos from the web UI once signed in — the CLI is just convenient for the first admin.)

6. Self-host anvil on anvil

From your local anvil checkout:

git remote add origin ssh://git@anvil.richardscollin.com/collin/anvil.git
git push -u origin main

Then browse it at https://anvil.richardscollin.com/collin/anvil. HTTPS clone also works: git clone https://anvil.richardscollin.com/collin/anvil.git (pushes over HTTPS require your account password as the git password).

7. CI: start a runner

anvil dispatches CI; it does not execute it. Nothing runs until a runner dials in, so this is a required step, not an optional one — see docs/remote-runners.md.

Set a shared secret in the config ([ci] runner_token), then on a machine with room to build — the Mac mini, not the droplet:

cargo build --release --bin anvil-worker
cp target/release/anvil-worker /usr/local/bin/

anvil-worker --url https://anvil.richardscollin.com \
             --token "$ANVIL_RUNNER_TOKEN" --name macmini

deploy/worker/com.anvil.worker.plist runs it under launchd on macOS. Run it natively there rather than in a container: a containerized runner needs the Docker socket mounted into it, which rebuilds the hole section 4 just removed.

Isolation is not weaker for being on a Mac. Docker Desktop runs every container inside one Linux VM, so --cap-drop=ALL, no-new-privileges and the cgroup limits are enforced by the same kernel primitives as on Linux — with the VM as an extra boundary a bare-metal Linux host does not have.

Architecture. The Mac is arm64 and hagrid is x86_64, so a job running cargo test on the runner tests an architecture you do not ship. Images are unaffected (the builder stage cross-compiles, and both it and compose.yaml pin amd64), and per-pipeline platform selection is designed but not yet wired to a config key.

8. The redeploy webhook (CD)

Any repo with a .anvil/ci.toml runs CI on push. A pipeline is just an image plus steps:

image = "anvil-runner:rust"

[[steps]]
name = "test"
run = "cargo test --workspace"

[[steps]]
run = "cargo build --release"

Runs show up at /{owner}/{repo}/ci, with a per-commit status badge on the commit list and a full log on each run's page.

Job sandbox. The runner is the only Docker client on its machine; the job container gets no socket, no mounts (the checkout is uploaded as a tar), all capabilities dropped, and no-new-privileges. Resource bounds come from the forge's [ci] config and travel with each job, so tightening one does not need runners redeployed: memory_mb (default 2048), cpus (2), pids_limit (512), timeout_secs (1800, then the container is killed), network (true), run_as (empty = image default), and allowed_images (empty = any; a tagless entry like "rust" allows every tag). The allowlist is applied on the forge when the job is built, so a runner cannot widen it. See docs/untrusted-mode.md for the threat model and what this does/doesn't protect against.

Secrets now leave the host. A pipeline's secrets are sent to the runner with its job and sit in plaintext in a container on a machine anvil does not own. Scope repository secrets accordingly, and treat a runner host as being as trusted as the forge itself.

Continuous deployment is deliberately scoped to one repository. On a successful run of deploy_branch (default main) in the repo named by [ci] deploy_repo, anvil POSTs JSON to [ci] deploy_webhook:

{ "repo": "collin/anvil", "ref": "main", "commit": "<oid>", "run_id": 42 }

No other repo can trigger this, even with passing CI. The webhook target is a host-local plaintext receiver (HTTPS is unsupported, to keep the build TLS-free) — typically a tiny script-runner on hagrid that, on a verified request, runs the actual redeploy. Verify the X-Anvil-Deploy-Secret header (set [ci] deploy_secret) before doing anything. Because anvil cross-compiles (section 3), "redeploy anvil" usually means: the receiver pulls the freshly built image and runs docker compose up -d --no-build in ~/anvil — it does not build in-place.

Building an image is not a CI job. Job containers get no Docker socket by design, so docker build/docker push cannot happen inside a pipeline. That work belongs to a separate deploy agent on the build host, triggered by this webhook, running outside the sandbox with Docker access — a different trust level from the runner, and deliberately a different process.

The deploy receiver runs with whatever privileges you give it — keep it minimal, secret-gated, and bound to localhost / the Docker host gateway only.

Operations

• Update: hag deploy (compile + push + restart in one command); or hag restart to recreate the container from the image already on the host (the anvil-data volume persists). NOTE: adding new DB tables in a future version won't auto-apply to an existing database yet (Toasty migration support is pending) — the git repos on disk are unaffected, but repo metadata in SQLite may need recreating until migrations land.
• Backup: snapshot the anvil-data volume, e.g. docker run --rm -v anvil-data:/data -v "$PWD":/out debian:bookworm-slim \ tar czf /out/anvil-data.tgz -C /data .
• Logs: hag logs -f (or docker logs -f anvil on the host).
• System routes live under /-/ (e.g. sign in at https://anvil.richardscollin.com/-/login); /{username} is the user/repo namespace.