Deploying anvil on hagrid

anvil runs as a single Docker container at anvil.richardscollin.com, fronted by hagrid's Caddy reverse proxy.

Deploying: compose.yaml + hag

compose.yaml at the repo root describes the deployment — the image, the network, the volume, the published SSH port. Running it on hagrid is hag, the shared deploy tool for every project on that host: it builds the image, pushes it to registry.vibe.richardscollin.com, copies compose.yaml up, then pulls and recreates the container there.

compose.override.yaml sits next to it for local development — loopback ports, the dev config, the Docker socket agent sessions need. Compose merges it automatically when you run docker compose in the repo root, and hag always passes -f compose.yaml explicitly, so none of it ever reaches hagrid. See the README for the local loop.

anvil needs one step in front of that, because the image carries a prebuilt binary — deploy/deploy.sh is the whole deploy:

./deploy/deploy.sh          # cross-compile, build, push, restart on hagrid
./deploy/deploy.sh -n       # dry run: print every step, change nothing

which is:

1. deploy/build.sh — cargo zigbuild --release --target x86_64-unknown-linux-musl cross-compiles a fully static binary natively (~2 min, no emulation) and stages it at docker/anvil/anvild,
2. hag deploy — docker compose build --push builds the thin image that just COPYs that binary in (the Dockerfile compiles nothing), pushes it, then drives the host.

Do not build on the VPS — a release build needs ~2–4 GB peak and OOMs a cheap, swap-less droplet. The VPS never compiles anything; it only pulls.

One-time toolchain setup:

dnf install zig          # or: brew install zig
cargo install cargo-zigbuild
docker login registry.vibe.richardscollin.com    # on this machine and hagrid

The Rust toolchain itself needs no setup step: rust-toolchain.toml pins the version and the x86_64-unknown-linux-musl target, and rustup installs both on the first cargo invocation in the repo. That file is the one place the Rust version is set — docker/runner/build.sh reads it to bake the same toolchain into anvil-runner:rust.

Every build is tagged twice: with the short git sha of the checkout (plus a -dirty suffix when the working tree has uncommitted changes) and with latest. The deploy pins the host to the exact sha it just pushed, so hag status always names the running build. To roll back, run an older tag on the host:

ssh hagrid 'cd anvil && IMAGE_TAG=<sha> docker compose up -d --no-build'

Managing it afterwards needs no ssh and no remembering where it lives:

hag status      # what the host is running
hag logs -f     # its logs
hag restart     # recreate its containers
hag down        # stop it
hag config      # what hag resolved for this project
hag ls          # every compose project on the host

4. What compose.yaml runs

image: registry.vibe.richardscollin.com/anvil:${IMAGE_TAG:-latest}
container_name: anvil
restart: unless-stopped
ports: ["165.232.162.167:22:2222"]
volumes: [anvil-data:/data]
networks: [hagrid]

• networks: [hagrid] — an external network, so Caddy can reach anvil:3000. The web port is deliberately not published.
• ports — git-over-SSH, published on the droplet's default public IPv4 only. The reserved IP (137.184.249.48) arrives on the anchor address 10.15.0.6, where the host's own sshd listens, so binding one specific IP keeps the two off each other. ANVIL_SSH_BIND_IP / ANVIL_SSH_PORT in the host's .env override it.
• anvil-data:/data — a named volume holding the SQLite DB, the bare repos and the persistent SSH host key. Pinned with name: to that exact string, so compose adopts the volume the pre-compose deploys created instead of deriving a fresh empty anvil_anvil-data from the project name. A named volume (not a host bind mount) keeps it owned by the in-container anvil user.
• No Docker socket. anvil does not execute CI — runners dial in and run jobs on their own daemons (see docs/remote-runners.md). The container has no reason to reach Docker, so the mount is gone, and with it the root-equivalent hold the internet-facing process used to have on the host.

Agent sessions are the exception. They still drive Docker locally, so turning them on means putting the socket back — a volumes: entry for /var/run/docker.sock and a matching group_add: — and accepting that the container again has root-equivalent control of the host. They are off by default and absent from deploy/anvil.toml; read docs/untrusted-mode.md §7 before changing that.

The baked config lives at /etc/anvil/anvil.toml (see deploy/anvil.toml). Override it by bind-mounting your own file over that path.

Host-side config. compose.yaml loads ~/anvil/.env on the host if it exists, and skips it if not — hag never copies a local .env up, so the workstation and the host keep separate config. Today that file carries the SSO client secret (§ docs/oidc.md):

ssh hagrid 'printf "ANVIL_OIDC_CLIENT_SECRET=%s\n" \
    "$(tr -d "[:space:]" < ~/.config/anvil/oidc-client-secret)" > ~/anvil/.env \
    && chmod 600 ~/anvil/.env'

Leave the variable out entirely rather than setting it empty: empty would override the baked config and turn a confidential OIDC client into a public one.

Migrating from the docker run deploys. The old container was created by docker run, so it carries no compose labels; compose reads container_name: anvil as taken by a foreign container and refuses to adopt it. Clear it once, before the first compose deploy:

ssh hagrid 'docker rm -f anvil'

The anvil-data volume is untouched by that and is what the new container picks up.

5. First run: create your account, key, and the repo

# admin user
docker exec anvil anvild -c /etc/anvil/anvil.toml \
    user create collin --email you@example.com --password '<password>' --admin

# your SSH public key (so you can push over SSH)
docker exec -i anvil anvild -c /etc/anvil/anvil.toml \
    user add-key collin --title laptop --key "$(cat ~/.ssh/id_ed25519.pub)"

# the anvil repo itself
docker exec anvil anvild -c /etc/anvil/anvil.toml repo create collin/anvil \
    --description "a minimal git forge in Rust"

(You can also create the user, add keys, and create repos from the web UI once signed in — the CLI is just convenient for the first admin.)

6. Self-host anvil on anvil

From your local anvil checkout:

git remote add origin ssh://git@anvil.richardscollin.com/collin/anvil.git
git push -u origin main

Then browse it at https://anvil.richardscollin.com/collin/anvil. HTTPS clone also works: git clone https://anvil.richardscollin.com/collin/anvil.git (pushes over HTTPS require your account password as the git password).

7. CI: start a runner

anvil dispatches CI; it does not execute it. Nothing runs until a runner dials in, so this is a required step, not an optional one — see docs/remote-runners.md.

Set a shared secret in the config ([ci] runner_token), then on a machine with room to build — the Mac mini, not the droplet:

cargo build --release --bin anvil-worker
cp target/release/anvil-worker /usr/local/bin/

anvil-worker --url https://anvil.richardscollin.com \
             --token "$ANVIL_RUNNER_TOKEN" --name macmini

deploy/worker/com.anvil.worker.plist runs it under launchd on macOS. Run it natively there rather than in a container: a containerized runner needs the Docker socket mounted into it, which rebuilds the hole section 4 just removed.

Isolation is not weaker for being on a Mac. Docker Desktop runs every container inside one Linux VM, so --cap-drop=ALL, no-new-privileges and the cgroup limits are enforced by the same kernel primitives as on Linux — with the VM as an extra boundary a bare-metal Linux host does not have.

Architecture. The Mac is arm64 and hagrid is x86_64, so a job running cargo test on the runner tests an architecture you do not ship. Images are unaffected (deploy/build.sh cross-compiles, and compose.yaml pins platforms: [linux/amd64]), and per-pipeline platform selection is designed but not yet wired to a config key.

8. The redeploy webhook (CD)

Any repo with a .anvil/ci.yml runs CI on push. A pipeline is just an image plus steps:

image: anvil-runner:rust
steps:
  - name: test
    run: cargo test --workspace
  - run: cargo build --release

Runs show up at /{owner}/{repo}/ci, with a per-commit status badge on the commit list and a full log on each run's page.

Job sandbox. The runner is the only Docker client on its machine; the job container gets no socket, no mounts (the checkout is uploaded as a tar), all capabilities dropped, and no-new-privileges. Resource bounds come from the forge's [ci] config and travel with each job, so tightening one does not need runners redeployed: memory_mb (default 2048), cpus (2), pids_limit (512), timeout_secs (1800, then the container is killed), network (true), run_as (empty = image default), and allowed_images (empty = any; a tagless entry like "rust" allows every tag). The allowlist is applied on the forge when the job is built, so a runner cannot widen it. See docs/untrusted-mode.md for the threat model and what this does/doesn't protect against.

Secrets now leave the host. A pipeline's secrets are sent to the runner with its job and sit in plaintext in a container on a machine anvil does not own. Scope repository secrets accordingly, and treat a runner host as being as trusted as the forge itself.

Continuous deployment is deliberately scoped to one repository. On a successful run of deploy_branch (default main) in the repo named by [ci] deploy_repo, anvil POSTs JSON to [ci] deploy_webhook:

{ "repo": "collin/anvil", "ref": "main", "commit": "<oid>", "run_id": 42 }

No other repo can trigger this, even with passing CI. The webhook target is a host-local plaintext receiver (HTTPS is unsupported, to keep the build TLS-free) — typically a tiny script-runner on hagrid that, on a verified request, runs the actual redeploy. Verify the X-Anvil-Deploy-Secret header (set [ci] deploy_secret) before doing anything. Because anvil cross-compiles (section 3), "redeploy anvil" usually means: the receiver pulls the freshly built image and runs docker compose up -d --no-build in ~/anvil — it does not build in-place.

Building an image is not a CI job. Job containers get no Docker socket by design, so docker build/docker push cannot happen inside a pipeline. That work belongs to a separate deploy agent on the build host, triggered by this webhook, running outside the sandbox with Docker access — a different trust level from the runner, and deliberately a different process.

The deploy receiver runs with whatever privileges you give it — keep it minimal, secret-gated, and bound to localhost / the Docker host gateway only.

Operations

• Update: ./deploy/deploy.sh (cross-compile + push + restart in one command); or hag restart to recreate the container from the image already on the host (the anvil-data volume persists). NOTE: adding new DB tables in a future version won't auto-apply to an existing database yet (Toasty migration support is pending) — the git repos on disk are unaffected, but repo metadata in SQLite may need recreating until migrations land.
• Backup: snapshot the anvil-data volume, e.g. docker run --rm -v anvil-data:/data -v "$PWD":/out debian:bookworm-slim \ tar czf /out/anvil-data.tgz -C /data .
• Logs: hag logs -f (or docker logs -f anvil on the host).
• System routes live under /-/ (e.g. sign in at https://anvil.richardscollin.com/-/login); /{username} is the user/repo namespace.