collin/anvil

5c9b6b3ef8fc178f0dc9c1e53a01cce893c9f8f4 / CLAUDE.md

1	# anvil
2
3	A minimal, self-hosted git forge in Rust, built on gix (gitoxide).
4
5	## Design rules
6
7	- Pure gitoxide — never the git binary. The product (server, CI broker,
8	mirroring, deploy image) must not invoke or depend on a `git` executable.
9	When gix lacks a capability, implement the protocol on gix plumbing instead
10	of shelling out — e.g. gix can't push yet, so `anvil-git/src/push.rs` speaks
11	the send-pack wire format itself (with `gitserver-core` covering the server
12	side). Tests may use the git CLI, but only as a fixture/interop check, never
13	on the code path under test.
14
15	Dev setup: `git config core.hooksPath .githooks` — the pre-commit hook checks
16	that `rust-toolchain.toml`'s pin satisfies Cargo.toml's `rust-version` floor,
17	then runs `cargo +nightly fmt` (nightly, for the unstable options in
18	rustfmt.toml), `cargo sort-derives` (`cargo install cargo-sort-derives`), and
19	`cargo clippy --workspace --all-targets -- -D warnings`.
20
21	Every crate opts into `[workspace.lints]`, which turns on
22	`clippy::multiple_crate_versions` — so adding a dependency that drags in a
23	second copy of something already in the tree fails the hook. The duplicates
24	that exist today are listed in `clippy.toml` with a note on who is holding
25	each back; delete an entry when a bump makes it unnecessary rather than
26	letting the list rot.
27
28	The Rust version is set in exactly one place: `rust-toolchain.toml`. It
29	pins the toolchain and the musl cross target for every `cargo` invocation here;
30	`docker/runner/build.sh` parses `[toolchain] channel` out of it to bake the
31	same version into `anvil-runner:rust`; and `docker/Dockerfile` copies it in
32	ahead of the sources and lets rustup materialize it, so the deployed binary
33	compiles with that rustc too. Bumping Rust means editing that file and
34	Cargo.toml's floor (the hook catches you if you forget the second), then
35	rebuilding the images.
36
37	`Cargo.lock` is committed, and `docker/Dockerfile` builds `--locked`. The
38	image is built from the dependency set the commit records rather than whatever
39	resolves that day. A change that needs the lockfile updated fails the build
40	instead of drifting; commit the new lockfile with the change.
41
42	## Viewing attachments referenced in tasks
43
44	TODO items and tickets may embed an uploaded image as
45	`![...](/{owner}/{repo}/-/attachments/{hash})`. These bytes live outside git
46	history, so to actually see one, get the bytes and `Read` the file. Two ways:
47
48	Preferred — over git, no credentials. anvil mirrors every attachment into
49	`refs/anvil/attachments` (a flat tree of `<hash> → blob`, off the branch
50	namespace so a default pull never drags it down). From a clone, opt in once:
51
52	```
53	git fetch origin '+refs/anvil/attachments:refs/anvil/attachments'
54	git cat-file -p "refs/anvil/attachments:<hash>" > /tmp/att && # then Read /tmp/att
55	```
56
57	This uses the clone's existing git auth — no PAT — and works offline afterward.
58
59	Fallback — HTTP with a PAT (no clone, or the ref isn't fetched). Credentials
60	live in the git-ignored `.anvil-credentials` at the repo root (`ANVIL_BASE_URL` +
61	a read-only `ANVIL_TOKEN`); `source` it, then:
62
63	```
64	curl -fsS -H "Authorization: Bearer $ANVIL_TOKEN" \
65	"$ANVIL_BASE_URL/{owner}/{repo}/-/attachments/{hash}" -o /tmp/att && # Read it
66	```
67
68	The PAT is read-only (GET/HEAD only), safe to hold. If neither the ref nor
69	`.anvil-credentials` is available, ask the user rather than guessing.
70
71	Attachments are often phone screenshots larger than the `Read` tool's 256KB
72	cap. Downscale before reading (macOS `sips`):
73
74	```
75	sips -Z 1100 -s formatOptions 70 /tmp/att --out /tmp/att-small.jpg # then Read that
76	```
77
78	note that we can't use portless for this app because of a websocets bug with http/2
79
80	## Remote runners: where this stands
81
82	anvil does not execute CI. Runners dial out, claim jobs, and run them on
83	their own Docker daemon. Design and rationale: `docs/remote-runners.md`.
84
85	M1 is done: `anvil-job` (wire format), `anvil-docker` (connect/ensure_image),
86	`anvil-worker` (the runner binary), five endpoints under `/-/runner/`,
87	in-memory leases, and no Docker socket on the deployed container.
88
89	M2 is done: `platform` in `.anvil/ci.toml`, a `[ci] platform` default, and
90	two-tier routing — a claiming runner is offered the runs that match its
91	architecture (or name none), then the runs no connected runner is native to,
92	so a lone arm64 Mac still runs `linux/amd64` pipelines under Rosetta instead of
93	stranding them. `Dispatch` tracks who is connected from claims and heartbeats
94	(`RUNNER_TTL`, 5 min); `anvil-docker::check_platform` fails a job whose image
95	came back the wrong architecture.
96
97	Invariants to not break. These are the reasons the split is shaped the way
98	it is, and each is easy to undo by accident:
99
100	- The runner never parses `.anvil/ci.toml`. Image resolution, the allowlist
101	check and script assembly happen in `build_job`, so a runner cannot widen
102	what it is permitted to run.
103	- `store_artifact` stays server-side. The runner uploads a raw tar; `browse`
104	decides whether that becomes a servable directory tree, which is anvil's
105	call, not a runner's.
106	- A job's secrets live on its lease, not the vault, once dispatched.
107	`Vault::take` fails after the repo's unlock TTL lapses, so re-reading at
108	finish time would silently skip log masking on exactly the long runs whose
109	logs most need it.
110	- `anvil-job` depends on serde and nothing else. It exists so the runner does
111	not link toasty, SQLite and gix.
112
113	Next, in order:
114
115	1. M3, the deploy agent. Image builds cannot be CI jobs, because job
116	containers get no Docker socket by design. `docker build --platform
117	linux/amd64` plus `docker push` belongs to a separate process on the build
118	host at a different trust level, triggered by the existing `deploy_webhook`.
119	Folding it into the dial-out channel as a privileged "publish" job kind,
120	authorized by the `is_deploy_target` check that already scopes CD to one
121	repo, would remove the last inbound path to the build host.
122	2. CI concurrency. "One job at a time" was a property of the old in-process
123	loop and is gone. Nothing bounds in-flight jobs now beyond how many runners
124	exist. Needs a `max_concurrent` equivalent — more pressing now that platform
125	routing makes a second runner worth having.
126	3. Per-runner credentials. One shared `[ci] runner_token` means one
127	revocation for every runner, no `last_used_at`, and any holder can claim any
128	job and receive its secrets. Wants the API-token write scope first (see
129	TODO.md).
130	4. Live logs. The result POST is a single write, matching the old
131	behaviour. Streaming needs chunked append with offsets and a UI that
132	tolerates gaps. Newly worth doing now that a producer exists.
133
134	Operationally: nothing runs until a runner is started and `[ci]
135	runner_token` is set; queued runs just sit, with a warning logged at startup.
136	Setting one up on the Mac mini (build, Rosetta, the launchd agent in
137	`deploy/worker/`) is written out in `docs/remote-runners.md` § Isolation on
138	macOS. Locally, `compose.override.yaml` runs two of them in containers
139	(`docker compose up -d --build`) so concurrency and routing are testable on one
140	machine; that mount is a development-only concession, never production's. Who
141	is connected, and what each is running, is at `/-/admin/runners` — presence
142	comes from ordinary claims
143	and heartbeats rather than a ping of its own, so anything watching for a runner
144	to go quiet has to allow a full claim poll first.
145	Agent sessions still drive Docker locally and are the one thing the dropped
146	socket mount gives up. They are off by default.
147
148	Current status, resume notes, the agreed next steps (a/b/c), and the roadmap live
149	in the TODO. Read it first:
150