anvilsign in

collin/anvil

89e731e4ddbb18faad73f7740967fd252af8739b / TODO.md

BoardRenderedSource

1# Misc TODO
2
3- [ ] do we have a way to view the source in a repo by branch or at a given commit?
4- [ ] design a CI artifact system, keyed by commit
5 - every push triggers CI on the tip commit (already true); a run should be
6 able to produce artifacts
7 - artifacts are stored per-commit and served from anvil (download from the
8 run page / commit page; latest-on-branch alias would be nice)
9 - jobs can declare how to extract metadata from artifacts (e.g. sizes,
10 test/coverage numbers, version strings) so it can be surfaced in the UI
11 next to the run/commit
12 - sketch needed: where artifacts live on disk, retention/GC, how
13 `.anvil/ci.yml` declares artifact paths + metadata extractors, and how
14 the broker gets files out of the sandboxed container
15- [ ] repo mirroring to/from GitHub
16 - push mirror: pushes to an anvil repo get forwarded to a configured
17 GitHub remote
18 - pull mirror (maybe): a repo that virtually mirrors a GitHub repo and
19 just displays it here — periodically fetched, read-only on the anvil
20 side
21
22---
23
24# Error in git push _(FIXED 2026-06-10, uncommitted)_
25
26**Root cause:** every push after the first sends a *thin pack* — deltas whose
27base objects aren't in the pack (the server already has them), referenced by
28object id (`REF_DELTA`). `vendor/gitserver-core/src/receive_pack.rs::write_pack`
29passed `None` as `thin_pack_base_object_lookup` to
30`gix_pack::Bundle::write_to_directory`, so gix couldn't resolve the bases and
31aborted. First-push-to-empty-repo worked because that pack is self-contained.
32
33**Fix:** pass the already-open `gix::Repository` as the lookup (it implements
34`gix_object::Find`). Regression test
35`receive_thin_pack_with_ref_deltas` builds a real thin pack via
36`git pack-objects --thin`, asserts it contains ref-deltas, and pushes it
37through `receive_pack`. Verified the test fails without the fix.
38
39Original report:
40
41```
42collin@mini ~/C/anvil (main)> git push
43Enter passphrase for key '/Users/collin/.ssh/id_ed25519':
44Enumerating objects: 117, done.
45Counting objects: 100% (117/117), done.
46Delta compression using up to 8 threads
47Compressing objects: 100% (62/62), done.
48Writing objects: 100% (66/66), 27.57 KiB | 3.94 MiB/s, done.
49Total 66 (delta 39), reused 0 (delta 0), pack-reused 0 (from 0)
50send-pack: unexpected disconnect while reading sideband packet
51fatal: the remote end hung up unexpectedly
52collin@mini ~/C/anvil (main) [128]>
53```
54
55server logs:
56
57```
5826-06-09T21:53:46.466577Z INFO anvil_ssh: ssh auth: accepted key SHA256:Rg41caN7vw2WYYxiJN6lrIlX0DTXYF0rC2QzKZW1tB0 (user 1)
592026-06-09T21:53:46.635946Z INFO anvil_ssh: ssh git-receive-pack on collin/anvil.git (user Some(1))
602026-06-09T21:53:46.805146Z ERROR anvil_ssh: git ssh git-receive-pack: protocol error: failed to write incoming pack: Ref delta objects are not supported as there is no way to look them up. Resolve them beforehand.
612026-06-09T21:54:28.571834Z INFO anvil_ssh: ssh auth: rejected unknown key SHA256:ZlWZyHqspqFeUQV84qaXtDQq4gcA33dR7y8dYbeg9u8
622026-06-09T21:54:35.565597Z INFO anvil_ssh: ssh auth: accepted key SHA256:Rg41caN7vw2WYYxiJN6lrIlX0DTXYF0rC2QzKZW1tB0 (user 1)
632026-06-09T21:54:35.676113Z INFO anvil_ssh: ssh git-receive-pack on collin/anvil.git (user Some(1))
642026-06-09T21:54:36.268520Z ERROR anvil_ssh: git ssh git-receive-pack: protocol error: failed to write incoming pack: Ref delta objects are not supported as there is no way to look them up. Resolve them beforehand.
652026-06-09T21:55:54.223033Z INFO anvil_ssh: ssh auth: rejected unknown key SHA256:ZlWZyHqspqFeUQV84qaXtDQq4gcA33dR7y8dYbeg9u8
662026-06-09T21:56:00.758384Z INFO anvil_ssh: ssh auth: accepted key SHA256:Rg41caN7vw2WYYxiJN6lrIlX0DTXYF0rC2QzKZW1tB0 (user 1)
672026-06-09T21:56:00.906553Z INFO anvil_ssh: ssh git-receive-pack on collin/anvil.git (user Some(1))
682026-06-09T21:56:01.067903Z ERROR anvil_ssh: git ssh git-receive-pack: protocol error: failed to write incoming pack: Ref delta objects are not supported as there is no way to look them up. Resolve them beforehand.
69```
70
71---
72
73# Session notes / resume point
74
75_Last updated: 2026-06-10 (second session). Working state is clean: `cargo
76build`, `cargo clippy --workspace`, `cargo fmt --all`, and `cargo test
77--workspace` all pass. Everything below is UNCOMMITTED (repo convention:
78commit only when asked). **All top-of-file TODO items are done.**_
79
80## Done this session (2026-06-10, second session)
81
82- **UI quick wins** (`crates/anvil-web/src/ui.rs`) — latest-commit bar on the
83 repo page (sha + subject + author/time, attached above the file box, links
84 to the commit); profile page no longer shows the email; repo header reads
85 `owner / repo` without the leading `anvil /`.
86- **(c) sandboxed CI broker** — job containers now run with `cap_drop=ALL` +
87 `no-new-privileges` unconditionally, plus config-driven `pids_limit` (512),
88 `memory_mb`+swap (2048), `cpus` (2), wall-clock `timeout_secs` (1800,
89 force-removed on expiry), optional `network = false`, `run_as`, and an
90 `allowed_images` allowlist (empty = any; tagless entry allows all tags).
91 `CiConfig` in `crates/anvil-core/src/config.rs` (with `image_allowed` test);
92 `execute()` in `crates/anvil-ci/src/lib.rs`. Read-only rootfs deliberately
93 skipped (workspace lives in the container fs; no volumes ever attached).
94 Docs: `DEPLOY.md` §7, `anvil.example.toml` `[ci]`.
95- **(b) threat model** — `docs/untrusted-mode.md`: severity-ranked analysis
96 (CI containment, pages/stored-XSS origin, git resource exhaustion,
97 registration anti-abuse, authz granularity, webhook SSRF), the already-right
98 list, and the stance: single-tenant supported, untrusted gated on items 1–4.
99- **Pages hosting** — `crates/anvil-web/src/pages.rs`: serves a repo's `pages`
100 branch at `/{owner}/{repo}/pages/...`; top-level dirs are separate sites
101 (rustdoc, book, …); `index.html` resolution with trailing-slash redirect so
102 relative links work; extension→content-type map + `nosniff`; listing page
103 with publish hint; "Pages" button on the repo header. Visibility follows the
104 repo (private → 404). Publish with `git push origin <built-branch>:pages`.
105
106## Done earlier (same day, first session)
107
108- **CI UI** — runs list `/{owner}/{repo}/ci`, run-detail (status/timing/log),
109 per-commit status badges, "CI" nav link. (`crates/anvil-web/src/ui.rs`)
110- **CD redeploy webhook** — on a green run of `[ci] deploy_branch` in the single
111 `[ci] deploy_repo`, POST to `[ci] deploy_webhook` (`X-Anvil-Deploy-Secret`
112 header). Scoped to ONE repo. `CiConfig` in `crates/anvil-core/src/config.rs`;
113 `deploy()` in `crates/anvil-ci/src/lib.rs`. Docs: `DEPLOY.md` §7,
114 `deploy/anvil.toml`. `reqwest` added with NO TLS feature (keeps musl
115 cross-compile aws-lc-free).
116- **Docker socket on hagrid** — `deploy/run.sh` mounts it + `--group-add`s the
117 gid for the non-root user; caveat in `DEPLOY.md` §4.
118- **Toasty ORM cleanup** — `ci.rs` `list_by_repo`/`latest_for_commit`/
119 `queued_ids` now sort/limit/filter in SQL, not in memory. Verified by the new
120 `ordering_and_limit_run_in_the_database` test. (Sweep: these were the only
121 real instances; `repos::list_all_with_owner` sorts by a joined username and
122 needs all rows — intentionally left.)
123- **(a) CSRF + cookie hardening** —
124 - Cookie: `HttpOnly` + `SameSite=Lax` + `Secure` (auto via
125 `Config::secure_cookies()` when base_url is https).
126 - Synchronizer token `HMAC-SHA256(server_secret, session)`; secret persisted
127 at `data_dir/csrf_secret` (`App::csrf_token` in `crates/anvil-core/src/lib.rs`).
128 Deps `hmac`, `sha2`.
129 - `Csrf` extractor + constant-time `verify_csrf` (`crates/anvil-web/src/auth.rs`).
130 Hidden `csrf` field + verification on add/delete SSH key, new repo, repo
131 settings. Login exempt; logout relies on SameSite.
132 - htmx insurance: `auth::csrf_context` middleware → request-scoped task-local;
133 `layout` sends the token via `hx-headers` on every htmx request.
134
135## The a/b/c plan — ALL DONE
136
137(a) CSRF + cookie hardening, (b) `docs/untrusted-mode.md` threat model,
138(c) sandboxed CI broker. Stronger isolation tiers (gVisor/Kata/Firecracker,
139egress filtering, CI-minute quotas) are recorded in the threat model as the
140gate for untrusted tenants, not planned work.
141
142## Loose ends
143
144- **CSRF header consumption:** `hx-headers` sends the token as a `csrf` header,
145 but `verify_csrf` only reads the form field. When we add a tokenless htmx
146 action (raw `hx-post`/`hx-delete`, no `<form>`), also read the `csrf` header.
147- **Toasty migrations:** schema only pushed on a fresh DB (`db::connect`); new
148 columns won't apply to an existing DB until migrations land. (The
149 `data_dir/csrf_secret` file is created automatically — no DB change.)
150- **Suggested commits when ready:** (1) CI UI, (2) CD webhook + deploy wiring,
151 (3) ci.rs ORM cleanup + test, (4) CSRF + cookies, (5) UI quick wins
152 (latest-commit bar, profile email, breadcrumb), (6) CI sandbox + threat-model
153 doc, (7) pages hosting. Trailer: `Co-Authored-By: Claude ...`.
154- **Pages caveats (single-tenant-acceptable):** served from the forge origin —
155 move to a separate origin before untrusted users (threat model §2); whole
156 blobs load into memory per request (fine at our scale).
157
158## Remaining roadmap (plan milestones beyond a/b/c)
159
1608. Issues · 9. Pull requests (gix merge) · 10. Webhooks (mind the SSRF item).