anvilsign in

collin/anvil

1f463de586fbfeb46c9a7f1d5a63230df5b3f3a7 / TODO.md

BoardRenderedSource

1# Misc TODO
2
3- [x] do we have a way to view the source in a repo by branch or at a given commit?
4 - yes: `/{owner}/{repo}/tree/{rev}` always accepted any branch/tag/commit;
5 what was missing was UI. Added: branch/tag switcher dropdown on the repo
6 and tree pages, "browse files" link on the commit page, percent-encoded
7 ref names so branches with `/` work, and an unborn-HEAD fallback so a
8 repo whose HEAD names a missing branch no longer renders as empty.
9- [ ] implement the CI artifact system _(designed — see `docs/ci-artifacts.md`;
10 implementation order is at the bottom of that doc)_
11 - every push triggers CI on the tip commit (already true); a run should be
12 able to produce artifacts
13 - artifacts are stored per-commit and served from anvil (download from the
14 run page / commit page; latest-on-branch alias would be nice)
15 - jobs can declare how to extract metadata from artifacts (e.g. sizes,
16 test/coverage numbers, version strings) so it can be surfaced in the UI
17 next to the run/commit
18 - sketch needed: where artifacts live on disk, retention/GC, how
19 `.anvil/ci.yml` declares artifact paths + metadata extractors, and how
20 the broker gets files out of the sandboxed container
21 - use rustdoc as an example when testing the feature: it generates a big
22 HTML subtree, so support rendering/serving a whole HTML artifact subtree
23 the way the pages feature does (rustdoc output as a served site)
24- [ ] repo mirroring to/from GitHub
25 - push mirror: pushes to an anvil repo get forwarded to a configured
26 GitHub remote
27 - pull mirror (maybe): a repo that virtually mirrors a GitHub repo and
28 just displays it here — periodically fetched, read-only on the anvil
29 side
30- [ ] per-repository issue tracker
31- [x] rename the `anvil` crate to `anvil_cli` _(named it `anvil-cli` to match
32 the workspace's hyphenated crate names; the binary is still `anvild`, so
33 deploy scripts and docs needed no changes)_
34
35---
36
37# Error in git push _(FIXED 2026-06-10, uncommitted)_
38
39**Root cause:** every push after the first sends a *thin pack* — deltas whose
40base objects aren't in the pack (the server already has them), referenced by
41object id (`REF_DELTA`). `vendor/gitserver-core/src/receive_pack.rs::write_pack`
42passed `None` as `thin_pack_base_object_lookup` to
43`gix_pack::Bundle::write_to_directory`, so gix couldn't resolve the bases and
44aborted. First-push-to-empty-repo worked because that pack is self-contained.
45
46**Fix:** pass the already-open `gix::Repository` as the lookup (it implements
47`gix_object::Find`). Regression test
48`receive_thin_pack_with_ref_deltas` builds a real thin pack via
49`git pack-objects --thin`, asserts it contains ref-deltas, and pushes it
50through `receive_pack`. Verified the test fails without the fix.
51
52Original report:
53
54```
55collin@mini ~/C/anvil (main)> git push
56Enter passphrase for key '/Users/collin/.ssh/id_ed25519':
57Enumerating objects: 117, done.
58Counting objects: 100% (117/117), done.
59Delta compression using up to 8 threads
60Compressing objects: 100% (62/62), done.
61Writing objects: 100% (66/66), 27.57 KiB | 3.94 MiB/s, done.
62Total 66 (delta 39), reused 0 (delta 0), pack-reused 0 (from 0)
63send-pack: unexpected disconnect while reading sideband packet
64fatal: the remote end hung up unexpectedly
65collin@mini ~/C/anvil (main) [128]>
66```
67
68server logs:
69
70```
7126-06-09T21:53:46.466577Z INFO anvil_ssh: ssh auth: accepted key SHA256:Rg41caN7vw2WYYxiJN6lrIlX0DTXYF0rC2QzKZW1tB0 (user 1)
722026-06-09T21:53:46.635946Z INFO anvil_ssh: ssh git-receive-pack on collin/anvil.git (user Some(1))
732026-06-09T21:53:46.805146Z ERROR anvil_ssh: git ssh git-receive-pack: protocol error: failed to write incoming pack: Ref delta objects are not supported as there is no way to look them up. Resolve them beforehand.
742026-06-09T21:54:28.571834Z INFO anvil_ssh: ssh auth: rejected unknown key SHA256:ZlWZyHqspqFeUQV84qaXtDQq4gcA33dR7y8dYbeg9u8
752026-06-09T21:54:35.565597Z INFO anvil_ssh: ssh auth: accepted key SHA256:Rg41caN7vw2WYYxiJN6lrIlX0DTXYF0rC2QzKZW1tB0 (user 1)
762026-06-09T21:54:35.676113Z INFO anvil_ssh: ssh git-receive-pack on collin/anvil.git (user Some(1))
772026-06-09T21:54:36.268520Z ERROR anvil_ssh: git ssh git-receive-pack: protocol error: failed to write incoming pack: Ref delta objects are not supported as there is no way to look them up. Resolve them beforehand.
782026-06-09T21:55:54.223033Z INFO anvil_ssh: ssh auth: rejected unknown key SHA256:ZlWZyHqspqFeUQV84qaXtDQq4gcA33dR7y8dYbeg9u8
792026-06-09T21:56:00.758384Z INFO anvil_ssh: ssh auth: accepted key SHA256:Rg41caN7vw2WYYxiJN6lrIlX0DTXYF0rC2QzKZW1tB0 (user 1)
802026-06-09T21:56:00.906553Z INFO anvil_ssh: ssh git-receive-pack on collin/anvil.git (user Some(1))
812026-06-09T21:56:01.067903Z ERROR anvil_ssh: git ssh git-receive-pack: protocol error: failed to write incoming pack: Ref delta objects are not supported as there is no way to look them up. Resolve them beforehand.
82```
83
84---
85
86# Session notes / resume point
87
88_Last updated: 2026-06-10 (second session). Working state is clean: `cargo
89build`, `cargo clippy --workspace`, `cargo fmt --all`, and `cargo test
90--workspace` all pass. Everything below is UNCOMMITTED (repo convention:
91commit only when asked). **All top-of-file TODO items are done.**_
92
93## Done this session (2026-06-10, second session)
94
95- **UI quick wins** (`crates/anvil-web/src/ui.rs`) — latest-commit bar on the
96 repo page (sha + subject + author/time, attached above the file box, links
97 to the commit); profile page no longer shows the email; repo header reads
98 `owner / repo` without the leading `anvil /`.
99- **(c) sandboxed CI broker** — job containers now run with `cap_drop=ALL` +
100 `no-new-privileges` unconditionally, plus config-driven `pids_limit` (512),
101 `memory_mb`+swap (2048), `cpus` (2), wall-clock `timeout_secs` (1800,
102 force-removed on expiry), optional `network = false`, `run_as`, and an
103 `allowed_images` allowlist (empty = any; tagless entry allows all tags).
104 `CiConfig` in `crates/anvil-core/src/config.rs` (with `image_allowed` test);
105 `execute()` in `crates/anvil-ci/src/lib.rs`. Read-only rootfs deliberately
106 skipped (workspace lives in the container fs; no volumes ever attached).
107 Docs: `DEPLOY.md` §7, `anvil.example.toml` `[ci]`.
108- **(b) threat model** — `docs/untrusted-mode.md`: severity-ranked analysis
109 (CI containment, pages/stored-XSS origin, git resource exhaustion,
110 registration anti-abuse, authz granularity, webhook SSRF), the already-right
111 list, and the stance: single-tenant supported, untrusted gated on items 1–4.
112- **Pages hosting** — `crates/anvil-web/src/pages.rs`: serves a repo's `pages`
113 branch at `/{owner}/{repo}/pages/...`; top-level dirs are separate sites
114 (rustdoc, book, …); `index.html` resolution with trailing-slash redirect so
115 relative links work; extension→content-type map + `nosniff`; listing page
116 with publish hint; "Pages" button on the repo header. Visibility follows the
117 repo (private → 404). Publish with `git push origin <built-branch>:pages`.
118
119## Done earlier (same day, first session)
120
121- **CI UI** — runs list `/{owner}/{repo}/ci`, run-detail (status/timing/log),
122 per-commit status badges, "CI" nav link. (`crates/anvil-web/src/ui.rs`)
123- **CD redeploy webhook** — on a green run of `[ci] deploy_branch` in the single
124 `[ci] deploy_repo`, POST to `[ci] deploy_webhook` (`X-Anvil-Deploy-Secret`
125 header). Scoped to ONE repo. `CiConfig` in `crates/anvil-core/src/config.rs`;
126 `deploy()` in `crates/anvil-ci/src/lib.rs`. Docs: `DEPLOY.md` §7,
127 `deploy/anvil.toml`. `reqwest` added with NO TLS feature (keeps musl
128 cross-compile aws-lc-free).
129- **Docker socket on hagrid** — `deploy/run.sh` mounts it + `--group-add`s the
130 gid for the non-root user; caveat in `DEPLOY.md` §4.
131- **Toasty ORM cleanup** — `ci.rs` `list_by_repo`/`latest_for_commit`/
132 `queued_ids` now sort/limit/filter in SQL, not in memory. Verified by the new
133 `ordering_and_limit_run_in_the_database` test. (Sweep: these were the only
134 real instances; `repos::list_all_with_owner` sorts by a joined username and
135 needs all rows — intentionally left.)
136- **(a) CSRF + cookie hardening** —
137 - Cookie: `HttpOnly` + `SameSite=Lax` + `Secure` (auto via
138 `Config::secure_cookies()` when base_url is https).
139 - Synchronizer token `HMAC-SHA256(server_secret, session)`; secret persisted
140 at `data_dir/csrf_secret` (`App::csrf_token` in `crates/anvil-core/src/lib.rs`).
141 Deps `hmac`, `sha2`.
142 - `Csrf` extractor + constant-time `verify_csrf` (`crates/anvil-web/src/auth.rs`).
143 Hidden `csrf` field + verification on add/delete SSH key, new repo, repo
144 settings. Login exempt; logout relies on SameSite.
145 - htmx insurance: `auth::csrf_context` middleware → request-scoped task-local;
146 `layout` sends the token via `hx-headers` on every htmx request.
147
148## The a/b/c plan — ALL DONE
149
150(a) CSRF + cookie hardening, (b) `docs/untrusted-mode.md` threat model,
151(c) sandboxed CI broker. Stronger isolation tiers (gVisor/Kata/Firecracker,
152egress filtering, CI-minute quotas) are recorded in the threat model as the
153gate for untrusted tenants, not planned work.
154
155## Loose ends
156
157- **CSRF header consumption:** `hx-headers` sends the token as a `csrf` header,
158 but `verify_csrf` only reads the form field. When we add a tokenless htmx
159 action (raw `hx-post`/`hx-delete`, no `<form>`), also read the `csrf` header.
160- **Toasty migrations:** schema only pushed on a fresh DB (`db::connect`); new
161 columns won't apply to an existing DB until migrations land. (The
162 `data_dir/csrf_secret` file is created automatically — no DB change.)
163- **Suggested commits when ready:** (1) CI UI, (2) CD webhook + deploy wiring,
164 (3) ci.rs ORM cleanup + test, (4) CSRF + cookies, (5) UI quick wins
165 (latest-commit bar, profile email, breadcrumb), (6) CI sandbox + threat-model
166 doc, (7) pages hosting. Trailer: `Co-Authored-By: Claude ...`.
167- **Pages caveats (single-tenant-acceptable):** served from the forge origin —
168 move to a separate origin before untrusted users (threat model §2); whole
169 blobs load into memory per request (fine at our scale).
170
171## Remaining roadmap (plan milestones beyond a/b/c)
172
1738. Issues · 9. Pull requests (gix merge) · 10. Webhooks (mind the SSRF item).