collin/anvil · 632e6157
docs: trim TODO.md to the open items
Collin Richards · 2026-06-10 10:21 UTC · 632e61574611a73f45f97fc0c47eb9c54ad419b7 · parent 02fbffce · browse files
modifiedTODO.md+2 −207
| ⋯ 1 unchanged line | |||
| 2 | 2 | ||
| 3 | 3 | - [ ] pull mirror (maybe): a repo that virtually mirrors a GitHub repo | |
| 4 | 4 | - just displays it here — periodically fetched, read-only on the anvil side | |
| 5 | - | ||
| 6 | - | # Done [x] | |
| 7 | - | ||
| 8 | - | - [x] render `TODO.md` per the todo-md spec (`~/Code/todo-md`) as a kanban board | |
| 9 | - | - `anvil-web/src/todomd.rs`: spec parser (sections by heading, `[x]`-marked | |
| 10 | - | done sections, checkbox tasks, indented details, fence-aware) + board | |
| 11 | - | renderer; prose sections render as a collapsible notes area below the | |
| 12 | - | board | |
| 13 | - | - blob pages for any `TODO.md` get Board/Rendered/Source pills (board | |
| 14 | - | default, falls back to markdown when the file has no tasks); the repo | |
| 15 | - | page shows the board in a box below the README | |
| 16 | - | - this file is itself spec-compliant now; structured updates come later | |
| 17 | - | ||
| 18 | - | - [x] browse source at any branch, tag, or commit | |
| 19 | - | - `/{owner}/{repo}/tree/{rev}` always accepted any rev; what was missing | |
| 20 | - | was UI. Added: branch/tag switcher dropdown on the repo and tree pages, | |
| 21 | - | "browse files" link on the commit page, percent-encoded ref names so | |
| 22 | - | branches with `/` work, and an unborn-HEAD fallback so a repo whose HEAD | |
| 23 | - | names a missing branch no longer renders as empty. | |
| 24 | - | - [x] CI artifact system | |
| 25 | - | - done per `docs/ci-artifacts.md`: `artifacts:` in ci.yml with | |
| 26 | - | in-container meta extractors, broker collection via | |
| 27 | - | `download_from_container`, run-page list, downloads + | |
| 28 | - | `/{owner}/{repo}/artifacts/{rev}/{name}` alias + `browse: true` static | |
| 29 | - | sites, quota GC with branch-tip pinning, and a schema shim so the table | |
| 30 | - | appears on existing DBs. | |
| 31 | - | - verified end-to-end against real Docker incl. the GC path; rustdoc was | |
| 32 | - | the test case (big HTML subtree served like pages) | |
| 33 | - | - repo-deletion cleanup deferred: repo deletion doesn't exist | |
| 34 | - | - [x] repo mirroring to GitHub (push mirror) | |
| 35 | - | - per-repo "Mirror push URL" in settings (`repositories.mirror_url`, | |
| 36 | - | column shim for existing DBs); after every successful push over HTTP or | |
| 37 | - | SSH a background mirror push fires | |
| 38 | - | - **pure gitoxide** (design rule in CLAUDE.md — no git binary): gix can't | |
| 39 | - | push yet, so `anvil-git/src/push.rs` implements the send-pack client | |
| 40 | - | itself — advertisement parse, mirror commands honoring `delete-refs`, | |
| 41 | - | `gitserver_core::pack::build_raw_pack`, report-status — over smart | |
| 42 | - | HTTP(S) (reqwest + rustls/*ring*; no aws-lc, musl zigbuild verified) or | |
| 43 | - | a local path served by gitserver-core in-process | |
| 44 | - | - credentials ride in the URL | |
| 45 | - | (`https://x-access-token:<token>@github.com/...`), stored as-is and | |
| 46 | - | redacted from logs; verified e2e: anvil→anvil over real HTTP incl. | |
| 47 | - | branch-deletion propagation, cloned back with real git | |
| 48 | - | - building this surfaced and fixed three gitserver-core server bugs: | |
| 49 | - | (1) `git clone` of any repo containing an *annotated tag* died | |
| 50 | - | mid-transfer on protocol v2 — tag-object wants weren't peeled into the | |
| 51 | - | pack walk (regression test `tag_object_wants_are_peeled_and_packed`); | |
| 52 | - | (2) the receive-pack advertisement peeled tag refs, so re-pushing an | |
| 53 | - | identical annotated tag was wrongly rejected (v0 advertisement now | |
| 54 | - | also emits proper `^{}` peeled lines); | |
| 55 | - | (3) ref deletion was prohibited outright — now advertised and allowed | |
| 56 | - | via `delete-refs`, with the HEAD (default) branch protected | |
| 57 | - | - [x] per-repository issue tracker | |
| 58 | - | - `Issue`/`IssueComment` models with schema shims; per-repo numbering | |
| 59 | - | (#1, #2, ...); list page with open/closed tabs, new-issue form, detail | |
| 60 | - | page with markdown bodies and comments, close/reopen (issue author or | |
| 61 | - | repo writer) | |
| 62 | - | - any logged-in reader can open issues and comment; visibility follows | |
| 63 | - | the repo; "Issues" link in the repo nav; CSRF on all forms; verified | |
| 64 | - | e2e through the login + form flow | |
| 65 | - | - no labels/assignees/editing yet — deliberately minimal | |
| 66 | - | - [x] render a root `README.md` below the file tree on the repo page | |
| 67 | - | - any case of `readme.md` at the root; reuses `render_markdown`, links to | |
| 68 | - | the blob view from the box header | |
| 69 | - | - [x] push-to-create: `git push` to a repo that doesn't exist yet creates it | |
| 70 | - | - it did NOT already work — both transports 404'd. Policy in | |
| 71 | - | `repos::create_on_push`: pusher must own the namespace or be admin; | |
| 72 | - | created repos are private. | |
| 73 | - | - over HTTP a missing repo now answers the receive-pack advertisement | |
| 74 | - | with a Basic challenge so git prompts; verified e2e over both HTTP and | |
| 75 | - | SSH, incl. the cross-namespace denial | |
| 76 | - | - [x] make SSH the default clone selection, first in the pill buttons | |
| 77 | - | - only when `[ssh] enabled`; HTTP stays the lone pill otherwise | |
| 78 | - | - [x] rename the `anvil` crate to `anvil_cli` | |
| 79 | - | - named it `anvil-cli` to match the workspace's hyphenated crate names; | |
| 80 | - | the binary is still `anvild`, so deploy scripts and docs needed no | |
| 81 | - | changes | |
| 82 | - | - [x] fix git push failing on thin packs (REF_DELTA) | |
| 83 | - | - root cause: every push after the first sends a thin pack; gitserver-core | |
| 84 | - | passed `None` as `thin_pack_base_object_lookup` to | |
| 85 | - | `gix_pack::Bundle::write_to_directory`, so gix couldn't resolve the | |
| 86 | - | delta bases and aborted ("Ref delta objects are not supported..."). | |
| 87 | - | First-push-to-empty-repo worked because that pack is self-contained. | |
| 88 | - | - fix: pass the already-open `gix::Repository` as the lookup (it | |
| 89 | - | implements `gix_object::Find`). Regression test | |
| 90 | - | `receive_thin_pack_with_ref_deltas` builds a real thin pack via | |
| 91 | - | `git pack-objects --thin` and pushes it through `receive_pack`; | |
| 92 | - | verified the test fails without the fix. | |
| 93 | - | ||
| 94 | - | # Session notes / resume point | |
| 95 | - | ||
| 96 | - | _Last updated: 2026-06-10 (third session). Working state is clean: build, | |
| 97 | - | clippy, fmt, sort-derives, and `cargo test --workspace` all pass (the | |
| 98 | - | pre-commit hook runs all of these)._ | |
| 99 | - | ||
| 100 | - | ## Done this session (2026-06-10, third session) | |
| 101 | - | ||
| 102 | - | All six remaining TODO items — see `# Done [x]` above for details. Headlines: | |
| 103 | - | ||
| 104 | - | - **CI artifacts** (`docs/ci-artifacts.md` is the canonical reference): | |
| 105 | - | `artifacts:` in `.anvil/ci.yml`; broker pulls them out of the stopped | |
| 106 | - | container via `download_from_container` (inverse of the checkout upload — | |
| 107 | - | still no mounts); meta extractors run *inside* the sandbox, one file per | |
| 108 | - | value under `/tmp/anvil-meta`; `browse: true` directories are served like | |
| 109 | - | pages (rustdoc-ready); `/{owner}/{repo}/artifacts/{rev}/{name}` is the | |
| 110 | - | latest-on-branch alias; per-repo quota GC pins branch tips. New table | |
| 111 | - | `ci_artifacts` + `[ci] artifact_*_mb` caps. | |
| 112 | - | - **Schema shims for existing DBs** (`anvil-core/src/db.rs`): Toasty still | |
| 113 | - | only pushes schema on a fresh file, so new tables/columns ship as | |
| 114 | - | idempotent DDL applied on connect (`SCHEMA_SHIMS`/`COLUMN_SHIMS`), with | |
| 115 | - | tests asserting fresh and migrated databases converge. New deps: rusqlite | |
| 116 | - | (pinned to toasty's), flate2. | |
| 117 | - | - **Issues** (`anvil-core/src/issues.rs`, `anvil-web/src/issues.rs`): | |
| 118 | - | minimal GitHub-shaped tracker; tables `issues` + `issue_comments`. | |
| 119 | - | - **Push mirroring** (`anvil-git/src/mirror.rs`): `repositories.mirror_url` | |
| 120 | - | → background `git push --mirror` after each push; Dockerfile now installs | |
| 121 | - | git (the one thing gix can't do yet is push). | |
| 122 | - | - **Push-to-create** (`repos::create_on_push` + both transports), **README | |
| 123 | - | on repo page**, **SSH-first clone pills**, **rev-switcher/browse-at-rev UI** | |
| 124 | - | (committed earlier this session, along with the `anvil-cli` rename). | |
| 125 | - | ||
| 126 | - | ## Done earlier (2026-06-10, second session) | |
| 127 | - | ||
| 128 | - | - **UI quick wins** (`crates/anvil-web/src/ui.rs`) — latest-commit bar on the | |
| 129 | - | repo page (sha + subject + author/time, attached above the file box, links | |
| 130 | - | to the commit); profile page no longer shows the email; repo header reads | |
| 131 | - | `owner / repo` without the leading `anvil /`. | |
| 132 | - | - **(c) sandboxed CI broker** — job containers now run with `cap_drop=ALL` + | |
| 133 | - | `no-new-privileges` unconditionally, plus config-driven `pids_limit` (512), | |
| 134 | - | `memory_mb`+swap (2048), `cpus` (2), wall-clock `timeout_secs` (1800, | |
| 135 | - | force-removed on expiry), optional `network = false`, `run_as`, and an | |
| 136 | - | `allowed_images` allowlist (empty = any; tagless entry allows all tags). | |
| 137 | - | `CiConfig` in `crates/anvil-core/src/config.rs` (with `image_allowed` test); | |
| 138 | - | `execute()` in `crates/anvil-ci/src/lib.rs`. Read-only rootfs deliberately | |
| 139 | - | skipped (workspace lives in the container fs; no volumes ever attached). | |
| 140 | - | Docs: `DEPLOY.md` §7, `anvil.example.toml` `[ci]`. | |
| 141 | - | - **(b) threat model** — `docs/untrusted-mode.md`: severity-ranked analysis | |
| 142 | - | (CI containment, pages/stored-XSS origin, git resource exhaustion, | |
| 143 | - | registration anti-abuse, authz granularity, webhook SSRF), the already-right | |
| 144 | - | list, and the stance: single-tenant supported, untrusted gated on items 1–4. | |
| 145 | - | - **Pages hosting** — `crates/anvil-web/src/pages.rs`: serves a repo's `pages` | |
| 146 | - | branch at `/{owner}/{repo}/pages/...`; top-level dirs are separate sites | |
| 147 | - | (rustdoc, book, …); `index.html` resolution with trailing-slash redirect so | |
| 148 | - | relative links work; extension→content-type map + `nosniff`; listing page | |
| 149 | - | with publish hint; "Pages" button on the repo header. Visibility follows the | |
| 150 | - | repo (private → 404). Publish with `git push origin <built-branch>:pages`. | |
| 151 | - | ||
| 152 | - | ## Done earlier (same day, first session) | |
| 153 | - | ||
| 154 | - | - **CI UI** — runs list `/{owner}/{repo}/ci`, run-detail (status/timing/log), | |
| 155 | - | per-commit status badges, "CI" nav link. (`crates/anvil-web/src/ui.rs`) | |
| 156 | - | - **CD redeploy webhook** — on a green run of `[ci] deploy_branch` in the single | |
| 157 | - | `[ci] deploy_repo`, POST to `[ci] deploy_webhook` (`X-Anvil-Deploy-Secret` | |
| 158 | - | header). Scoped to ONE repo. `CiConfig` in `crates/anvil-core/src/config.rs`; | |
| 159 | - | `deploy()` in `crates/anvil-ci/src/lib.rs`. Docs: `DEPLOY.md` §7, | |
| 160 | - | `deploy/anvil.toml`. `reqwest` added with NO TLS feature (keeps musl | |
| 161 | - | cross-compile aws-lc-free). | |
| 162 | - | - **Docker socket on hagrid** — `deploy/run.sh` mounts it + `--group-add`s the | |
| 163 | - | gid for the non-root user; caveat in `DEPLOY.md` §4. | |
| 164 | - | - **Toasty ORM cleanup** — `ci.rs` `list_by_repo`/`latest_for_commit`/ | |
| 165 | - | `queued_ids` now sort/limit/filter in SQL, not in memory. Verified by the new | |
| 166 | - | `ordering_and_limit_run_in_the_database` test. (Sweep: these were the only | |
| 167 | - | real instances; `repos::list_all_with_owner` sorts by a joined username and | |
| 168 | - | needs all rows — intentionally left.) | |
| 169 | - | - **(a) CSRF + cookie hardening** — | |
| 170 | - | - Cookie: `HttpOnly` + `SameSite=Lax` + `Secure` (auto via | |
| 171 | - | `Config::secure_cookies()` when base_url is https). | |
| 172 | - | - Synchronizer token `HMAC-SHA256(server_secret, session)`; secret persisted | |
| 173 | - | at `data_dir/csrf_secret` (`App::csrf_token` in `crates/anvil-core/src/lib.rs`). | |
| 174 | - | Deps `hmac`, `sha2`. | |
| 175 | - | - `Csrf` extractor + constant-time `verify_csrf` (`crates/anvil-web/src/auth.rs`). | |
| 176 | - | Hidden `csrf` field + verification on add/delete SSH key, new repo, repo | |
| 177 | - | settings. Login exempt; logout relies on SameSite. | |
| 178 | - | - htmx insurance: `auth::csrf_context` middleware → request-scoped task-local; | |
| 179 | - | `layout` sends the token via `hx-headers` on every htmx request. | |
| 180 | - | ||
| 181 | - | ## The a/b/c plan — ALL DONE | |
| 182 | - | ||
| 183 | - | (a) CSRF + cookie hardening, (b) `docs/untrusted-mode.md` threat model, | |
| 184 | - | (c) sandboxed CI broker. Stronger isolation tiers (gVisor/Kata/Firecracker, | |
| 185 | - | egress filtering, CI-minute quotas) are recorded in the threat model as the | |
| 186 | - | gate for untrusted tenants, not planned work. | |
| 187 | - | ||
| 188 | - | ## Loose ends | |
| 189 | - | ||
| 190 | - | - **CSRF header consumption:** `hx-headers` sends the token as a `csrf` header, | |
| 191 | - | but `verify_csrf` only reads the form field. When we add a tokenless htmx | |
| 192 | - | action (raw `hx-post`/`hx-delete`, no `<form>`), also read the `csrf` header. | |
| 193 | - | - **Toasty migrations:** still no real migration system; the shim approach in | |
| 194 | - | `db::connect` (`SCHEMA_SHIMS` for tables, `COLUMN_SHIMS` for columns, both | |
| 195 | - | test-verified against a fresh `push_schema`) covers what we've needed so | |
| 196 | - | far. New columns must be declared last in the model. | |
| 197 | - | - **Mirror URL secrecy:** `repositories.mirror_url` may embed a token and is | |
| 198 | - | stored plaintext in SQLite (logged redacted). Fine single-tenant; revisit | |
| 199 | - | if the DB ever leaves the box. | |
| 200 | - | - **Pages caveats (single-tenant-acceptable):** served from the forge origin — | |
| 201 | - | move to a separate origin before untrusted users (threat model §2); whole | |
| 202 | - | blobs load into memory per request (fine at our scale). The same applies to | |
| 203 | - | `browse: true` CI artifacts. | |
| 204 | - | - **Issue tracker minimalism:** no labels, assignees, milestones, or | |
| 205 | - | editing/deleting of posts. Per-repo numbering assumes a single server | |
| 206 | - | process (matches deployment; noted in `models.rs`). | |
| 207 | - | ||
| 208 | - | ## Remaining roadmap (plan milestones beyond a/b/c) | |
| 209 | - | ||
| 210 | - | 9. Pull requests (gix merge) · 10. Webhooks (mind the SSRF item). | |
| 211 | - | (8. Issues shipped 2026-06-10.) | |
| 5 | + | - [ ] pull requests (gix merge) | |
| 6 | + | - [ ] webhooks (mind the SSRF item in `docs/untrusted-mode.md`) | |