anvilsign in

collin/anvil

02fbffce0d2e3994e74163498c7ec58b9b679037 / TODO.md

BoardRenderedSource

Todo1 open

  1. pull mirror (maybe): a repo that virtually mirrors a GitHub repo
    • just displays it here — periodically fetched, read-only on the anvil side

Done10 done

  1. render TODO.md per the todo-md spec (~/Code/todo-md) as a kanban board
    • anvil-web/src/todomd.rs: spec parser (sections by heading, [x]-marked done sections, checkbox tasks, indented details, fence-aware) + board renderer; prose sections render as a collapsible notes area below the board
    • blob pages for any TODO.md get Board/Rendered/Source pills (board default, falls back to markdown when the file has no tasks); the repo page shows the board in a box below the README
    • this file is itself spec-compliant now; structured updates come later
  2. browse source at any branch, tag, or commit
    • /{owner}/{repo}/tree/{rev} always accepted any rev; what was missing was UI. Added: branch/tag switcher dropdown on the repo and tree pages, "browse files" link on the commit page, percent-encoded ref names so branches with / work, and an unborn-HEAD fallback so a repo whose HEAD names a missing branch no longer renders as empty.
  3. CI artifact system
    • done per docs/ci-artifacts.md: artifacts: in ci.yml with in-container meta extractors, broker collection via download_from_container, run-page list, downloads + /{owner}/{repo}/artifacts/{rev}/{name} alias + browse: true static sites, quota GC with branch-tip pinning, and a schema shim so the table appears on existing DBs.
    • verified end-to-end against real Docker incl. the GC path; rustdoc was the test case (big HTML subtree served like pages)
    • repo-deletion cleanup deferred: repo deletion doesn't exist
  4. repo mirroring to GitHub (push mirror)
    • per-repo "Mirror push URL" in settings (repositories.mirror_url, column shim for existing DBs); after every successful push over HTTP or SSH a background mirror push fires
    • pure gitoxide (design rule in CLAUDE.md — no git binary): gix can't push yet, so anvil-git/src/push.rs implements the send-pack client itself — advertisement parse, mirror commands honoring delete-refs, gitserver_core::pack::build_raw_pack, report-status — over smart HTTP(S) (reqwest + rustls/ring; no aws-lc, musl zigbuild verified) or a local path served by gitserver-core in-process
    • credentials ride in the URL (https://x-access-token:<token>@github.com/...), stored as-is and redacted from logs; verified e2e: anvil→anvil over real HTTP incl. branch-deletion propagation, cloned back with real git
    • building this surfaced and fixed three gitserver-core server bugs: (1) git clone of any repo containing an annotated tag died mid-transfer on protocol v2 — tag-object wants weren't peeled into the pack walk (regression test tag_object_wants_are_peeled_and_packed); (2) the receive-pack advertisement peeled tag refs, so re-pushing an identical annotated tag was wrongly rejected (v0 advertisement now also emits proper ^{} peeled lines); (3) ref deletion was prohibited outright — now advertised and allowed via delete-refs, with the HEAD (default) branch protected
  5. per-repository issue tracker
    • Issue/IssueComment models with schema shims; per-repo numbering (#1, #2, ...); list page with open/closed tabs, new-issue form, detail page with markdown bodies and comments, close/reopen (issue author or repo writer)
    • any logged-in reader can open issues and comment; visibility follows the repo; "Issues" link in the repo nav; CSRF on all forms; verified e2e through the login + form flow
    • no labels/assignees/editing yet — deliberately minimal
  6. render a root README.md below the file tree on the repo page
    • any case of readme.md at the root; reuses render_markdown, links to the blob view from the box header
  7. push-to-create: git push to a repo that doesn't exist yet creates it
    • it did NOT already work — both transports 404'd. Policy in repos::create_on_push: pusher must own the namespace or be admin; created repos are private.
    • over HTTP a missing repo now answers the receive-pack advertisement with a Basic challenge so git prompts; verified e2e over both HTTP and SSH, incl. the cross-namespace denial
  8. make SSH the default clone selection, first in the pill buttons
    • only when [ssh] enabled; HTTP stays the lone pill otherwise
  9. rename the anvil crate to anvil_cli
    • named it anvil-cli to match the workspace's hyphenated crate names; the binary is still anvild, so deploy scripts and docs needed no changes
  10. fix git push failing on thin packs (REF_DELTA)
    • root cause: every push after the first sends a thin pack; gitserver-core passed None as thin_pack_base_object_lookup to gix_pack::Bundle::write_to_directory, so gix couldn't resolve the delta bases and aborted ("Ref delta objects are not supported..."). First-push-to-empty-repo worked because that pack is self-contained.
    • fix: pass the already-open gix::Repository as the lookup (it implements gix_object::Find). Regression test receive_thin_pack_with_ref_deltas builds a real thin pack via git pack-objects --thin and pushes it through receive_pack; verified the test fails without the fix.

Session notes / resume point6 open

  1. Done this session (2026-06-10, third session)

    All six remaining TODO items — see # Done [x] above for details. Headlines:

    • CI artifacts (docs/ci-artifacts.md is the canonical reference): artifacts: in .anvil/ci.yml; broker pulls them out of the stopped container via download_from_container (inverse of the checkout upload — still no mounts); meta extractors run inside the sandbox, one file per value under /tmp/anvil-meta; browse: true directories are served like pages (rustdoc-ready); /{owner}/{repo}/artifacts/{rev}/{name} is the latest-on-branch alias; per-repo quota GC pins branch tips. New table ci_artifacts + [ci] artifact_*_mb caps.
    • Schema shims for existing DBs (anvil-core/src/db.rs): Toasty still only pushes schema on a fresh file, so new tables/columns ship as idempotent DDL applied on connect (SCHEMA_SHIMS/COLUMN_SHIMS), with tests asserting fresh and migrated databases converge. New deps: rusqlite (pinned to toasty's), flate2.
    • Issues (anvil-core/src/issues.rs, anvil-web/src/issues.rs): minimal GitHub-shaped tracker; tables issues + issue_comments.
    • Push mirroring (anvil-git/src/mirror.rs): repositories.mirror_url → background git push --mirror after each push; Dockerfile now installs git (the one thing gix can't do yet is push).
    • Push-to-create (repos::create_on_push + both transports), README on repo page, SSH-first clone pills, rev-switcher/browse-at-rev UI (committed earlier this session, along with the anvil-cli rename).
  2. Done earlier (2026-06-10, second session)
    • UI quick wins (crates/anvil-web/src/ui.rs) — latest-commit bar on the repo page (sha + subject + author/time, attached above the file box, links to the commit); profile page no longer shows the email; repo header reads owner / repo without the leading anvil /.
    • (c) sandboxed CI broker — job containers now run with cap_drop=ALL + no-new-privileges unconditionally, plus config-driven pids_limit (512), memory_mb+swap (2048), cpus (2), wall-clock timeout_secs (1800, force-removed on expiry), optional network = false, run_as, and an allowed_images allowlist (empty = any; tagless entry allows all tags). CiConfig in crates/anvil-core/src/config.rs (with image_allowed test); execute() in crates/anvil-ci/src/lib.rs. Read-only rootfs deliberately skipped (workspace lives in the container fs; no volumes ever attached). Docs: DEPLOY.md §7, anvil.example.toml [ci].
    • (b) threat model — docs/untrusted-mode.md: severity-ranked analysis (CI containment, pages/stored-XSS origin, git resource exhaustion, registration anti-abuse, authz granularity, webhook SSRF), the already-right list, and the stance: single-tenant supported, untrusted gated on items 1–4.
    • Pages hosting — crates/anvil-web/src/pages.rs: serves a repo's pages branch at /{owner}/{repo}/pages/...; top-level dirs are separate sites (rustdoc, book, …); index.html resolution with trailing-slash redirect so relative links work; extension→content-type map + nosniff; listing page with publish hint; "Pages" button on the repo header. Visibility follows the repo (private → 404). Publish with git push origin <built-branch>:pages.
  3. Done earlier (same day, first session)
    • CI UI — runs list /{owner}/{repo}/ci, run-detail (status/timing/log), per-commit status badges, "CI" nav link. (crates/anvil-web/src/ui.rs)
    • CD redeploy webhook — on a green run of [ci] deploy_branch in the single [ci] deploy_repo, POST to [ci] deploy_webhook (X-Anvil-Deploy-Secret header). Scoped to ONE repo. CiConfig in crates/anvil-core/src/config.rs; deploy() in crates/anvil-ci/src/lib.rs. Docs: DEPLOY.md §7, deploy/anvil.toml. reqwest added with NO TLS feature (keeps musl cross-compile aws-lc-free).
    • Docker socket on hagrid — deploy/run.sh mounts it + --group-adds the gid for the non-root user; caveat in DEPLOY.md §4.
    • Toasty ORM cleanup — ci.rs list_by_repo/latest_for_commit/ queued_ids now sort/limit/filter in SQL, not in memory. Verified by the new ordering_and_limit_run_in_the_database test. (Sweep: these were the only real instances; repos::list_all_with_owner sorts by a joined username and needs all rows — intentionally left.)
    • (a) CSRF + cookie hardening —
      • Cookie: HttpOnly + SameSite=Lax + Secure (auto via Config::secure_cookies() when base_url is https).
      • Synchronizer token HMAC-SHA256(server_secret, session); secret persisted at data_dir/csrf_secret (App::csrf_token in crates/anvil-core/src/lib.rs). Deps hmac, sha2.
      • Csrf extractor + constant-time verify_csrf (crates/anvil-web/src/auth.rs). Hidden csrf field + verification on add/delete SSH key, new repo, repo settings. Login exempt; logout relies on SameSite.
      • htmx insurance: auth::csrf_context middleware → request-scoped task-local; layout sends the token via hx-headers on every htmx request.
  4. The a/b/c plan — ALL DONE

    (a) CSRF + cookie hardening, (b) docs/untrusted-mode.md threat model, (c) sandboxed CI broker. Stronger isolation tiers (gVisor/Kata/Firecracker, egress filtering, CI-minute quotas) are recorded in the threat model as the gate for untrusted tenants, not planned work.

  5. Loose ends
    • CSRF header consumption: hx-headers sends the token as a csrf header, but verify_csrf only reads the form field. When we add a tokenless htmx action (raw hx-post/hx-delete, no <form>), also read the csrf header.
    • Toasty migrations: still no real migration system; the shim approach in db::connect (SCHEMA_SHIMS for tables, COLUMN_SHIMS for columns, both test-verified against a fresh push_schema) covers what we've needed so far. New columns must be declared last in the model.
    • Mirror URL secrecy: repositories.mirror_url may embed a token and is stored plaintext in SQLite (logged redacted). Fine single-tenant; revisit if the DB ever leaves the box.
    • Pages caveats (single-tenant-acceptable): served from the forge origin — move to a separate origin before untrusted users (threat model §2); whole blobs load into memory per request (fine at our scale). The same applies to browse: true CI artifacts.
    • Issue tracker minimalism: no labels, assignees, milestones, or editing/deleting of posts. Per-repo numbering assumes a single server process (matches deployment; noted in models.rs).
  6. Remaining roadmap (plan milestones beyond a/b/c)
    1. Pull requests (gix merge) · 10. Webhooks (mind the SSRF item). (8. Issues shipped 2026-06-10.)