collin/anvil
d230eb4f5808259b9820e050172e0506429b450d / TODO.md
| 1 | # Todo |
| 2 | |
| 3 | ## ability to link to deployed / live site |
| 4 | |
| 5 | ## agent view, we have list of repos what about list of agents |
| 6 | |
| 7 | # Backlog |
| 8 | |
| 9 | |
| 10 | - [ ] agent sessions, next milestones (docs/agent-sessions.md): |
| 11 | - a real checkout: the container clones from anvil's smart-HTTP endpoint and |
| 12 | pushes `agent/<id>` back. Needs a session-scoped push credential, which |
| 13 | does not exist (tokens are read-only, Bearer only on GET/HEAD) |
| 14 | - ref-scope that credential to `refs/heads/agent/*` — needs a ref filter in |
| 15 | receive-pack. Until it lands a session credential could write `main` |
| 16 | - trigger surfaces: a start button on a TODO item, an issue, a red CI run |
| 17 | - rate limiting, so automated pushes can't queue sessions endlessly once |
| 18 | triggers exist (`max_concurrent` bounds concurrency, not churn) |
| 19 | - a finished session's transcript rendered on its page (it is already on |
| 20 | disk under `sessions/<id>.log`; nothing reads it back yet) |
| 21 | |
| 22 | - [ ] pull requests (gix merge) |
| 23 | - [ ] pull mirror (maybe): a repo that virtually mirrors a GitHub repo |
| 24 | - just displays it here — periodically fetched, read-only on the anvil side |
| 25 | |
| 26 | - [ ] richer file editing: a real markdown editor with a live render preview |
| 27 | (reuse `render_markdown`) before committing |
| 28 | - [ ] webhooks (mind the SSRF item in `docs/untrusted-mode.md`) |
| 29 | - [ ] attachment reclaim: an orphan sweep (delete attachments no committed file |
| 30 | references) and/or a per-attachment delete action — the recourse once a repo |
| 31 | hits its quota. Deferred: deletion is destructive and "orphaned" is fuzzy |
| 32 | (tip-only vs any-ref), so it wants its own design pass |
| 33 | - [ ] admin usage: per-repo drill-down, and a cheap cached/periodic variant if |
| 34 | the on-demand disk walk gets slow on large instances |
| 35 | - [ ] periodic disk usage cache: run `usage::compute()` on a timer (e.g., hourly) |
| 36 | and store the result so the admin dashboard doesn't block on disk walks |
| 37 | - [ ] repository preview images: extract the first "real" image (>few hundred px) |
| 38 | from README.md on a periodic scan, cache the attachment hash, and display in |
| 39 | repo listings for visual browsing |
| 40 | - [ ] API tokens: a `write` scope (would need CSRF-exempt write paths) and |
| 41 | `last_used_at` tracking |
| 42 | - [ ] single sign-on follow-ups (docs/oidc.md): silent renewal |
| 43 | (`prompt=none` on a short local session, which is what makes revoking an SSO |
| 44 | session propagate here), an admin view of who is linked to which `sub`, and |
| 45 | unlinking an account from the settings page |
| 46 | - [ ] secrets follow-ups (docs/secrets.md): authenticate `anvild secret` with an |
| 47 | ssh signature instead of the account password; per-step rather than per- |
| 48 | pipeline scoping; `ssh-rsa` recipients (needs an RSA-OAEP branch in both the |
| 49 | Rust and the browser halves) |
| 50 | |
| 51 | ### Ideas from Origin https://cursor.com/blog/git-at-any-scale |
| 52 | |
| 53 | Cursor's writeup of Continuity, their Spokes replacement. Most of the post is |
| 54 | scale machinery anvil does not need (replicas, consensus, rendezvous hashing, |
| 55 | S3 as the source of truth) because that exists to serve a monorepo's CI from a |
| 56 | hundred read replicas. Three things do transfer, ranked by value per line. |
| 57 | |
| 58 | - [ ] **packfile compaction. anvil has none at all.** Every push writes a new |
| 59 | pack via `gix_pack::Bundle::write_to_directory` |
| 60 | (`vendor/gitserver-core/src/receive_pack.rs`, `write_pack`) and nothing ever |
| 61 | consolidates them. Object lookup is O(packs) because each index is only |
| 62 | efficient per-pack, so every push makes every later browse, clone and CI |
| 63 | checkout slower, permanently. The periodic runner already exists |
| 64 | (`crates/anvil-core/src/periodic.rs`), so this is a new `PeriodicJob`, not new |
| 65 | infrastructure. Two levels: |
| 66 | - multi-pack-index via `gix_pack::multi_index::File::write_from_index_paths` |
| 67 | (pure gix, no CLI). One binary-searchable lookup across all packs. Start |
| 68 | here: small, self-contained, fixes a problem already accumulating |
| 69 | - geometric repack via `gix_pack::data::output`, the same machinery |
| 70 | upload-pack uses to build packs. More work. The post's warning about |
| 71 | repacking being an availability hazard is a replica problem; with one |
| 72 | node there is nothing to fail over |
| 73 | |
| 74 | - [ ] **SQLite is not in WAL mode.** `db::connect` |
| 75 | (`crates/anvil-core/src/db.rs`) sets no pragmas, so it runs on the default |
| 76 | rollback journal with web, ssh, the CI dispatcher and four periodic jobs all |
| 77 | against one file in one process. Readers block the writer, and copying a live |
| 78 | `.db` under a rollback journal can yield a corrupt file, which makes the |
| 79 | documented backup (tar the running volume, DEPLOY.md § Operations) unsound. |
| 80 | `PRAGMA journal_mode=WAL` plus `busy_timeout`. |
| 81 | |
| 82 | - [ ] **a push log (the WAL idea, minus S3, consensus and replicas).** What |
| 83 | transfers is the observation that the pack bytes plus the ref transaction are |
| 84 | a complete description of a push, so recording them stops the disk from being |
| 85 | precious. Both are already in scope at one place: `apply_commands` |
| 86 | (`vendor/gitserver-core/src/receive_pack.rs`) writes the pack, builds `edits`, |
| 87 | then calls `edit_references`. The entry goes between those two steps. |
| 88 | - buys, in order of how much we would use it: force-push undo as a UI button |
| 89 | (every ref's prior value is recorded), a reflog that survives gc, |
| 90 | rebuildable repos, and eventually continuous off-box backup |
| 91 | - keep it simple by ordering it right: a local append-only file first. No S3 |
| 92 | client, no dependency, no network in the push path. That alone gets undo |
| 93 | and provenance. Shipping entries off-box is a separate additive step |
| 94 | - the rule that makes it worth anything, and the easy one to skip: do not |
| 95 | ack the push until the entry is durable. A log that might be missing the |
| 96 | entry you need is worse than no log, because you will trust it |
| 97 | - caveat: this covers git only. Issues, users, CI runs, secrets, attachments |
| 98 | and artifacts are not in it. Build this and keep tarring the volume for |
| 99 | the rest and we have added a system without retiring one, so either frame |
| 100 | it as provenance plus undo (a feature) rather than backup (an ops story), |
| 101 | or pair it with continuous SQLite replication so both halves match. |
| 102 | |
| 103 | Related, prompted by the post rather than in it: `App` |
| 104 | (`crates/anvil-core/src/lib.rs`) mixes durable state (`db`, disk) with |
| 105 | process-local state (`ci_tx`, `vault`, `user_vault`, `sessions`, `jobs`) with |
| 106 | nothing marking which is which. Each in-memory field is documented as "lost on |
| 107 | restart, which is safe because...", which is correct, but the invariant lives in |
| 108 | comments. The discipline underneath Continuity is knowing exactly what is truth |
| 109 | and what is cache. Costs nothing at one process; first thing to break at two. |