anvilsign in

collin/anvil · 215b6942

TODO: ideas from Cursor's Origin/Continuity post

Collin Richards · 2026-08-25 04:30 UTC · 215b6942e0f97fc965b201fd4d5c6b7627eb957e · parent 0edc9bd2 · browse files

modifiedTODO.md+61 −1
⋯ 45 unchanged lines
4646 - [ ] secrets follow-ups (docs/secrets.md): authenticate `anvild secret` with an
4747 ssh signature instead of the account password; per-step rather than per-
4848 pipeline scoping; `ssh-rsa` recipients (needs an RSA-OAEP branch in both the
49- Rust and the browser halves)
49+ Rust and the browser halves)
50+
51+### Ideas from Origin https://cursor.com/blog/git-at-any-scale
52+
53+Cursor's writeup of Continuity, their Spokes replacement. Most of the post is
54+scale machinery anvil does not need (replicas, consensus, rendezvous hashing,
55+S3 as the source of truth) because that exists to serve a monorepo's CI from a
56+hundred read replicas. Three things do transfer, ranked by value per line.
57+
58+- [ ] **packfile compaction. anvil has none at all.** Every push writes a new
59+ pack via `gix_pack::Bundle::write_to_directory`
60+ (`vendor/gitserver-core/src/receive_pack.rs`, `write_pack`) and nothing ever
61+ consolidates them. Object lookup is O(packs) because each index is only
62+ efficient per-pack, so every push makes every later browse, clone and CI
63+ checkout slower, permanently. The periodic runner already exists
64+ (`crates/anvil-core/src/periodic.rs`), so this is a new `PeriodicJob`, not new
65+ infrastructure. Two levels:
66+ - multi-pack-index via `gix_pack::multi_index::File::write_from_index_paths`
67+ (pure gix, no CLI). One binary-searchable lookup across all packs. Start
68+ here: small, self-contained, fixes a problem already accumulating
69+ - geometric repack via `gix_pack::data::output`, the same machinery
70+ upload-pack uses to build packs. More work. The post's warning about
71+ repacking being an availability hazard is a replica problem; with one
72+ node there is nothing to fail over
73+
74+- [ ] **SQLite is not in WAL mode.** `db::connect`
75+ (`crates/anvil-core/src/db.rs`) sets no pragmas, so it runs on the default
76+ rollback journal with web, ssh, the CI dispatcher and four periodic jobs all
77+ against one file in one process. Readers block the writer, and copying a live
78+ `.db` under a rollback journal can yield a corrupt file, which makes the
79+ documented backup (tar the running volume, DEPLOY.md § Operations) unsound.
80+ `PRAGMA journal_mode=WAL` plus `busy_timeout`.
81+
82+- [ ] **a push log (the WAL idea, minus S3, consensus and replicas).** What
83+ transfers is the observation that the pack bytes plus the ref transaction are
84+ a complete description of a push, so recording them stops the disk from being
85+ precious. Both are already in scope at one place: `apply_commands`
86+ (`vendor/gitserver-core/src/receive_pack.rs`) writes the pack, builds `edits`,
87+ then calls `edit_references`. The entry goes between those two steps.
88+ - buys, in order of how much we would use it: force-push undo as a UI button
89+ (every ref's prior value is recorded), a reflog that survives gc,
90+ rebuildable repos, and eventually continuous off-box backup
91+ - keep it simple by ordering it right: a local append-only file first. No S3
92+ client, no dependency, no network in the push path. That alone gets undo
93+ and provenance. Shipping entries off-box is a separate additive step
94+ - the rule that makes it worth anything, and the easy one to skip: do not
95+ ack the push until the entry is durable. A log that might be missing the
96+ entry you need is worse than no log, because you will trust it
97+ - caveat: this covers git only. Issues, users, CI runs, secrets, attachments
98+ and artifacts are not in it. Build this and keep tarring the volume for
99+ the rest and we have added a system without retiring one, so either frame
100+ it as provenance plus undo (a feature) rather than backup (an ops story),
101+ or pair it with continuous SQLite replication so both halves match.
102+
103+Related, prompted by the post rather than in it: `App`
104+(`crates/anvil-core/src/lib.rs`) mixes durable state (`db`, disk) with
105+process-local state (`ci_tx`, `vault`, `user_vault`, `sessions`, `jobs`) with
106+nothing marking which is which. Each in-memory field is documented as "lost on
107+restart, which is safe because...", which is correct, but the invariant lives in
108+comments. The discipline underneath Continuity is knowing exactly what is truth
109+and what is cache. Costs nothing at one process; first thing to break at two.