collin/strudel-claude

pre-demo / README.md

1	# Strudel × Claude
2
3	Live-code music by chatting. A [Strudel](https://strudel.cc) IDE on the left,
4	a Claude chat panel on the right. Describe the music you want and Claude writes
5	a runnable Strudel pattern into the editor and plays it.
6
7	```
8	┌────────────────────────────┬──────────────────────┐
9	│ 🌀 Strudel IDE │ ✦ Claude chat │
10	│ (CodeMirror editor) │ "dark techno with │
11	│ ▶ Play / ■ Stop (toggle) │ a rolling bass" │
12	│ stack( s("bd*4") ... ) │ → writes + plays it │
13	└────────────────────────────┴──────────────────────┘
14	```
15
16	## Architecture
17
18	- Frontend (Vite): CodeMirror editor + `@strudel/web` audio engine + chat UI.
19	Sounds come from Strudel's standard prebake: 71 drum machines, the GM soundfont
20	set, VCSL instruments, and a handful of character samples.
21	- Backend (Express): holds your Anthropic key, calls the Claude Messages API
22	(`claude-opus-4-8`) with the system prompt from `PROMPT.md` and structured
23	output (`{ message, code }`). The browser never sees the key.
24
25	```
26	browser ──/api/generate──▶ Express ──▶ Claude API
27	◀── { message, code } ──
28	```
29
30	## Setup
31
32	Credentials — any of these works:
33
34	- `ant auth login` (recommended), or
35	- `cp .env.example .env` and set `ANTHROPIC_API_KEY`, or
36	- nothing at all: click 🔑 Add API key at the bottom of the sidebar and paste
37	your own. It is kept in that browser's localStorage and sent to the backend in
38	an `x-anthropic-key` header, which uses it for that request only — never
39	logged, never stored server-side. A personal key takes precedence over the
40	server's, so several people can share one instance with their own billing.
41
42	```bash
43	npm install
44	npm run dev # starts Vite (5173) + backend (8787) together
45	```
46
47	Open http://localhost:5173, press ▶ Play for the starter pattern, or ask
48	Claude on the right (Cmd/Ctrl+Enter to send).
49
50	## Driving it from a terminal
51
52	The app broadcasts what it is playing — the code, the timing of every event in
53	it, and the chat transcript — over a WebSocket at `/ws`. A terminal client
54	draws all three and can send prompts back:
55
56	```bash
57	npm run tui # from a checkout
58	curl -s http://localhost:8787/tui.py \| python3 - localhost:8787 # from anywhere
59	```
60
61	`tui/strudel-tui.py` is one file, standard library only — no pip install, and
62	the server hands it out at `/tui.py` as `text/plain`, so you can read it in a
63	browser or `curl -sO` it before running it. It takes the app's address
64	(`localhost:8787`, `https://you.ngrok.app`, `ws://…/ws` — all understood) and
65	otherwise defaults to `localhost:8787`.
66
67	```
68	● playing cycle 2.000 0.500 cps 120 bpm
69	────────────────────────────────────────────────────────────────────────────
70	1 stack( │› you
71	2 s("bd*4").gain(0.9), │ give me a dark rolling techno
72	3 s("~ cp").room(0.3), │
73	4 note("<c2 g1>").s("sawtooth") │✦ claude
74	5 ) │ Rolling kick on every beat,
75	↑ the characters that made each event │ offbeat hats, a low saw bass.
76	light up as it sounds and fade out │
77	across its length │✦ claude is thinking…
78	╷·····┃···············╷······················╷············· ← one cycle, ┃ = now
79	› make it half time and add a clap on 3
80	```
81
82	Panes sit side by side when the terminal is at least 100 columns wide and stack
83	when it isn't. Keys:
84
85	\| \| \|
86	\|---\|---\|
87	\| `q` \| quit \|
88	\| `i` or `Enter` \| write a prompt (`Enter` sends, `Esc` cancels, `^U`/`^W` edit) \|
89	\| `j` / `k` \| scroll the code \|
90	\| `J` / `K` \| scroll the chat \|
91	\| `f` \| follow the sounding code (on by default) \|
92
93	A prompt typed here goes through the same path as the composer in the page, so
94	the turn appears in both, reverts in both, and is saved to the session like any
95	other. If a turn is already in flight the new prompt is refused rather than
96	queued — two user messages in a row is an API error — and the terminal says so.
97
98	`--light` for a light-background terminal. It needs 24-bit colour, which every
99	current terminal has.
100
101	### How the timing gets there
102
103	Each message carries a window of upcoming events plus the playhead position
104	and tempo, twice a second — not a frame per redraw. The viewer runs its own
105	clock and animates between messages, so it stays smooth over a slow link and
106	survives the browser tab throttling its timers to 1 Hz in the background. That
107	last part matters: Strudel's own highlighting is `requestAnimationFrame`-driven
108	and stops dead in a hidden tab, which is exactly when you are looking at the
109	terminal instead — so `src/broadcast.js` polls the scheduler rather than
110	hanging off the draw loop.
111
112	The hub takes one producer (the browser tab that owns the audio) and any number
113	of viewers. A viewer may send exactly one kind of message — a prompt — and the
114	hub drops everything else it sends, so a terminal cannot rewrite the editor,
115	work the transport, or push anything onto another viewer's screen.
116
117	Open the app in two tabs and the first one keeps the slot; the second is told
118	to stand by and stays silent until the first goes away, at which point it is
119	promoted and re-announces itself. The terminal shows `(+1 idle tab)` so it is
120	clear why it isn't following the tab you happen to be looking at. The slot only
121	changes hands when it is free — an earlier version gave it to whoever connected
122	last, and since a displaced tab reconnects half a second later, two open tabs
123	took turns evicting each other and the terminal flipped between them twice a
124	second, forever.
125
126	It is unauthenticated, though. If you expose the port, anyone who can reach it
127	can read your session and spend your API credit on prompts, which is a bigger
128	deal than it was when viewers could only watch. Keep it on localhost, or put
129	something in front of it, unless you mean to share the console.
130
131	## How it works
132
133	- `src/strudel.js` — wraps `initStrudel()` / `evaluate()` / `hush()`.
134	- `src/main.js` — editor, transport, and chat; drops Claude's code into the
135	editor and auto-plays it.
136	- `src/sessions.js` — localStorage-backed sessions (code + chat per session).
137	- Revert: every user message stores the code as it was before that turn, so
138	hovering Claude's reply and clicking ↩ undoes that turn — rewinding both the
139	chat and the pattern. The status bar then offers a one-level Undo.
140	- `src/apikey.js` — the bring-your-own-key control; `GET /api/config` tells it
141	whether the server has a key of its own.
142	- `server/index.js` — the Claude call, history trimming and prompt caching.
143	- `server/broadcast.js` — the `/ws` timing hub; `src/broadcast.js` feeds it and
144	`tui/strudel-tui.py` consumes it.
145	- `PROMPT.md` — the entire system prompt, read from disk at boot. Edit it to
146	teach Claude new sounds or change house style; no code change needed, and the
147	prompt diffs like any other file. Restart the API to pick up edits.
148
149	### Keeping the prompt honest
150
151	`PROMPT.md` tells Claude not to invent sound names, because an unknown name in
152	Strudel plays silence rather than raising — a wrong name is a part the user
153	never hears. So section 3 of the prompt has to match what `src/strudel.js`
154	actually prebakes. If you add or remove a sample pack, update that section.
155
156	## Ideas to extend
157
158	- Stream Claude's reply token-by-token into the chat bubble.
159	- Add a "remix current selection" action.
160	- Swap the plain editor for `@strudel/codemirror` to get Strudel-native
161	highlighting and event flashing.
162	- Persist sessions / share patterns by URL.