An AI-assisted audiovisual system. The human writes and performs the music. The AI helps build the visuals.
Ableton Live is the musical, timing and performance source. TouchDesigner is the visual engine. AI sits above the real-time layer as a TouchDesigner engineering and creative-assistance layer — it does not sit in the signal path, and it does not write the music.
TD_Ableton_OS/ — the AV system
The main body of work. Architecture, MCP stack, control interface contract, benchmark design, technique catalog, asset pipeline.
North star: show the agent a visual reference and say "build something like this in TouchDesigner using these musical and control inputs" — and it understands the technique, plans the implementation, constructs it, sees its own output, iterates toward the reference, debugs it, optimizes it, and leaves an editable performance-ready network.
Start with ROADMAP.md, then README.md and TODO.md.
Architecture is frozen at v0.2. Current phase: environment verified, control interface next, Benchmark #1 is the gate.
CDJ_Ableton_Clock/ — tempo topology
Separate, independent track. Bridging Pro DJ Link ↔ Ableton Link with Beat Link Trigger so that one control — a CDJ pitch fader or the APC40 tempo encoder — moves tempo across CDJs, Ableton, the Elektrons and TouchDesigner.
Start with CLOCK_TABLE.md for the reference tables, CLOCK.md for the reasoning.
This does not unfreeze the AV architecture. No new channel, no Control Surface slot. It promotes bpm / beat_phase / bar_phase from estimated to authoritative.
| Mac | TouchDesigner 2025.31550 · Ableton Live 12.4.3 Suite |
| Audio | MOTU UltraLite → PA · Digis + MOTU aggregate device |
| DJ | 2 × CDJ-2000NXS2 · DJM-900NXS2 |
| Control | Akai APC40 mk2 · Akai APC Key 25 MK2 |
| Instruments | Elektron Digitakt · Elektron Digitone |
Two planes, and they must not mix.
Real-time signal travels as audio, MIDI, OSC and Ableton Link — deterministic, low latency. MCP is where the AI inspects, builds, debugs and documents. No kick drum ever travels through JSON.
HUMAN
│
ABLETON LIVE ──► TDAbleton / Link / MIDI / Audio ──► TOUCHDESIGNER ──► OUTPUT
▲ ▲
│ │
Ableton MCP TDPilot
└────────────────── CLAUDE ──────────────────────────┘
(control plane)
| Project | State |
|---|---|
| TD_Ableton_OS | architecture frozen v0.2 · environment verified · Benchmark #1 pending |
| CDJ_Ableton_Clock | proposal · rig confirmed · not yet built |
Nothing is installed or built yet. This repo is currently design and decision record.