feat: add Studio NLE playback controls

[miguel-heygen]

Problem

HyperFrames Studio made frame-accurate playback review slower than expected for editor-style workflows. Issue #527 called out missing loop playback, frame display/jump controls, preview-focused Space handling, frame stepping, and NLE-style J/K/L shuttle controls.

What this fixes

• Adds a persistent Studio loop toggle and makes the playback loop restart when enabled.
• Adds a time/frame display toggle plus a jump-to-frame input in the player controls.
• Adds frame math helpers and frame-step behavior at the Studio preview frame rate.
• Expands keyboard handling so preview-focused Space toggles playback, ArrowLeft/ArrowRight step frames, Shift+Arrow steps 10 frames, and J/K/L shuttle controls work from the preview/timeline surface while ignoring form/button/slider targets.
• Adds J/K/L shuttle behavior: J plays backward, K pauses, L plays forward, repeated J/L ramps 1x -> 2x -> 4x, and K-held J/L frame-steps.
• Makes the preview wrapper focusable so keyboard playback shortcuts work after focusing the preview area.

Root cause

The Studio playback layer only exposed mouse scrubbing, basic play/pause, a seconds-based readout, and slider-local arrow-key nudges. The global Space shortcut was also gated to document.body, so it stopped working once the actual preview/editor surface had focus. Studio needed a single playback-control layer above the runtime adapter that could translate editor keyboard intent into deterministic seek/play/pause operations.

Verification

Local checks

• bun install
• bun run --filter @hyperframes/core build:hyperframes-runtime
• bunx oxfmt --check packages/studio/src/player/lib/time.ts packages/studio/src/player/lib/time.test.ts packages/studio/src/player/store/playerStore.ts packages/studio/src/player/store/playerStore.test.ts packages/studio/src/player/hooks/useTimelinePlayer.ts packages/studio/src/player/components/PlayerControls.tsx packages/studio/src/player/components/PlayerControls.test.ts packages/studio/src/components/nle/NLEPreview.tsx
• bunx oxlint packages/studio/src/player/lib/time.ts packages/studio/src/player/lib/time.test.ts packages/studio/src/player/store/playerStore.ts packages/studio/src/player/store/playerStore.test.ts packages/studio/src/player/hooks/useTimelinePlayer.ts packages/studio/src/player/components/PlayerControls.tsx packages/studio/src/player/components/PlayerControls.test.ts packages/studio/src/components/nle/NLEPreview.tsx
• bun run --filter @hyperframes/studio test -- src/player/lib/time.test.ts src/player/store/playerStore.test.ts src/player/components/PlayerControls.test.ts src/player/hooks/useTimelinePlayer.test.ts -> 4 files passed, 52 tests passed
• bun run --filter @hyperframes/studio typecheck
• bun run --filter @hyperframes/studio build
• git diff --check
• Lefthook during commit -> lint, format, typecheck, commitlint pass

Browser verification

• Created a temp project at /tmp/hf-studio-nle-controls with an animated 10s GSAP timeline.
• Started local Studio preview via bun run --filter @hyperframes/cli dev -- preview /tmp/hf-studio-nle-controls at http://localhost:5194.
• Used agent-browser to verify:
  • loop toggle changes to active state
  • frame display shows current / total frames
  • jump-to-frame input moves the seek position to frame 45 / frame 150
  • focused preview accepts Space play/pause
  • ArrowRight advances one frame from preview focus
  • J plays backward from frame 150 to a lower frame, then K stops
  • agent-browser-driven recording of the tested flow completed

Notes

• Local proof artifacts are intentionally not committed:
  • qa-artifacts/studio-nle-controls/frame-controls.png
  • qa-artifacts/studio-nle-controls/playback-controls.webm
• Closes Studio should support Remotion/NLE-style playback controls #527.

[vanceingalls]

Staff review: requesting changes.

The NLE control set is useful and most of the playback mechanics are going in the right direction, but the keyboard shortcut scoping needs tightening before this lands.

packages/studio/src/player/hooks/useTimelinePlayer.ts installs capture-phase keydown listeners on window and the iframe, then handles Space, ArrowLeft/ArrowRight, and J/K/L for any target that is not an input/textarea/select/contenteditable. That is broader than the PR intent of preview/timeline-surface shortcuts and it breaks existing interactive controls:

• focused buttons are not excluded, so pressing Space on Loop, the time/frame toggle, speed menu, timeline toggle, or any other Studio button toggles playback instead of activating the focused button
• the seek bar is role="slider" and already has its own arrow-key handler in PlayerControls; the capture listener runs first, steps a frame, and prevents default before the slider-local behavior gets a clean event
• other focusable Studio UI outside the preview/timeline surface can now unexpectedly play, pause, or seek the composition

Please either scope these shortcuts to the preview/timeline surface explicitly, or expand the target guard to ignore native/focusable interactive controls such as button, a[href], [role="button"], [role="slider"], [role="spinbutton"], and similar controls. I would also add regression coverage for Space on a focused toolbar button and ArrowRight on the seek slider so this does not drift again.

I did not find another blocker in the frame math, loop handling, or reverse-shuttle implementation from this patch.

[miguel-heygen]

@vanceingalls addressed the shortcut scoping blocker.

Changes made:

• expanded the playback shortcut target guard so focused native/focusable controls are ignored (button, a[href], relevant ARIA controls including role="slider", role="spinbutton", etc.)
• kept preview/body playback shortcuts working for Space, J/K/L, and ←/→
• added regression coverage for focused toolbar buttons and the seek slider
• re-verified locally that Space on the focused Loop button activates the button without advancing playback, ArrowRight on the focused seek slider reaches the slider handler, and preview/body shortcuts still work

Validation:

• oxlint
• oxfmt --check
• targeted Studio tests
• Studio typecheck
• Studio build
• local browser verification

[vanceingalls]

Staff follow-up review: still requesting changes.

The earlier focused-button / seek-slider scoping blocker is largely addressed: the shortcut guard now ignores native controls and the slider role, and the targeted unit tests pass. I found two remaining shortcut-scope regressions before this is ready:

1. packages/studio/src/player/hooks/useTimelinePlayer.ts: handlePlaybackKeyDown handles Space, ArrowLeft/ArrowRight, and J/K/L without excluding modified key chords. Because the listener is installed capture-phase on window, chords like Alt+ArrowLeft, Cmd/Ctrl+L, or Cmd/Ctrl+K can be consumed as frame-step/shuttle commands instead of reaching browser/app shortcuts. Please ignore playback shortcuts when metaKey, ctrlKey, or altKey is set, except for any explicitly supported modifier combo.

2. packages/studio/src/player/hooks/useTimelinePlayer.ts: Arrow playback shortcuts still conflict with caption editing. CaptionOverlay already uses Arrow keys to nudge selected caption words; the new capture listener sees those Arrow events first and seeks a frame before the caption nudge handler runs. Please scope Arrow frame-stepping to the preview/timeline playback surface, or otherwise suppress it while caption edit mode has a selected caption word / caption overlay is active.

Validation I ran locally on the fetched branch:

• oxfmt --check on changed Studio files
• oxlint on changed Studio files
• bun run --filter @hyperframes/studio test -- src/player/lib/time.test.ts src/player/store/playerStore.test.ts src/player/components/PlayerControls.test.ts src/player/hooks/useTimelinePlayer.test.ts
• bun run --filter @hyperframes/studio typecheck

[miguel-heygen]

@vanceingalls addressed the remaining shortcut-scope issues; current head is 5fe53a0f.

Changes made:

• playback shortcuts now ignore metaKey, ctrlKey, and altKey before tracking pressed keys, so modified chords like Alt+ArrowLeft and Cmd/Ctrl+K/L are not consumed by the capture listener
• Arrow frame-stepping now defers while caption edit mode has a selected caption word, so CaptionOverlay owns plain Arrow nudges instead of the player seeking first
• caption nudging now also ignores metaKey/ctrlKey/altKey, so modified browser/app chords stay unhandled after playback defers
• moved the caption nudge key guard into src/captions/keyboard.ts so the React component file stays Fast Refresh clean
• added regression coverage for modified playback shortcuts, caption-selected Arrow deferral, and caption nudge modifier handling

Validation:

• bun run --filter @hyperframes/studio test -- src/player/lib/time.test.ts src/player/store/playerStore.test.ts src/player/components/PlayerControls.test.ts src/player/hooks/useTimelinePlayer.test.ts src/captions/keyboard.test.ts
• bun run --filter @hyperframes/studio typecheck
• bun run --filter @hyperframes/studio build
• bunx oxlint / bunx oxfmt --check on changed Studio files
• agent-browser playback shortcut pass: plain Shift+ArrowRight still frame-steps; Alt+ArrowLeft and Ctrl+K are not prevented and do not move the playhead
• agent-browser caption pass: selected caption word moves on plain ArrowRight; Alt+ArrowRight is not prevented and does not move the caption or playhead

Review was re-requested after validation.

[miguel-heygen]

Merge activity

• Apr 29, 3:04 AM UTC: @miguel-heygen merged this pull request with Graphite.