Sync upstream PR #549 + upstream/main, Studio polish (HOOK_BIGTEXT, chromatic-glow), surface catalog#42
Open
Sync upstream PR #549 + upstream/main, Studio polish (HOOK_BIGTEXT, chromatic-glow), surface catalog#42
Conversation
- Link to claude.ai/design instead of claude.ai - Remove raw.githubusercontent download links (just GitHub with ↓ button) - Fix stale SKILL.md link text in prompting guide Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
…ign-links fix(docs): use claude.ai/design link, remove raw download option
…er-side-waveform-generation
The snapshot command resolved the HyperFrames runtime IIFE via a
relative path that walked up three directories to packages/core/dist/.
This only worked in the monorepo dev layout — npm/npx installs have
a flat dist/ folder with cli.js and the runtime side by side.
Without the runtime, window.__player was never created and the
snapshot fell back to seeking every __timelines entry to the same
absolute time. Sub-composition timelines expect relative time
(offset from their data-start), so all beats rendered beat-1 content.
Fix: resolve("hyperframe.runtime.iife.js") from __dirname (the dist/
folder itself), where the build already copies the runtime IIFE.
…eview-handoff docs(skills): clarify preview handoff URL
Change build order from concurrent to staged to prevent @hyperframes/engine from starting TypeScript compilation before @hyperframes/core generates src/generated/runtime-inline.ts. This fixes intermittent "Cannot find module './generated/runtime-inline'" errors when running bun run build. Co-Authored-By: Claude Opus 4.7 <[email protected]>
fix: resolve build race condition by ensuring core builds first
Adds the directory + SKILL.md frontmatter for a new skill that translates Remotion (React) compositions to HyperFrames (HTML+GSAP). This is the foundation PR; subsequent PRs in the stack add the eval harness, test corpus, translation references, and finally the SKILL.md body. The frontmatter description enumerates trigger phrases and explicit out-of-scope cases (useState/useEffect, async metadata, @remotion/lambda) so the skill bows out cleanly when a Remotion composition isn't a clean translation target — those should use the runtime interop pattern from PR heygen-com#214 instead. Validated with skill-creator's package_skill.py.
Extends RenderConfig.format with "png-sequence" and patches two correctness
gaps so the existing "webm" / "mov" values actually preserve the alpha
channel end-to-end.
Engine fixes:
- screenshotService.pageScreenshotCapture: drop optimizeForSpeed for PNG
captures. The fast path uses an alpha-unaware codec that crushes real
alpha values; kept for opaque jpeg captures where it is harmless.
- frameCapture: replace the inline setDefaultBackgroundColorOverride
block (which fired pre-navigation and was reset by page.goto) with a
proper initTransparentBackground() call inside initializeSession,
after the window.__hf readiness poll. This also injects the
html/body/[data-composition-id]{background:transparent !important}
stylesheet so compositions with custom body / #root backgrounds do not
defeat the override. Wired into both screenshot-mode and beginframe-mode
branches.
Producer:
- RenderConfig.format extended to "mp4" | "webm" | "mov" | "png-sequence"
with full JSDoc.
- Streaming encode is bypassed for png-sequence (frames go straight to
disk). FORMAT_EXT extended.
- New Stage-5 png-sequence branch: mkdir outputPath, copy captured PNGs as
frame_NNNNNN.png, copy audio.aac sidecar when audio is present.
- Stage-6 mux/faststart and the debug copy are wrapped in !isPngSequence.
- README.md: new "Transparent Video Output" section.
Tests:
- New fixture tests/transparency-regression/ tagged "transparency".
- New tsx script src/transparency-test.ts asserts pixel-level alpha for
webm + png-sequence outputs. Wired as "test:transparency".
- Default "test" / "test:update" scripts pass --exclude-tags transparency
so the golden-MP4 harness ignores the new fixture.
Verified locally on macOS arm64: typecheck clean across engine + producer,
producer renderOrchestrator vitest 10/10, transparency-test passes for
both webm and png-sequence with end-to-end pixel assertions.
…ails on the base SHA Per Miguel's review: the previous fixture had no body / root background, so it passed against both the buggy and fixed code. The fix this PR makes (the initTransparentBackground stylesheet injection in initializeSession) only matters when a composition paints over the CDP default-background-color override — exactly what we tell users not to do, but exactly what a regression test must do. Reproduced locally: - base SHA (2935be6): pixel (10,10) decodes as rgba [16,16,16,255] (opaque heygen-com#111 body bg leaks through the pre-navigation override that Chrome resets on goto) - this head: pixel (10,10) decodes as rgba [0,0,0,0] (initTransparentBackground injects [data-composition-id]{background:transparent !important} AFTER navigation, force-overriding the body bg) The pixel-level assertions in transparency-test.ts are unchanged — they already require alpha=0 at (10,10). With the body bg painted, that assertion now fails on any code path that doesn't actually preserve alpha end-to-end.
…et-next docs: clarify alpha PR target branch
…ackage-resolution fix(skills): load helper dependencies outside repo installs
…-export feat(producer): true alpha output for webm, mov, and png-sequence
… extraction misses A <video src='../assets/foo.mp4'> inside a sub-composition silently dropped from extraction; the rendered output froze on the first decoded frame for the entire clip, with no error in stdout. Root cause: browser URL resolver clamps '..' at origin root (studio preview loads fine), but path.join(projectDir, '../assets/foo.mp4') normalizes to parent-of-project/assets/foo.mp4, which usually doesn't exist. existsSync returns false, extraction is skipped, no frame lookup is built, the per-frame injector has nothing to swap, and the <video> element's first decoded frame paints every screenshot. - Adds resolveProjectRelativeSrc in videoFrameExtractor that mirrors browser clamping (literal join first, then leading '..' stripped). - Surfaces a loud stderr warning when the resolver misses. - Mirrors fix in audioMixer.ts (same bug for <audio src='../'>) and renderOrchestrator HDR probe loop. - +6 regression tests. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Skill (hyperframes-cli): three-pattern table (cutout-over-different-scene vs over-its-own-source vs over-different-take) + the two non-obvious rules (wrap video in non-timed div for opacity control, both videos data-start=0 for sync). Skill (hyperframes/patterns): worked text-behind-subject example. Docs: --quality flag, compositing pitfalls section, quality preset table. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Newer libavformat builds write the VP9-alpha sidecar tag as 'ALPHA_MODE' (uppercase); older builds write 'alpha_mode'. ffprobe.ts only checked the lowercase form, so files produced by recent ffmpeg encoders (including the output of 'hyperframes remove-background' itself) were misclassified as having no alpha channel. Knock-on effect: the producer extracted them as JPGs (no alpha), the injected <img> overlays were fully opaque rectangles, and any element below them on the z-stack (text, captions, other layers) silently disappeared from the rendered output — even though the studio preview rendered the same composition correctly via native <video> playback. Symptom in our repro: a text-behind-subject composition showed the headline correctly in studio preview but the production render covered the headline entirely with the opaque avatar image. Fix: read videoStream.tags.alpha_mode OR videoStream.tags.ALPHA_MODE. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Locks in the case-insensitive behavior alongside the existing alpha_mode (lowercase) test. If either path regresses, the producer would silently extract alpha-having webms as opaque JPGs and the injected <img> overlays would cover every element below them on the z-stack — a bug that doesn't surface in the studio preview, only in production renders. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
…ng on tags Tag-based alpha detection (alpha_mode / ALPHA_MODE / pix_fmt yuva*) is fundamentally brittle. Failure modes seen in the wild: - case-sensitivity across ffmpeg versions (alpha_mode vs ALPHA_MODE) - older muxers that omit the sidecar tag entirely - mp4-as-webm rewraps that drop the tag - ffprobe reporting yuv420p for VP9-with-alpha because the alpha plane lives in a Matroska BlockAdditional sidecar, not the main pix_fmt Each of those silently strips alpha at extraction time. The bug doesn't surface until the rendered output is missing layers — frustrating to debug, silent in stdout. The previous case-insensitive fix patched one of the failure modes; this commit removes the class. The robust alternative is codec-based: any bitstream that CAN carry alpha (VP9, VP8, ProRes 4444) gets the alpha-aware decoder and PNG output by default, regardless of what the tag says. The cost is a small file-size increase on opaque VP9/VP8 sources (cached PNGs vs JPGs); the benefit is no class of silent alpha loss from tag misdetection. - Adds codecMayHaveAlpha() + decoderForCodec() helpers and exports them. - Updates extractVideoFramesRange to force libvpx-vp9 / libvpx for VP9 / VP8 unconditionally (was: only when metadata.hasAlpha). - Updates resolveFrameFormat to default to PNG for any alpha-capable codec (was: only when metadata.hasAlpha). - +4 unit tests covering the codec table. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
- engine/chunkEncoder, engine/streamingEncoder: extend `-bf 0` to GPU h264
paths (nvenc, qsv, vaapi) and `-b_strategy 0` for qsv so GPU-encoded
outputs avoid negative-DTS freezes too — not just SW libx264.
- engine/videoFrameExtractor: detect mid-path traversal (e.g.
`assets/../../foo.mp4`) by normalizing first and re-anchoring at the
project root. Adds a regression test.
- engine/videoFrameExtractor: dedupe stderr "src not resolvable" warnings
by `video.src` so a comp with N broken sources logs once, not N times.
- engine/videoFrameExtractor.test: drop dynamic `require("node:fs")`,
use ES `import { writeFileSync } from "node:fs"`.
- engine/ffprobe: extract `readTagCI` helper for case-insensitive ffprobe
tag reads (will recur for other libavformat-versioned sidecar tags).
- cli/background-removal/pipeline: collapse Quality / QUALITIES /
QUALITY_CRF / DEFAULT_QUALITY / isQuality surface using
`Quality = keyof typeof QUALITY_CRF`.
- producer/renderOrchestrator: replace `v.src.startsWith("/")` with
`isAbsolute(v.src)` in the HDR probe path so Windows absolute paths
(`C:\...`) aren't treated as relative — matches the audioMixer guard.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
…person video The model removes background from any video with a person — we tested with avatars because they were convenient, but anyone can bring a talking-head clip, presenter footage, vlog, etc. Replace avatar-specific filenames (avatar.mp4 / brandon.mp4) with neutral subject.mp4 (or presenter.mp4 in the text-behind-subject example) and rephrase copy that read as if avatars were the only use case. Touches docs/guides/remove-background.mdx, hyperframes-media SKILL.md, and hyperframes/patterns.md. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
…ed+snapshot Four regression tests (font-variant-numeric, many-cuts, missing-host-comp-id, variables-prod) failed on this PR with `Unable to parse PSNR output at <last checkpoint>s`. Root cause: the harness derived all 100 checkpoints from the *rendered* video's container duration, then asked ffmpeg's PSNR filter to compare the same frame index from both videos. The encoder changes earlier in this PR add `-avoid_negative_ts make_zero` to the mux step. With AAC audio that shifts the first audio sample to t=0 instead of the encoder-delay offset, extending reported container duration by ~20ms without changing video frame count. For the four failing tests, the i=99 checkpoint then landed on a frame index that exists in the rendered video but not in the snapshot baseline (e.g. round(2.98998 * 24) = 72 in a 72-frame baseline). ffmpeg's PSNR filter ran on zero matched frames and emitted no `average:` line, so the parser threw. Fix: probe both videos and use min(rendered, snapshot) duration when spreading checkpoints. This is the correct semantics for symmetric PSNR comparison anyway — both videos must have a frame at every sampled time. The change is local to the harness; no encoder behavior changes, no baselines regenerated. Other regression tests with audio (chat, sub-composition-video, vignelli-stacking) passed because their checkpoint-99 frame index landed inside the baseline's frame range with several frames of slack. The four failing tests had round-number durations where a 20ms drift was enough to push the last checkpoint past `nb_frames - 1`.
…path-resolution-and-render-fixes fix: render robustness — sub-comp src paths, alpha tag case, encoder + matter improvements
## Problem The Blue Sweater intro HyperFrames project was only available as a standalone exported project zip. It was not installable from the public registry or visible in the Catalog Showcases group. ## What this fixes Adds `blue-sweater-intro-video` as a registry block with its composition, avatar image, and sound mix asset. The block is exposed through the generated catalog page, `docs/public/catalog-index.json`, and the Showcases navigation. The manifest and generated catalog page credit the creator as [Joe Sai](https://x.com/_blue_sweater_). ## Root cause Catalog-visible blocks are driven by `registry/registry.json`, each block's `registry-item.json`, generated docs/catalog files, and CDN-hosted preview media. The exported project had a valid standalone composition, but it had not been converted into that registry/catalog contract or uploaded to the docs preview CDN. ## Verification ### Local checks - `bun install` - `bun run build` - `bunx tsx scripts/generate-catalog-pages.ts` - `bun run generate:catalog-previews -- --only blue-sweater-intro-video` - `bun packages/cli/src/cli.ts add blue-sweater-intro-video --dir /tmp/hf-blue-sweater-install-test --no-clipboard --json` against a locally served registry - `bun packages/cli/src/cli.ts lint /tmp/hf-blue-sweater-install-test` returned 0 errors and 3 static GSAP overlap warnings from the supplied timeline/parser path - `bun packages/cli/src/cli.ts validate /tmp/hf-blue-sweater-install-test --timeout 5000` returned 0 runtime errors and 0 warnings, with contrast audit warnings only - `bun packages/cli/src/cli.ts inspect /tmp/hf-blue-sweater-install-test --at 0.5,2.5,5.5,9.8,11.2 --json` returned 0 layout issues - `bun packages/cli/src/cli.ts render /tmp/hf-blue-sweater-install-test --output /tmp/hf-blue-sweater-install-test/blue-sweater-intro-video-render.mp4 --fps 24 --quality draft --workers 3` - `ffprobe` reported the installed render duration as `12.000000` - `bunx oxfmt --check registry/registry.json registry/blocks/blue-sweater-intro-video/registry-item.json registry/blocks/blue-sweater-intro-video/blue-sweater-intro-video.html docs/docs.json docs/public/catalog-index.json docs/catalog/blocks/blue-sweater-intro-video.mdx` - `git diff --check` - `bunx vitest run packages/cli/src/commands/add.test.ts packages/core/src/registry/types.test.ts` ### Browser verification - Started a real local HyperFrames preview for the installed test project. - Used `agent-browser` to open `http://localhost:5198/api/projects/hf-blue-sweater-install-test/preview` at 1920x1080. - Verified the runtime registered `install-test` and `blue-sweater-intro-video` timelines. - Sought the block to the final card and verified `@_blue_sweater_` and the following state were visible. - Recorded an `agent-browser`-driven full animation pass; `ffprobe` confirmed a 1920x1080 WebM with 110 video frames. - Checked the fresh `agent-browser` session for page errors after the direct preview flow: `errors: []`. - Used `agent-browser` to load an HTML page with the exact generated CDN `video`/`poster` URLs; the browser reported `readyState: 4`, `videoWidth: 1920`, `videoHeight: 1080`, and `paused: false`. ### CDN upload Uploaded the generated preview media with AWS CLI to the existing docs image bucket path: - `s3://heygen-public/hyperframes-oss/docs/images/catalog/blocks/blue-sweater-intro-video.mp4` - `s3://heygen-public/hyperframes-oss/docs/images/catalog/blocks/blue-sweater-intro-video.png` Verified both public CDN URLs return `HTTP 200` with correct content type and immutable cache headers: - `https://static.heygen.ai/hyperframes-oss/docs/images/catalog/blocks/blue-sweater-intro-video.mp4` (`video/mp4`) - `https://static.heygen.ai/hyperframes-oss/docs/images/catalog/blocks/blue-sweater-intro-video.png` (`image/png`) ## Notes - Local-only browser proof artifacts: - `/tmp/hf-blue-sweater-browser-proof/fresh-final-card.png` - `/tmp/hf-blue-sweater-browser-proof/fresh-browser-flow.webm` - `/tmp/hf-blue-sweater-cdn-check.png` - Local-only installed render artifact: - `/tmp/hf-blue-sweater-install-test/blue-sweater-intro-video-render.mp4`
) ## Summary - Replace the flat 30s upload timeout with a size-adaptive calculation: `max(120s, bytes / 500KB/s)` - Metadata requests (presigned URL, complete) keep the original 30s timeout - Companion to the backend change removing the 64 MB upload limit in experiment-framework ## Context With the backend size limit removed, large projects (78 MB+) need proportionally longer to upload. A 78 MB project now gets ~164s, a 500 MB project ~17 min. The old 30s timeout would abort any upload over ~15 MB on a typical connection. ## Test plan - [x] All 4 existing vitest tests pass - [x] Build succeeds, no type errors - [x] Lint + format pass (oxlint + oxfmt) - [x] Timeout values verified for 10/78/200/500/1000 MB archives
Emit an inverse-alpha background plate alongside the cutout in a single inference pass. Same source RGB, alpha = 255 − mask. Dual-encoder pipeline runs in parallel; both outputs share the same --quality preset. This is a hole-cut plate (subject region transparent), not an inpainted clean plate — composite something opaque under it to fill the hole. Docs and skill cover when each is the right tool. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
… reasoning
Both `composition_file_too_large` and `timeline_track_too_dense` previously
said "Agents produce better results when large scenes are split into smaller
sub-compositions." The audience-flavored framing ("Agents produce better
results") doesn't tell a reader (agent or human) WHY smaller is better.
Reframe to concrete properties of smaller compositions: easier to read,
iterate on, and diff. The fixHint already covers the inspect/revise/validate
detail; the message now leads with a tight reason.
Per Abhay in #C0ACCNHLG3U:
> "an agent reading 'Agents produce better results' sounds weird. We should
> give the agent an actual reason why smaller is better for them."
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
…rding fix(lint): rephrase too-large composition warnings to give actionable reasoning
- Extract applyMask helper from postprocess and add 5 unit tests pinning the contract this PR is selling: fg.alpha + bg.alpha === 255 per pixel, RGB triples byte-identical between fg and bg, and bg=null path leaves the bg buffer untouched. Without these, a future postprocess change (mask threshold, premultiplied alpha, gamma) could silently break the inverse-alpha relationship and the existing plumbing tests would all still pass. - Add stdin 'error' listener inside spawnFfmpeg. If either encoder dies mid-render, Node emits an unhandled error on the dead writable on the next .write() and crashes the CLI before waitForExit's reject path can surface the encoder's stderr tail. Doubled encoder count = doubled failure surface, so this is worth pinning down. - Tighten stdio param to a 3-tuple so an accidental 1-element array fails at type-check. - Sharpen backpressure comment: write→true means "highWaterMark not exceeded," not "libuv flushed." Reuse-without-corruption is safe only because session.process is slow enough that libuv drains in between. Addresses review on PR heygen-com#637. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
…und-bg-output feat(cli): add --background-output to remove-background
…atch binding
Three issues in `bundleToSingleHtml` reported via Abhay's LLM-based code-validity
eval against the bundled output. Each is independently small; they share a single
PR because they're all artifacts of the bundler-output shape.
1. Empty `src=""` runtime placeholder (real bug)
`htmlBundler.ts:injectInterceptor` emitted
`<script data-hyperframes-preview-runtime="1" src=""></script>`
when no `HYPERFRAME_RUNTIME_URL` was configured. Empty `src` resolves to the
page URL itself; Chrome flags this as an infinite-fetch hazard. Three other
consumers (studioServer, validate, snapshot) post-process the placeholder to
substitute either a real URL or an inlined body — `bundleToSingleHtml` did
not, so the bundle wasn't actually self-contained despite the function name.
Fix: when no URL is configured, inline the runtime IIFE directly via
`getHyperframeRuntimeScript()`. Otherwise emit `src=…` as before.
2. Bare-semicolon lines between joined JS chunks (cosmetic)
Three sites used `chunks.join("\n;\n")` (body-script coalesce, local JS,
composition scripts) which produced a lone `;` on its own line between
chunks. Valid JS but a code smell. Replace with a `joinJsChunks()` helper
that ensures each chunk ends in `;` and joins on `\n`.
3. Empty `catch (_err) {}` in compositionScoping.ts (lint-noisy)
The `_err` underscore prefix signals "intentionally swallowed" but bundle-time
linters often don't honor that convention. Replaced with `catch { /* ... */ }`
(no binding, explanatory comment) — same behavior, no rule fires.
Tests: 2 new regression guards (runtime-not-empty-src, no-bare-semi) plus
existing tests updated to reflect the new inlined-runtime shape (the previous
"runtime block must not contain getElementById" assertion no longer holds
because the inlined body itself uses getElementById; replaced with a more
specific "author script not merged into runtime tag" check).
Issue #4 from the original report (Unterminated string at line 1111 col 18,
char 65497) was not directly reproducible after applying these fixes — esbuild
parses all 4 inline scripts in the rebundled output cleanly. The unterminated-
string symptom was likely a downstream artifact of the bare-semicolon joining
or the empty-src placeholder confusing the lint tool. If the original symptom
persists on a clean re-run against the fixed bundle, will open a follow-up PR
with a focused repro.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
…subs Per @vai-bot's review on hf#641: Important #1: dead `src=""` substitution sites ============================================= Now that `bundleToSingleHtml` inlines the runtime IIFE by default, the empty `src=""` placeholder is never emitted in the no-env-var path — the 5 downstream substitution sites that grep for `src=""` were dead. Two of them (studio dev server + studio vite preview) genuinely WANT the placeholder so they can hot-reload a local /api/runtime.js endpoint without re-inlining ~150 KB on every composition edit. Three of them (CLI validate, snapshot, layout) were just doing the same inlining the bundler already does. Resolution: - Add a `runtime: "inline" | "placeholder"` option to `BundleOptions`. Default is "inline" (matches the self-contained-bundle promise the function name makes). The two studio surfaces explicitly pass `{ runtime: "placeholder" }` to opt in. - studioServer.ts + studio/vite.config.ts: pass the option, keep their existing string-replace logic unchanged. - validate.ts + snapshot.ts + layout.ts: delete the now-redundant runtime substitution code (regex never matches the new inlined-runtime shape). Important #2: joinJsChunks ASI hazard ====================================== The new helper appended `;` to chunks not already ending in `;` and joined on `\n`. If a chunk ended with a `// line comment`, the appended semicolon was eaten by the comment, leaving the next chunk's first statement attached to the previous chunk's last expression — exactly the ASI hazard the helper exists to prevent. Fix: append `\n;` instead of `;` for chunks not already terminated. The newline closes the line comment, the standalone `;` becomes the statement separator. For typical chunks (already ending in `;`), output is unchanged — still clean `\n`-joined chunks with no bare-semicolon lines. Also added a trailing `;` to `wrapScopedCompositionScript`'s IIFE close (`})()` → `})();`) so composition scripts join cleanly without falling through to the `\n;` fallback. New test: regression guard at the chunk boundary verifies every inline script body in the bundle parses cleanly via esbuild even when a source JS file ends with a line comment. Verification ============ - `bun run --filter @hyperframes/core test` — 653/653 pass - `bun run --filter @hyperframes/cli test` — 243/243 pass - `bun run --filter @hyperframes/{core,cli,studio} typecheck` — clean - `bunx oxfmt --check` + `bunx oxlint` on all touched files — clean Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
CodeQL flagged the inline `<script>...</script>` regex as case-sensitive, which would miss `<SCRIPT>` tags. The bundler always emits lowercase, so this is a defense-in-depth fix matching the `/i` flag already used by the sibling regexes in this file (lines 37 & 75). Addresses CodeQL review on heygen-com#641.
CodeQL's `js/bad-tag-filter` rule flagged `</script>` as too strict — `</script >` (with whitespace before `>`) is valid HTML and would slip past the matcher. Changed to `</script\s*>` for full defense-in-depth. The bundler always emits the canonical form, so no real-traffic miss — this is hardening the test's parse-loop, not fixing a downstream bug. Addresses CodeQL alert on heygen-com#641.
CodeQL still flagged `</script\s*>` as too narrow — the rule wants tolerance for `</script\t\n bar>` (HTML parser treats trailing content in a close tag as part of the tag). Switched to `</script[^>]*>` for full coverage. The bundler still always emits the canonical `</script>`; this is test-side hardening, not a runtime fix.
Per CodeQL's `js/bad-tag-filter` recommendation, replace the regex-based `<script>` body extraction with a `parseHTML` + `querySelectorAll` walk. The rule explicitly says "use a parser library" — and linkedom is already imported in this file, so the diff is small. This eliminates the regex entirely, so the rule can no longer fire on this site (instead of chasing whitespace / case / trailing-content edge cases one at a time).
…c on scrub (heygen-com#639) * fix(runtime): clear play guard after hard seek to prevent audio desync on scrub When scrubbing the timeline during playback, syncRuntimeMedia detects the offset jump and hard-seeks the media element. But the in-flight play() guard (playRequested WeakSet) from the previous play() call prevented the next sync tick from re-issuing play() — leaving the element paused at the new position for 50-150ms while the GSAP timeline continued advancing. This caused audible audio desync after every scrub. Fix: clear playRequested on the element after a hard seek so the very next sync tick can re-issue play(). Also adds a lint rule (video_audio_double_source) that catches compositions where an unmuted <video> and a separate <audio> point to the same source — a pattern that causes double playback at runtime. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix(runtime): detect failed seeks past MP3 buffer and force full fetch Root cause: streaming MP3 with preload="metadata" only buffers the first ~15 seconds. Seeking past the buffered range silently fails — currentTime stays at 0 while the timeline advances, causing permanent audio desync that only a page refresh fixes. Three changes: 1. Move preload="auto" enforcement to run for ALL active elements on every sync tick (not just during play). This catches elements whose preload was overridden after init.ts set it. 2. After a hard seek, check if currentTime actually reached the target. If not (drift > 0.5s), call load() once to force the browser to fully fetch the media and build a complete seek index. 3. Clear the load-retry guard when the clip leaves its active window so re-entry can retry if needed. Reproduced on hyperframes.dev Hermes launch video: vo.mp3 buffered to 15.96s, seeking to 20s failed silently. bg-music.wav (fully buffered) was unaffected. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
…-and-joins fix(bundler): inline runtime body, drop bare-semi joins, drop empty catch binding
* feat: cache shader transition preview frames * fix: move shader transition loading to player
* feat: cache shader transition preview frames * fix: move shader transition loading to player * fix: render shader transitions for sdr compositions
…into hyperframes skill Cherry-pick the PR heygen-com#549 design-first pipeline from upstream and renumber the authoring flow to discovery → step 1 (design) → step 2 (prompt expansion) → step 3 (plan), so this fork's retention work can sit on the same intermediate brief that upstream agents produce. Adds shared references the fork was missing: video-composition (density, color presence, scale), beat-direction (rhythm templates), techniques (11 visual technique recipes), narration (voiceover pacing), prompt-expansion (step-2 brief format), design-picker (visual design.md generator + HTML template). Updates motion-principles with image-treatment and load-bearing GSAP rules, patterns with the text-behind-subject transparent-webm pattern, and the contrast-report and animation-map scripts to bootstrap their dependencies via the new package-loader. SKILL.md is the upstream version with three fork-specific entry points spliced into the references list: tts.md (kept here because we don't yet ship a hyperframes-media skill), retention-ladder.md, and retention-overdrive.md (previously orphaned — never linked from SKILL.md). Preserved as fork-only: - data-in-motion.md — the editorial chart-pack vocabulary (lollipop-timeline, grouped-bars, annotated-area, cyber-counter-burst) - visual-styles.md — the lighter style-library variant - references/transcript-guide.md, references/tts.md — full CLI guidance, since this fork does not (yet) ship a hyperframes-media skill split - retention-ladder.md, retention-overdrive.md — fork-only retention work Verified: 24/24 SKILL.md markdown links resolve, oxlint clean on the .mjs scripts, oxfmt clean across all changed files, npx tsx packages/cli/src/cli.ts --help lists every command including the fork-only score/optimize/el-tts/ script/images set. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Brings the production-grade engine, runtime, and Studio fixes from upstream that the fork was missing, on top of Phase 1's PR heygen-com#549 skill sync. Conflicts resolved (11 files): - bun.lock, packages/{cli,core}/package.json: dep unions, lockfile regenerated via bun install - skills/hyperframes/SKILL.md: re-merged Phase 1 fork entry points (retention-ladder, retention-overdrive, prompt-expansion, tts) on top of upstream's post-heygen-com#549 skill cleanup - packages/cli/src/help.ts: union — fork's el-tts/script/score/optimize/ images preserved, upstream's remove-background added - packages/cli/src/commands/render.ts: union — kept fork's logRenderCost accounting, added upstream's exit-after-complete + scheduleRenderProcessExit - packages/core/src/studio-api/createStudioApi.ts: union — fork's elevenlabs/anthropic/script/costs/images/storyline routes + upstream's new waveform route - packages/studio/src/player/hooks/useTimelinePlayer.ts: kept upstream's refactor to createTimelineElementFromManifestClip + findTimelineDomNodeForClip, re-attached fork's data-timeline-group passthrough as post-processing - packages/studio/src/player/components/Timeline.test.ts: union — fork's computeEffectiveTimelineDuration + deriveTimelineLaneLabel suites kept alongside upstream's formatTimelineTickLabel suite - packages/studio/src/components/sidebar/LeftSidebar.tsx: kept fork's mode-driven tab system, added upstream's onToggleCollapse as optional Hide button - packages/studio/src/App.tsx: union — fork's Direct/Edit toggle, sidebar toggle, Timeline visibility toggle restored alongside upstream's Capture frame link; states unioned; LeftSidebar receives both mode and onToggleCollapse Post-merge fixes: - packages/studio/src/player/store/playerStore.ts: dropped duplicate label field that auto-merge introduced - packages/cli/src/commands/render.ts: cost-log meta uses hdrMode (typed field) instead of dropped legacy hdr key - packages/studio/src/App.tsx: removed dead timelineEditorHintDismissed useState and matching import (was never read) Verified: 1736/1736 tests passing (core 1148, cli 257, studio 331); tsc, oxlint, oxfmt all clean; CLI prints fork's commands (el-tts, script, score, optimize, images) and upstream's new remove-background side by side. Brings: bundler runtime fixes (heygen-com#641), audio-desync-on-scrub fix (heygen-com#639), shader-transition cache (heygen-com#634), SDR shader transitions (heygen-com#640), PSNR sampling fix (heygen-com#627), --background-output flag (heygen-com#637), v0.4.45 release, PR heygen-com#546 Studio Design panel scaffolding, PR heygen-com#495 claude-design move, PR heygen-com#496 link fixes. Fork has chosen NOT to adopt the upstream hyperframes-media skill split — fork's tts.md and transcript-guide.md remain inside the hyperframes skill. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
…re, surface catalog Three targeted Studio-polish improvements layered on the freshly-merged upstream/main + PR heygen-com#546 Design panel scaffolding. The fourth handoff item (intermittent Timeline editor flicker on play / tab-switch) is deferred — the handoff filed three hypotheses but no confirmed repro, so any "fix" would be speculative; needs a fresh repro session. 1. HOOK_BIGTEXT — letter-cascade safety net The user's render of `my-first-video` showed the kinetic-text scene dropping arbitrary letters mid-headline ("y failing", "compre e si e fede al", "framewo k", "neithe goal") for the entire scene. The handoff filed two hypotheses — stagger-overruns-duration and layout- clipping. The fix below makes both failure modes structurally impossible regardless of which one was the cause: - `.hb-letter` defaults to `opacity: 1` (graceful degradation — if GSAP fails to register, every letter still renders visible) - A pre-cascade `tl.set(letters, { opacity:0, y:50 }, 0)` records the initial state at t=0 so seeks into the cascade window observe a real tween rather than a default-state element - A defensive `tl.set(letters, { opacity:1, y:0, clearProps:'transform' }, sceneDur - 0.01)` at the absolute end of the scene window guarantees the last captured frame has every letter at opacity:1 even if the earlier cascade math drifts under load Also adds an 8-test regression suite that pins the safety net's structure: one `.hb-letter` span per character, opacity:1 default in CSS, pre-tween set at t=0, final-frame snap regex, no stagger overrun at minimum scene duration, single-letter edge case, accent-word highlight preservation. 2. chromatic-glow atmosphere preset (cinematic-card foundation) Adds the eleventh atmosphere — a Vision-Pro / Apple-keynote spotlight halo built from a single intense radial gradient at the focal point plus a soft chromatic-aberration ring. Pure CSS, scoped per sceneId for cross-scene independence, intensifies for hook scenes, uses the active theme's accent color so each design.md gets its own ambient hue. Opt-in only — defaultAtmosphereForTemplate intentionally does not yet route any template to this preset, so existing renders are unchanged. This is the foundation for the upcoming spotlight-card / cinematic-card aesthetic upgrade tracked in the Studio polish handoff (Apple-keynote / Vision-Pro reference imagery). Adds an 8-test regression suite covering registry uniqueness, sceneId scoping, theme-color usage, hook-vs-body intensity, and the opt-in-only pin. 3. Catalog reference (skills surfacing) The upstream merge brought 44 catalog blocks (hyperframes.dev/Apple Money Count, Blue Sweater, Spotify, North Korea Locked Down, all transitions categories, all shader transitions, etc.) and 3 components (Grain Overlay, Grid Pixelate Wipe, Shimmer Sweep) into registry/. The new skills/hyperframes/references/catalog.md groups them by purpose (social/UI mockups, hooks/openers, logo/outro, CSS transitions, shader transitions, components, examples) with a short "use when" line per entry. SKILL.md now links to it from the References section so an authoring agent picks the right block to `hyperframes add` before scaffolding from empty. Verified: core 1164/1164 tests passing (was 1148; +16 new tests), tsc + oxlint + oxfmt clean across all changed files. CLI + Studio typecheck unaffected. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
9 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Three layered phases to bring the fork onto upstream/main and ship the concrete Studio polish work from the
studio-polish-and-cinematic-aesthetic.mdhandoff.1. Skill sync —
e45c91cPulls upstream PR heygen-com#549's design.md-first architecture: renumbered 1/2/3 flow, 6 new shared references (
beat-direction,design-picker,narration,prompt-expansion,techniques,video-composition),package-loader.mjs,design-picker.htmltemplate. Fork-only files preserved (chart-packdata-in-motion.md,visual-styles.md,retention-ladder.md,retention-overdrive.md, plustranscript-guide.md/tts.mdsince this fork has NOT adopted upstream'shyperframes-mediaskill split).2. Upstream merge —
3236a19Pulls 96 upstream commits / 386 files. 11 conflicts resolved as deliberate unions:
bun.lock,packages/{cli,core}/package.json— dep unions, lockfile regeneratedskills/hyperframes/SKILL.md— re-merged Phase 1 fork entries against post-PR-feat(skills): design.md integration, shared video references, Claude Design gaps heygen-com/hyperframes#549 upstream cleanuppackages/cli/src/help.ts— fork'sel-tts/script/score/optimize/imagespreserved alongside upstream's newremove-backgroundpackages/cli/src/commands/render.ts— fork'slogRenderCost+ upstream'sexit-after-complete/scheduleRenderProcessExitpackages/core/src/studio-api/createStudioApi.ts— fork's elevenlabs/anthropic/script/costs/images/storyline routes + upstream's new waveform routepackages/studio/src/player/hooks/useTimelinePlayer.ts— kept upstream'screateTimelineElementFromManifestClip+findTimelineDomNodeForCliprefactor, re-attached fork'sdata-timeline-grouppassthrough as post-processingpackages/studio/src/player/components/Timeline.test.ts— fork'scomputeEffectiveTimelineDuration+deriveTimelineLaneLabelsuites kept alongside upstream'sformatTimelineTickLabelsuitepackages/studio/src/components/sidebar/LeftSidebar.tsx— kept fork's mode-driven 7-tab system + fullscreen-expand, accepted upstream'sonToggleCollapseas additional optional proppackages/studio/src/App.tsx— union: fork's Direct/Edit toggle + sidebar-collapse + Timeline visibility toggle restored alongside upstream's Capture frame link (auto-merge had silently dropped the Timeline button); states unioned; LeftSidebar receives bothmodeandonToggleCollapseBrings: bundler runtime fixes (heygen-com#641), audio-desync-on-scrub fix (heygen-com#639), shader-transition cache (heygen-com#634), SDR shader transitions (heygen-com#640), PSNR sampling fix (heygen-com#627),
--background-outputflag (heygen-com#637), v0.4.45 release, PR heygen-com#546 Studio Design panel scaffolding, PR heygen-com#495 + heygen-com#496 docs reorgs.3. Studio polish —
37e7a5aThree of the four handoff items:
.hb-letterdefaults toopacity:1(graceful degradation), pre-cascadetl.set(letters, {opacity:0,y:50}, 0)so seeks observe a real tween, defensive final-frame snap atsceneDur - 0.01so the last captured frame always has every letter visible. 8-test regression suite pins the structure.sceneId, intensifies for hooks, uses the active theme's accent. Opt-in only —defaultAtmosphereForTemplatedoes not yet route any template to it, so existing renders are unchanged. Foundation for the cinematic-card upgrade. 8-test regression suite.skills/hyperframes/references/catalog.mdgroups the 44 blocks + 3 components + 8 examples by purpose (social/UI mockups, hooks/openers, logo/outro, CSS transitions, shader transitions). SKILL.md links to it so an authoring agent picks the right block tohyperframes addbefore scaffolding empty.Deferred: the fourth handoff item — intermittent Timeline editor flicker on play / tab-switch. Three hypotheses on file (cleanup race, ResizeObserver/layout-effect timing, missing
min-height: 0) but no confirmed repro path. Held back until a fresh repro session.Verified
tscclean across cli / core / studiooxlint+oxfmtclean across all changed filesnpx tsx packages/cli/src/cli.ts --helplists every command including fork-onlyel-tts/script/score/optimize/imagesand upstream's newremove-backgroundTest plan
bun install && bun run --cwd packages/core test && bun run --cwd packages/cli test && bun run --cwd packages/studio testnpx tsx packages/cli/src/cli.ts cataloglists all 44 blocksatmosphere: "chromatic-glow"— confirm halo renders with active theme accentnpx hyperframes preview— Studio header has Direct/Edit + sidebar + Capture + Timeline + Renders, all functional🤖 Generated with Claude Code