Add runanywhere-electron: on-device AI SDK for Electron (Windows-first)#561
Merged
Conversation
…ate mmproj, drop extracted archives
…uotes, block remote speech models
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@sdk/runanywhere-electron/src/process/main.ts`:
- Around line 40-42: Update the connection tracking logic in connect() so the
webContents.destroyed listener is attached only when the WebContents is not
already present in this.connected. Preserve the existing Set tracking and
removal behavior, while avoiding duplicate listeners for repeated connect()
calls.
- Around line 56-58: Update the exit handler associated with the child process
in the process manager to clear this.child only when it still references the
exiting child instance. Preserve the existing renderer notification behavior,
while preventing an old child’s delayed exit event from clearing a newly
connected child.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 68b0dad8-597a-42b8-9d0c-242ed5d0a58d
📒 Files selected for processing (12)
.github/workflows/electron-sdk-ci.ymlexamples/electron/RunAnywhereAI/main.jsexamples/electron/RunAnywhereAI/renderer.jssdk/runanywhere-electron/.gitignoresdk/runanywhere-electron/src/download.tssdk/runanywhere-electron/src/process/dispatch.tssdk/runanywhere-electron/src/process/host.tssdk/runanywhere-electron/src/process/main.tssdk/runanywhere-electron/src/process/preload.tssdk/runanywhere-electron/src/process/rpc.tssdk/runanywhere-electron/test/unit/dispatch.test.jssdk/runanywhere-electron/test/unit/main.test.js
💤 Files with no reviewable changes (1)
- sdk/runanywhere-electron/.gitignore
🚧 Files skipped from review as they are similar to previous changes (9)
- sdk/runanywhere-electron/src/process/rpc.ts
- .github/workflows/electron-sdk-ci.yml
- examples/electron/RunAnywhereAI/main.js
- sdk/runanywhere-electron/src/process/dispatch.ts
- sdk/runanywhere-electron/test/unit/dispatch.test.js
- sdk/runanywhere-electron/src/download.ts
- sdk/runanywhere-electron/test/unit/main.test.js
- sdk/runanywhere-electron/src/process/host.ts
- sdk/runanywhere-electron/src/process/preload.ts
…sumable downloads
There was a problem hiding this comment.
Actionable comments posted: 6
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@sdk/runanywhere-electron/src/download.ts`:
- Around line 118-120: Update the redirect handling and progress-callback paths
in downloadFile so synchronous exceptions, including malformed redirect URLs and
throwing onProgress callbacks, are caught and reject the returned Promise.
Ensure recursive get calls and callback invocation remain within the
rejection-aware control flow rather than escaping asynchronous handlers.
- Around line 74-80: Update modelStatus() in
sdk/runanywhere-electron/src/download.ts to set downloaded only when every
artifact required by the catalog entry exists, including VLM mmproj files and
embedder vocabulary files, while preserving sizeBytes reporting. Extend
sdk/runanywhere-electron/test/unit/download.test.js lines 116-124 to populate
partial VLM and embedder directories and assert downloaded is false until all
required artifacts are present.
- Around line 123-138: Validate resumable responses before finalizing or
appending: in the download response handling around the 416/206 logic, accept
416 only when Content-Range has the bytes */N form with N equal to startAt, and
accept 206 only when its range begins at startAt; otherwise restart the partial
download instead of trusting it. Update
sdk/runanywhere-electron/test/unit/download.test.js lines 518-530 to provide the
416 Content-Range header and verify oversized partial files restart rather than
finalize.
- Around line 100-105: Update the finalize function’s rename failure path to
preserve the completed temporary .part file instead of deleting it. Remove the
fs.rmSync cleanup from the catch around fs.renameSync, while still rejecting
with the rename error and retaining explicit promise settlement.
In `@sdk/runanywhere-electron/test/unit/download.test.js`:
- Around line 116-124: Extend the modelStatus test to cover a partially
downloaded multi-file catalog entry, verifying that an entry is not reported as
downloaded when any required secondary file is missing. Use the existing catalog
and download test fixtures/helpers, and retain the current shape assertions for
every catalog id.
- Around line 518-530: Update the downloadFile 416 test fixture to return
Content-Range: bytes */<length>, and add coverage for an oversized .part file.
Ensure the 416 handling finalizes the partial file only when its size exactly
matches the server-reported length; oversized files must not be treated as
complete.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 249d854a-f124-4996-954a-449ceaeadd29
📒 Files selected for processing (6)
examples/electron/RunAnywhereAI/renderer.jssdk/runanywhere-electron/src/RunAnywhere.tssdk/runanywhere-electron/src/download.tssdk/runanywhere-electron/src/process/host.tssdk/runanywhere-electron/src/process/preload.tssdk/runanywhere-electron/test/unit/download.test.js
🚧 Files skipped from review as they are similar to previous changes (3)
- sdk/runanywhere-electron/src/process/host.ts
- examples/electron/RunAnywhereAI/renderer.js
- sdk/runanywhere-electron/src/RunAnywhere.ts
Comment on lines
+118
to
+120
| if (code >= 300 && code < 400 && res.headers.location) { | ||
| res.resume(); | ||
| get(new URL(res.headers.location, u).toString(), redirects + 1); |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
Convert callback exceptions into Promise rejections.
A malformed redirect URL or throwing onProgress callback escapes these asynchronous handlers and can crash the utility host instead of rejecting downloadFile().
Proposed guards
- get(new URL(res.headers.location, u).toString(), redirects + 1);
+ try {
+ get(new URL(res.headers.location, u).toString(), redirects + 1);
+ } catch (e) {
+ reject(e as Error);
+ }- onProgress?.({
- file: path.basename(dest),
- received,
- total,
- percent: total ? Math.round((100 * received) / total) : 0,
- });
+ try {
+ onProgress?.({
+ file: path.basename(dest),
+ received,
+ total,
+ percent: total ? Math.round((100 * received) / total) : 0,
+ });
+ } catch (e) {
+ fail(e as Error);
+ }Also applies to: 155-162
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@sdk/runanywhere-electron/src/download.ts` around lines 118 - 120, Update the
redirect handling and progress-callback paths in downloadFile so synchronous
exceptions, including malformed redirect URLs and throwing onProgress callbacks,
are caught and reject the returned Promise. Ensure recursive get calls and
callback invocation remain within the rejection-aware control flow rather than
escaping asynchronous handlers.
…ange, guard redirect/onProgress, require all files for downloaded
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
@runanywhere/electron— a 6th platform SDK bringing on-device LLM / VLM / STT / TTS / embeddings to Electron & Node, over a native N-API addon on the RunAnywhererac_*C ABI (llama.cpp / ONNX Runtime / sherpa-onnx). Windows-first (x64, arm64-ready); the QHexRT NPU backend seam is kept open. Inference runs in an isolated Electron utility process, streaming to the renderer over aMessagePort.This is an MVP at feature-parity with the Swift/Kotlin SDKs for on-device inference + voice, with a full test suite, packaging, and a tested sample app.
mainis untouched.What's included
sdk/runanywhere-electron/native/) — MSVC-ABI.nodeexposing all 5 modalities, built as a root CMake target over the proven static-lib graph. ~11 commons Windows-portability fixes along the way (char8_t,ERROR_SEVERITYmacro collision,dirent.hshim, NOMINMAX, zlib naming,/MDCRT).src/) —RunAnywhere.initialize/loadLLM/loadVLM/loadEmbedder/loadSTT/loadTTS, streaminggenerate()as anAsyncIterable,Chat(multi-turn),VoiceAgent, model catalog + download.generateObject(prompt, { schema })returns guaranteed-valid JSON via GBNF grammar-constrained decoding (incl.maxItems-bounded arrays).generateToolCall(prompt, tools)forces a well-formed{ name, arguments }call; also first-class on the renderer preload.MicRecorder/SpeakerPlayer; STT/TTS exposed over the preload for a full mic → STT → LLM → TTS → speaker loop.utilityProcesshost +MessageChannelMainbroker + contextIsolated preload exposingwindow.runanywhere.bundle-native.jsassemblesprebuilds/win32-x64/(the.node+ DLLs);package.jsonexports/files/os/cpu/prepack;npm packis self-contained (36 files, ~10.5 MB) and publish-ready. README + GitHub Actions CI (hermetic unit tests).examples/demo-app/) — a tabbed desktop app (Chat, Structured, Tools, Vision, Embeddings, Voice) mirroring the Swift/Kotlin RunAnywhereAI demos, with a headless self-test mode.Cross-SDK commons fixes
Grammar-constrained decoding was never exercised through the shared llama.cpp decode path before, surfacing two latent bugs in
engines/llamacpp/llamacpp_backend.cppthat affect all SDKs:run_decode_loopcalledllama_sampler_accept()afterllama_sampler_sample()(which already accepts), double-advancing the stateful grammar →Unexpected empty grammar stackcrash.Both fixed; plus the previously-swallowed inference-failure
e.what()is now logged.Tests
RunAnywhereMain+ preload (electron mocked).qwen2.5-0.5b(structured / tools / chat, incl. a sampler-reset regression).Not yet (post-MVP)
DPAPI secure store (currently a plaintext stub), auth/telemetry/two-phase init, code-signing + prebuild-download-from-releases, win-arm64 native build, and the QHexRT-on-Windows NPU tier.
Notes
prebuilds/, ~27 MB) are gitignored; runnpm run bundle:nativeafter building the addon.private: true(flip to publish).🤖 Generated with Claude Code
Note
Cursor Bugbot is generating a summary for commit 7b003ea. Configure here.
Summary by CodeRabbit