ML flavor front end, formatter, effects, modules, and docs/website overhaul#165
Merged
Conversation
Baseline commit of the in-flight tracing-GC work that was already present in the working tree (memory_gc.c, osp_gc_shim.h, --memory=gc flag, GC archive build). Snapshotted as-is so the WebAssembly feature work can layer cleanly on top; this commit is not part of the wasm change.
…targets
The emitted IR baked in two host C type widths that differ on wasm32 (ILP32):
- string length called libc strlen, declared i64, but size_t is 32-bit on
wasm32 -> a wasm-ld signature mismatch that traps at runtime. Route both the
`length` builtin and string concatenation through a new runtime shim
osp_strlen(const char*) -> int64_t, so the size_t->int64 cast lives in C
(correct per target) and the IR stays i64 on every target.
- int-to-string used sprintf("%ld", i64). On wasm32 `long` is 32-bit, so big
ints printed wrong. Use "%lld" (long long is 64-bit everywhere); identical
to %ld on LP64 native.
Native output is byte-identical: 48/48 differential golden tests pass.
…ssembly Add `--target=wasm32` (and `-o <out>`) to the osprey CLI: lower the existing target-agnostic IR to wasm32-wasip1 and link a command module with wasm-ld. - crates/osprey-cli/src/wasm.rs: the wasm backend driver. Appends a __main_void entry thunk (sidesteps the clang/wasi-libc main-mangling skew), lowers IR to a wasm object with `clang -c` (so clang never demands the absent libclang_rt.builtins-wasm32.a), then drives wasm-ld directly: crt1-command.o + object + the portable runtime archive + libc. Sysroot/tool discovery via env overrides + conventional locations. Pure helpers are unit-tested. - Makefile: `make wasm` builds libosprey_runtime_wasm.a from the portable C subset (no pthreads/sockets/OpenSSL/syscalls) and compiles the example; `make wasm-test` validates + smoke-runs it. - examples/wasm/: hello.osp, a self-contained browser loader with an inline WASI shim (fd_write -> console), README, expected output. - scripts/wasm-smoke.mjs: validate + run under Node's built-in WASI, assert stdout. [WASM-TARGET]
…2-browser-target"
…RGET] Add deterministic, toolchain-free unit tests for the wasm32 CLI driver (run_tool/compile_object/link/run_host/build/run, stubbing clang/wasm-ld/ wasmtime via OSPREY_WASM_* + a temp sysroot + a discoverable stub archive) and the --memory / --debug / --target=wasm32 dispatch paths in main.rs. Restores osprey-cli to the 95% coverage floor (was 89.9%) without relying on a wasm toolchain being present in the coverage job.
Add the generated function-reference pages for the CSPRNG builtins and link them from the function index.
…00% [DEBUGGER] Add the Osprey debug panel (stack frames, locals, program details) wired into the activity bar, a comprehensive real-lldb-dap VSIX e2e suite (step over/in/out, continue, conditional/multi breakpoints, watch/evaluate, scopes/variables) on a shared DAP test harness, and drive osprey-debug to 100% line coverage. Full VSIX suite 76 passing; extension coverage 90.84%.
- cargo fmt the new osprey-cli wasm unit tests (rust job Format check diff). - wasm job: the wasi-sdk-24 sysroot tarball unpacks to a versioned top dir (wasi-sysroot-24.0/), so 'test -d .../wasi-sysroot' failed silently. Extract with --strip-components=1 into a stable path and assert lib/wasm32-wasip1. - setup-osprey-compiler: install lldb and expose lldb-dap at /usr/bin so the VS Code debugger E2E resolve it and run in CI instead of failing 'not found'.
…UGGER-LLDB-DAP-VERSION] The VS Code debugger E2E suite failed on ubuntu-latest: its distro lldb-dap (LLVM 18) DAP server mis-resolves a stopped frame's source path to "." and ignores breakpoint conditions on x86_64. liblldb 18 reads the osprey-emitted DWARF correctly (confirmed via lldb CLI and a direct DAP client), and lldb-dap 22 on macOS passes the whole suite — the defect is the old DAP server, fixed from lldb-dap 19. Install lldb-dap 20 from apt.llvm.org in the compiler setup action and pin /usr/bin/lldb-dap to it; clang/llvm 18 still builds the C runtime. Expand the VSIX breakpoint E2E so the same test asserts F10 / Step Over executes a call (bump) without descending into it — a regression guard for "step over behaved like step in". [DEBUGGER-STEP-OVER] Makefile: collapse vsix-rebuild-reinstall to a single `code --install-extension --force` (drop the separate uninstall) so a running VS Code reconciles its extension host once, and id-scope the vsix glob to osprey-*.vsix.
…an [DEBUGGER-DBG-DECLARE] An inline `@llvm.dbg.value` for a `let` binding lowers to a DBG_VALUE whose line-table row is line 0; between two statements that leaves a stray line-0 entry. lldb-dap reads the osprey DWARF fine on most toolchains, but on x86_64 that stray row derails breakpoint line resolution — a frame at the print line or a conditional hit reports `line 0`, so `assertCurrentLine` sees "stop on line 0". (Verified against lldb-dap 20 on x86_64/CI and arm64, and lldb-dap 22 on macOS.) Emit locals with `@llvm.dbg.declare` over a dedicated, debug-only stack slot instead: the variable lives at a fixed frame offset (readable at every PC in scope) and no inline DBG_VALUE means no stray line-0 row. Parameters keep `@llvm.dbg.value` — they are SSA arguments live from entry, so they read correctly even at a conditional breakpoint on the function's first line (a slot store would not have run yet there), and at function entry they never introduce an inter-statement line-0 row. Verified end to end: all debugger E2E tests pass on lldb-dap 22 (macOS) and the previously-failing conditional/scopes/line-6/step scenarios resolve to the correct lines with readable locals on lldb-dap 20.
…DEBUGGER-STEP-OVER] lldb-dap can briefly report a freshly-stopped frame's source path as "." (and its line as 0) on the first `stackTrace` of a session, before the module's line table finishes resolving, then correct both. The osprey DWARF is clean — the frame resolves to the real file on a subsequent request (verified: arm64 lldb-dap 20 and macOS lldb-dap 22 always resolve; only x86_64 cold starts race) — so this is a poll race, not a debug-info defect. `waitForStop` now keeps polling for a top frame whose source path is resolved (not ".") on a real 1-based line, holding the first unresolved stop only as a fallback returned once a short grace elapses. A frame that genuinely has no source (the C entry stub a stopOnEntry launch pauses in) still returns via that fallback, so nothing hangs. Assertions are unchanged — this only stops sampling the cold-start transient.
…LDB-DAP-VERSION] lldb-dap 18 (ubuntu-latest distro) mis-resolves source paths and ignores breakpoint conditions on x86_64; lldb-dap 20 fixed those but flakes on x86_64 step/line resolution (a different debugger E2E test failed each CI run, with "custom request failed" timeouts). lldb-dap 22 (LLVM 22.1.8) is the exact toolchain developers run on macOS, where the suite passes deterministically; verified stable on Linux too (all scenarios + step-over correct, repeated). apt.llvm.org ships lldb-22 for noble, so install that and pin /usr/bin/lldb-dap to it. The dbg.declare line-table fix and the harness's source-resolution wait remain as version-independent robustness.
- vsix-rebuild-reinstall: drop the redundant `code --uninstall-extension` step and rely on `--install-extension --force` (idempotent in-place upgrade) so the running VSCode reconciles its extension host once, not twice; remove the now-dead _vsix_uninstall target. - Pin both .vsix globs to osprey-*.vsix (clean + install) so no non-osprey extension is ever referenced. - Add `make wasm-serve`: builds via `make wasm`, static-hosts examples/wasm at localhost:8080 and opens the browser. `make wasm` stays headless for CI. - CLAUDE.md: add act-autonomously directive (user edit).
Replace the 3-line width-fix snippet with a one-screen tour of the wasm-safe heart of the language, every value really computed: - algebraic effects: one pure `account()` run in two worlds (real stateful handler vs frozen mock ledger) — meaning chosen by the handler - union type + exhaustive pattern matching (Tier/badge) - Hindley-Milner inference (no annotations) + functional pipeline (range |> filter |> map |> fold) - persistent List<T> + Map<K,V> - the original osp_strlen (i64) / %lld width fixes preserved as the finale Verified green via `make wasm`: compiles to wasm32, byte-matches the native oracle under Node's WASI and the browser shim, golden suite PASS=30 FAIL=0. Numeric recursion is intentionally omitted — checked arithmetic makes `n - 1` / `n * …` yield Result<int, MathError>, which won't unify with the int base case; `fold` is the idiomatic iteration anyway. Update expectedoutput, the tracked build/ artifacts, and the README.
Rescued from the pre-merge auto-stash: a complete benchmark run across all 22 cases and the osprey-gc backend that was uncommitted in the working tree (the committed snapshot held only the listops case).
resolveOspreyOnPath had no caller in any suite (33/50 lines). Exercise it directly plus resolveBuiltOsprey's PATH fallback, lifting the counted osprey-test-env.ts toward the 95% vsix coverage gate.
…E2E coverage The 'configured bare command resolves against a Windows PATH too' assertion was impossible on a POSIX CI runner (path.delimiter ':' collides with Windows drive-letter colons), and the injected-platform lldb unit branches never registered in the instrumented host run anyway. Replace them with real extension-host E2E: dead lldbDapPath adapter refusal, unsaved-buffer save-before-compile, .osp mis-association self-heal, and a live ML (.ospml) LSP sweep — covering the previously-uncovered activation/debug branches. Authored by the vsix-coverage agent; verified tsc-clean.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TLDR
Adds a second ML (offside/curry-by-default) surface to Osprey — parser, lexer, lowering, formatter, LSP, and VSCode support — proven byte-for-byte equivalent to the Default flavor, alongside a full spec/docs/website overhaul and a browser WebAssembly SQL demo.
Details
ML flavor frontend (
crates/osprey-syntax/src/ml/) — a complete indentation-delimited, curry-by-default front end that lowers to the same AST as the Default flavor:lexer.rs/token.rs— offside-rule layout lexer (INDENT/DEDENT, newline-terminated strings).parser.rs/cst.rs— ML grammar: whitespace application (f x),\x => elambdas,:=mutation,=binding, layout blocks — nofn/let/braces.lower.rs(1730 LOC) — lowers the ML CST to the shared AST.crates/osprey-syntax/src/default/— the existing brace-flavor parser moved underdefault/so the two surfaces sit side by side;strings.rsfactored out as shared interpolation lexing..osp→ Default,.ospml→ ML, selected by extension; a flavor marker conflict is a hard error.Formatter (
crates/osprey-fmt/+osprey fmt) — new crate with brace (brace.rs) and layout (layout.rs) formatters plus a shared scanner; wired into the CLI asosprey fmt(stdin/dir-recursive/check/rewrite modes) and intomake fmt.Algebraic effects / first-class handlers —
compiler/runtime/effects_runtime.c(+250) andosprey-codegen/src/effects.rs(+515) extend effect resumption and make handlers first-class values.Modules & namespaces — spec
0025plus supporting type-checker and lowering changes.LSP / VSCode —
osprey-lspgains flavor-aware diagnostics/features; newosprey-ml.tmLanguage.json, ML snippets, andlanguage-configuration-ml.json.Website & docs — every
website/src/spec/*chapter revised; new chapters0023 Language Flavors,0024 ML Flavor Syntax,0025 Modules and Namespaces; mirrored underdocs/specs/./docs/index gains an inline Flavors explainer (one language, two ways to write it). New/wasm/page: an Osprey program compiled towasm32-wasip1seeds a real in-browser SQLite (sql.js) database with a tabbed query/add-data/source UI, and both the Osprey and SQL are Prism-highlighted (js/wasm-studio.js). Playground gains a Default⇄ML flavor toggle.Examples — most
examples/tested/*programs gain a byte-identical.ospmltwin; several.ospfiles were de-annotated to lean on inference.How Do The Automated Tests Prove It Works?
make ci(lint + test + build) passes end-to-end, exit 0. Specific coverage:cross_flavor_ir_equiv.rs—every_ml_example_has_a_default_twinandml_and_default_twins_emit_identical_ircompile each.ospml/.osppair and assert the emitted LLVM IR is identical, proving the ML surface lowers to the same program.cross_flavor_equiv.rs—ml_multiparam_equals_default_explicit_curry/ml_uncurried_tuple_equals_default_multiparampin ML currying semantics against the Default flavor.corpus.rs—default_examples_format_idempotentlyandml_examples_format_idempotentlyproveosprey fmtis a fixpoint over the whole example corpus (format twice == format once).cli_e2e.rs—fmt_*,flavor_marker_conflict_exits_two,check_*, andllvm_emits_ir_and_rejects_ill_typedexercise thefmt/check/llvmCLI surface end-to-end including error exit codes.no_needless_main.rs—no_example_wraps_a_trivial_program_in_mainenforces the style rule across examples.ml_coverage.rs(823 LOC) — parser/lowering unit coverage for the ML front end.diff_examples.sh/diff_wasm_examples.sh) — every.osp/.ospmlinexamples/tested/runs and must match its.expectedoutputbyte-for-byte.website/tests/interactions.spec.js) —wasm demo seeds a browser SQLite db and runs queriesboots the wasm module, asserts#bannerreaches "Database ready", the query tab auto-renders a result table with highlighted SQL tokens, the source tab shows highlighted Osprey, and adding a row then re-querying reflects the new data;playground flavor toggle swaps the sample between .osp and .ospmlverifies the Default⇄ML toggle.