docs: replace compiler-internal macros with self-hosted prelude model

docs/compiler-architecture.md

@@ -72,9 +72,10 @@ Source (.vx)
 
 Macro expansion sits between Parser and Type Checker:
 
-- Compiler-internal macros (`cond`, `and`, `or`) expand to primitive forms (see `language-design.md` §14, Decisions 8–9)
-- User-defined macros (`defmacro`) execute at compile time via a dedicated AST evaluator (see `language-design.md` §4.5):
-  - Pass 1: collect `defmacro` forms, store body AST + parameter names in a macro registry
+- Core macros (`cond`, `and`, `or`) are self-hosted — defined with `defmacro` in a prelude module embedded in the compiler binary (see `language-design.md` §14, Decisions 8–9)
+- All macros (prelude and user-defined) execute at compile time via a dedicated AST evaluator (see `language-design.md` §4.5):
+  - The prelude source is expanded first, populating the macro registry with `cond`, `and`, `or`
+  - Pass 1: collect `defmacro` forms from user code, store body AST + parameter names in the macro registry
   - Pass 2: walk the AST — on macro call, convert arguments to `Syntax` values, evaluate the macro body via the AST evaluator, apply hygiene, convert the result back to AST, re-expand
 - Macro helper functions (`list`, `cons`, `first`, `rest`, `symbol?`, `list?`, `concat`) exist only in the AST evaluator — they are not global builtins
 - Macros operate on `ast::TopForm` (untyped S-expression-derived trees) and produce `ast::TopForm`
@@ -98,7 +99,7 @@ src/
   lexer.rs          Lexer struct, TokenKind enum, Token struct, lex() function
   ast.rs            Untyped AST: Expr, TopForm, Pattern, TypeExpr, Param, Field, etc.
   parser.rs         Parser struct, parse() function (tokens → AST)
-  macro_expand.rs   expand() function (AST → AST), compiler-internal macros (cond, and, or), user-defined macro execution via AST evaluator
+  macro_expand.rs   expand() function (AST → AST), prelude loading, user-defined macro execution via AST evaluator
 
   hir.rs            Typed AST: mirrors ast.rs but every node has a resolved type
   types.rs          VexType enum (semantic types: Int, Float, Function, etc.), TypeEnv
@@ -233,12 +234,10 @@ fn parse(tokens: &[Token]) -> (Vec<ast::TopForm>, Vec<Diagnostic>)
 fn expand(program: Vec<ast::TopForm>) -> (Vec<ast::TopForm>, Vec<Diagnostic>)
 ```
 
-- Handles two kinds of macros:
-  - **Compiler-internal** — `cond` → nested `if`, `and`/`or` → `if` expressions
-  - **User-defined** — `defmacro` bodies execute via a dedicated AST evaluator at compile time
-- For user-defined macros:
-  - Pass 1: collect `defmacro` forms, store body AST and parameter names in a macro registry
-  - Pass 2: on macro call, convert arguments to `Syntax` values, evaluate body via AST evaluator, apply hygiene (rename macro-introduced bindings), convert result back to AST, re-expand
+- Loads the prelude (embedded `prelude.vx`) first, expanding its `defmacro` definitions to populate the macro registry with `cond`, `and`, `or`
+- Then processes user code:
+  - Pass 1: collect `defmacro` forms, store body AST and parameter names in the macro registry (which already contains prelude macros)
+  - Pass 2: on macro call (prelude or user-defined), convert arguments to `Syntax` values, evaluate body via AST evaluator, apply hygiene (rename macro-introduced bindings), convert result back to AST, re-expand
 - Macro helper functions (`list`, `cons`, `first`, `rest`, `symbol?`, `list?`, `concat`) exist only in the AST evaluator — not as global builtins
 - Produces an AST with only primitive forms — no `defmacro` forms or macro calls survive this phase
 - Diagnostics: malformed macro invocations, evaluation errors in macro bodies, expansion depth limit exceeded
@@ -631,7 +630,7 @@ Build bottom-up, one phase at a time, each immediately testable.
 | 2 | `lexer.rs` | Lexer | `lex()` tokenizes `(defn main [] (println "Hello, World!"))` into the correct token sequence. Tests assert token kinds, values, and spans. |
 | 3 | `ast.rs` | AST | All untyped AST node types (`Expr`, `TopForm`, `Param`, `TypeExpr`, etc.) are defined and can represent the hello world program. |
 | 4 | `parser.rs` | Parser | `parse()` converts hello world tokens into the expected AST. Tests assert the resulting tree structure by pattern-matching on nodes. |
-| 5 | `macro_expand.rs` | Macro expansion | `expand()` rewrites compiler-internal macros (`cond` → nested `if`, `and`/`or` → `if`) and executes user-defined macros (`defmacro`) via a dedicated AST evaluator with automatic hygiene. |
+| 5 | `macro_expand.rs` | Macro expansion | `expand()` loads the prelude (self-hosted `cond`, `and`, `or` defined with `defmacro`), then expands all macro calls via a dedicated AST evaluator with automatic hygiene. |
 | 6 | `types.rs`, `hir.rs`, `builtins.rs` | Type system | `VexType` enum covers all primitive and compound types. `hir::Module` mirrors the AST with resolved types. `BuiltinRegistry` contains `println` with its type signature. Unit tests pass. |
 | 7 | `typechecker.rs` | Type checker | `check()` transforms the expanded AST into a valid `hir::Module` where every node carries a resolved type. Tests assert HIR types and diagnostic output for invalid programs. |
 | 8 | `codegen.rs` | Codegen | `generate()` produces valid Go source from the hello world HIR. Tests assert the output contains `package main`, `func main()`, and the `fmt.Println` call. |
@@ -651,7 +650,7 @@ Planned PR sequence:
 | 2 | `lexer` | `lexer.rs` — tokenizer for hello world |
 | 3 | `ast` | `ast.rs` — untyped AST types |
 | 4 | `parser` | `parser.rs` — recursive descent parser |
-| 5 | `macro-expand` | `macro_expand.rs` — compiler-internal macros (cond, and, or) + user-defined macros (defmacro) with AST evaluator and hygiene |
+| 5 | `macro-expand` | `macro_expand.rs` — prelude loading (self-hosted cond, and, or) + user-defined macros (defmacro) with AST evaluator and hygiene |
 | 6 | `types-hir-builtins` | `types.rs` + `hir.rs` + `builtins.rs` — type representations and built-in registry |
 | 7 | `typechecker` | `typechecker.rs` — AST → HIR |
 | 8 | `codegen` | `codegen.rs` — HIR → Go source |
@@ -704,7 +703,7 @@ The following serve MCP server authors specifically and will be part of the futu
 |---------|-----------|
 | **String interning** | Use `String` everywhere. Optimize later if profiling shows it matters. |
 | **Arena allocation** | Use `Box` and `Vec`. Swap for arenas later if needed. |
-| **User-defined macros (`defmacro`)** | Implemented. The macro expansion phase handles both compiler-internal macros (`cond`, `and`, `or`) and user-defined macros via a dedicated AST evaluator with automatic hygiene. See `language-design.md` §4.5 and §14.11. |
+| **User-defined macros (`defmacro`)** | Implemented. All macros — including core control flow (`cond`, `and`, `or`) defined in the self-hosted prelude — execute via a dedicated AST evaluator with automatic hygiene. See `language-design.md` §4.5 and §14.11. |
 | **Multi-file compilation** | `SourceMap` supports `FileId` from day one, but the pipeline processes one file at a time. |
 | **Error recovery in parser** | Stop at first error initially. Accumulate multiple errors later. |
 | **LSP / incremental compilation** | Not a concern at this stage. |

docs/language-design.md

@@ -189,8 +189,6 @@ Source (.vx)
 │         macro-introduced bindings)   │
 │      d. Convert Syntax → AST        │
 │      e. Re-expand the result         │
-│    → apply compiler-internal macros  │
-│      (cond, and, or)                 │
 │                                       │
 └───────────────────┬───────────────────┘
                     │ expanded AST (no defmacro, no macro calls)
@@ -199,6 +197,18 @@ Source (.vx)
             │ Type Checker │
 ```
 
+#### Prelude — Self-Hosted Core Macros
+
+The compiler ships a prelude module (`prelude.vx`) containing `defmacro` definitions for `cond`, `and`, and `or`. During compilation, the prelude source is embedded in the compiler binary and expanded before any user code. These macros expand to primitive `if` and `let` forms:
+
+- `cond` → nested `if` expressions
+- `and` → `(if a b false)`
+- `or` → `(let [tmp a] (if tmp tmp b))` (with hygienic `tmp`)
+
+This keeps the core language small — `if` and `let` are the only control flow primitives the compiler needs to understand — while making the standard control flow forms available in every Vex program without explicit imports.
+
+#### AST Evaluator
+
 The AST evaluator supports a deliberate subset of the language: literals, symbols, `quote`, `if`, `let`, and calls to macro helper functions. This mirrors the approach taken by Zig's comptime (a separate tree-walking evaluator for compile-time code) and matches how every typed Lisp (Typed Racket, Carp) handles macros — macro bodies run in a dynamic context, and the type checker validates the expanded output.
 
 #### The `Syntax` Type
@@ -256,7 +266,7 @@ Functions available only inside macro bodies for constructing and inspecting `Sy
 
 Macros are hygienic by default. The expander automatically renames all bindings introduced by the macro to unique names, preventing variable capture at the call site. The macro author does not need to manage name uniqueness.
 
-For example, the compiler-internal `or` macro introduces a temporary binding. With automatic hygiene, this binding receives a unique compiler-generated name that cannot conflict with user code.
+For example, the `or` macro in the prelude introduces a temporary binding. With automatic hygiene, this binding receives a unique compiler-generated name that cannot conflict with user code.
 
 #### Constraints
 
@@ -1011,19 +1021,19 @@ Distributed as a native Rust binary per platform:
 - Non-tail recursive functions get a compiler warning about stack usage proportional to input size
 - Mutual tail recursion is **not** optimized — trampolining adds allocation overhead for a rare case in MCP code
 
-### 8. Short-circuit operators — `and` and `or` are macros
+### 8. Short-circuit operators — `and` and `or` are self-hosted macros
 
-- `and` and `or` expand to `if` expressions during macro expansion:
+- `and` and `or` are defined with `defmacro` in the prelude and expand to `if` expressions:
   - `(and a b)` → `(if a b false)`
   - `(or a b)` → `(let [tmp a] (if tmp tmp b))`
 - This preserves short-circuit semantics in both the interpreter and compiled paths
 - Treating them as regular functions would evaluate both arguments eagerly — the interpreter would produce different behavior than the compiled output
 - The compiled path would accidentally short-circuit (Go's `&&`/`||` are lazy), but the interpreter would not — a silent correctness bug
-- Macros make the semantics explicit and consistent across execution modes
+- Self-hosted macros make the semantics explicit and consistent across execution modes
 
-### 9. `cond` is a macro over `if`
+### 9. `cond` is a self-hosted macro over `if`
 
-- `cond` expands to nested `if` expressions during macro expansion:
+- `cond` is defined with `defmacro` in the prelude and expands to nested `if` expressions:
   - `(cond test1 val1 test2 val2 :else default)` → `(if test1 val1 (if test2 val2 default))`
 - The parser, AST, and type checker do not need to know `cond` exists — it is gone before type checking
 - `defn` remains a special form (not a macro) because it is the most common form in Vex programs, and first-class parser support produces better error messages
@@ -1037,16 +1047,18 @@ Distributed as a native Rust binary per platform:
 - Testing is covered by Result types and dependency injection
 - If purity annotations are ever wanted, an opt-in `:pure` marker can be added later without breaking existing code
 
-### 11. User-defined macros — AST-evaluated, hygienic
+### 11. Macro system — self-hosted, AST-evaluated, hygienic
 
 - `defmacro` bodies are Vex code that transforms `Syntax` values at compile time
 - A dedicated AST evaluator in `macro_expand.rs` executes macro bodies — no type checking, no HIR, no interpreter involvement
 - The AST evaluator supports a deliberate subset: literals, symbols, `quote`, `if`, `let`, and calls to macro helper functions (`list`, `cons`, `first`, `rest`, `symbol?`, `list?`, `concat`)
 - Macro helpers exist only in the compile-time evaluator — they are not global builtins and cannot conflict with user code
+- Core control flow macros (`cond`, `and`, `or`) are self-hosted — defined with `defmacro` in a prelude module that the compiler loads automatically before user code
 - Alternatives rejected:
   - **Type-check-to-HIR, evaluate via interpreter** — Quote/Unquote/Splice are AST-level concepts with no natural HIR representation; routing them through the type checker and interpreter adds cross-phase coupling for marginal benefit. Every typed Lisp (Typed Racket, Carp) runs macros in a dynamic context and type-checks the expanded output, not the macro body. The useful type errors are in the expanded code.
   - **Compile-to-Go-and-exec** — compiling each macro to Go and running it as a subprocess adds seconds of latency and IPC complexity for every macro invocation
   - **Two-phase compilation** — building a "macro plugin" binary first, then using it for expansion, adds toolchain complexity without proportional benefit
+  - **Hardcoded Rust implementations** — implementing `cond`, `and`, `or` directly in Rust defeats the purpose of building a `defmacro` system; self-hosting validates the macro infrastructure and keeps the compiler core minimal
 - Hygiene is automatic — the expander renames macro-introduced bindings to unique names without macro author intervention
   - Manual `gensym` was rejected: it shifts hygiene responsibility to the macro author, making accidental variable capture a common bug
   - Automatic hygiene matches the design principle "Macros are hygienic" (§2.6) and follows Scheme's proven approach

docs/mvp.md

@@ -148,7 +148,7 @@ Exit codes: `0` success, `1` compilation error, `2` Go build error, `3` CLI usag
 | Traits | Requires `deftype` first |
 | `Result` / `Option` | Requires `defunion` and pattern matching on ADTs |
 | Pattern matching (`match`) | Only useful with ADTs; `if`/`cond` cover MVP needs |
-| User-defined macros (`defmacro`) | Entire subsystem deferred to post-MVP; compiler-internal macros (`cond`, `and`, `or`) are included |
+| User-defined macros (`defmacro`) | Entire subsystem deferred to post-MVP; core macros (`cond`, `and`, `or`) are self-hosted via `defmacro` in the prelude |
 | Modules / imports | Single-file compilation only |
 | Go interop (`import-go`) | Not needed until stdlib work begins |
 | Concurrency (`spawn`, `channel`) | Requires runtime support beyond basic codegen |