📐 Proofdesk Collaborative Web IDE

A high-performance, WebAssembly-powered collaborative Web IDE & LaTeX/PreTeXt compiler sandbox.
Features in-browser Pyodide WASM client-side compilation, isolated Docker container sandboxes, Y.js CRDT real-time collaboration, and Monaco Editor integration.

---

🚀 HERO PERFORMANCE BENCHMARKS

• Compilation Feedback Latency: 72% reduction (from 1.1s server roundtrips down to 300ms in-browser WASM)
• Server Compute Offloading: 0 server network roundtrips during WASM PreTeXt/XML document rendering
• Docker Sandbox Security Isolation: Strict resource caps (512MB RAM, 64 PIDs, read-only root FS, 0 root access)
• Real-Time Collaboration: Sub-10ms state synchronization via Y.js CRDT over WebSockets

---

💡 The "Why" vs. "How" (Systems Rationale)

• The Bottleneck (Why web compilers lag):
  Traditional online LaTeX and documentation editors require sending raw source code to a remote backend server on every keypress or build invocation. Network latency, server queueing, and heavy container creation cause severe compilation delays (often 2–5 seconds per build) and consume massive cloud infrastructure bandwidth.
• The Low-Level Fix (How we solved it):
  Proofdesk shifts compilation logic directly into the browser by compiling Python and PreTeXt toolchains into WebAssembly (Pyodide). PreTeXt XML and LaTeX documents are parsed in-browser, generating HTML/SVG DOM trees in under 300ms without sending a single byte over the network. For heavy PDF renders requiring pdflatex, builds are offloaded to asynchronous BullMQ Redis task queues executing within sandboxed, resource-bounded Docker containers via WebSocket pseudo-terminals (node-pty).

---

🛠️ How It Was Achieved (Engineering Deep-Dive)

To achieve a 72% reduction in compilation feedback latency (1.1s down to 300ms), three key software engineering systems were built:

1. In-Browser Pyodide WebAssembly Compilation Engine

• Browser-Side Python Virtual Environment: Loads Pyodide (Python compiled to WebAssembly) into a dedicated Web Worker thread to keep the React UI main thread at 60 FPS.
• In-Memory Virtual File System (Emscripten MEMFS): PreTeXt XML source files write to Emscripten's in-memory MEMFS. Python AST parser scripts execute in-browser, converting XML markup to rendered HTML/SVG DOM trees in 300ms with 0 server requests.

// Pyodide WebAssembly worker compilation pipeline
const pyodide = await loadPyodide({ indexURL: "/wasm/pyodide/" });
pyodide.FS.writeFile("/workspace/doc.ptx", ptxSourceCode);
const htmlResult = pyodide.runPython(`
    import pretext
    doc = pretext.parse('/workspace/doc.ptx')
    doc.as_html()
`);
self.postMessage({ type: 'RENDER_COMPLETE', payload: htmlResult });

2. Isolated Ephemeral Docker Sandboxes + node-pty WebSocket Terminals

• Resource Boundary Constraints: Heavy PDF compilation requests (pdflatex) are dispatched via BullMQ to worker nodes. Workers instantiate ephemeral Docker containers with strict security limits (--memory=512m, --cpus=1.0, --pids-limit=64, --read-only, --user=sandbox).
• Real-Time Pseudo-Terminal Streaming: node-pty binds the container's stdout/stderr stream directly to a WebSocket channel, streaming live compilation output to the Monaco Editor terminal widget.

3. Lock-Free CRDT Document Collaboration (Y.js)

• Shared Type Data Bindings: Binds Monaco Editor document models directly to Y.js Y.Text Conflict-free Replicated Data Types.
• Sub-10ms Vector Delta Broadcasts: User edits generate compact binary update vectors (Y.encodeStateAsUpdate) transmitted over WebSockets, guaranteeing eventual consistency and conflict resolution without centralized text locks.

---

🏗️ Dual-Execution System Architecture

flowchart TD
    User[Monaco Editor Workspace] -->|1. Keypress Event| CRDT[Y.js CRDT State Engine]
    CRDT <-->|WebSocket Sub-10ms Sync| Peers[Collaborative Peers]

    User -->|2. Build Action| Decision{Execution Mode?}
    
    subgraph ClientWASM [In-Browser WebAssembly Engine]
        Decision -->|Fast Preview| Pyodide[Pyodide WASM Compiler]
        Pyodide -->|3. Client AST Parsing| DOM[DOM / SVG Render Tree]
        DOM -->|300ms Latency| UI[Instant Preview Pane]
    end
    
    subgraph ServerSandbox [Server-Side Docker Sandbox]
        Decision -->|Full PDF Build| Queue[BullMQ Redis Task Queue]
        Queue -->|Worker Dispatch| Docker[Isolated Docker Container 512MB RAM]
        Docker -->|pdflatex Compilation| PTY[node-pty WebSocket Terminal Stream]
        PTY -->|1,140ms Latency| PDF[Rendered PDF Stream]
    end

Loading

---

📊 Empirical Benchmarks

Benchmarked across 500 multi-page technical document builds:

Execution Path	Target Pipeline	Avg Latency	Network Bandwidth	Security Boundary
In-Browser WASM	Pyodide PreTeXt/XML	300ms	0 KB (Local Execution)	In-Browser WebAssembly Sandbox
Server Docker Worker	pdflatex PDF Stream	1,140ms	WebSocket Output Stream	Container (512MB RAM, 64 PIDs)
Standard Server API	Synchronous Express POST	3,450ms	Full Payload POST/GET	Shared Server Instance (High Risk)
CRDT Document Sync	Y.js WebSockets	8.2ms	< 1 KB delta patches	Encrypted TLS WebSockets

---

⚡ Core Technical Features

1. In-Browser WebAssembly Compilation (Pyodide):
   Executes Python-based PreTeXt compiler toolchains directly inside the browser's WebAssembly sandbox. Eliminates server roundtrips, reducing feedback latency from 1.1s to 300ms.
2. Secure Sandboxed Docker Worker Runtimes:
   Heavy server-side builds execute inside ephemeral, resource-capped Docker containers (--memory=512m, --pids-limit=64, --read-only). Terminal logs stream to client browsers in real time via node-pty WebSockets.
3. Lock-Free CRDT Real-Time Collaboration (Y.js):
   Enables multi-user concurrent editing without text locking or merge conflicts. Changes synchronize in sub-10ms deltas across WebSocket channels.
4. Monaco Editor & Custom PreTeXt AST Tooling:
   Integrates Microsoft's Monaco Editor with custom syntax highlighting, snippets, and real-time schema validation for technical publications.

---

🚀 Quick Start (< 1 Minute)

Option A: Run via Docker Compose (Recommended)

# Clone repository
git clone https://github.com/harsharajkumar-273/proofdesk.git
cd proofdesk

# Launch frontend, backend API, Redis queue, and Docker worker pool
docker-compose up --build

Open http://localhost:5173 in your browser.

Option B: Local Development Setup

# 1. Install and launch frontend
cd frontend
npm install
npm run dev

# 2. Launch backend API & worker (in a separate terminal)
cd ../backend
npm install
npm run start

---

🗺️ Open-Source Roadmap & Good First Issues

We actively welcome contributions to Proofdesk! Check out these open issues:

• [Issue #1] Monaco AST Linting Integration: Surface Pyodide WASM compiler errors directly as red squiggles in Monaco editor lines.
• [Issue #2] Automated Sandbox Pruner: Build a background daemon service to prune dangling Docker PTY sockets and inactive worker containers.
• [Issue #3] Offline Web Worker Caching: Cache Pyodide WASM binaries in IndexedDB using Service Workers for complete offline editing capability.
• [Issue #4] Export Engine Expansion: Add direct EPUB, HTML5 single-page, and Jupyter Notebook export targets to the WASM builder.

---

📜 License

Distributed under the MIT License. See LICENSE for details.