bibleLLM

A tiny, from-scratch character-level GPT (nanoGPT-style) trained on the text of the King James Version of the Bible. Two implementations side by side:

• model.py / train.py / sample.py — single-file PyTorch, no HuggingFace. Trains in a few minutes on a modern GPU.
• rust-cuda/ — the same model in Rust with hand-written CUDA kernels. No autograd framework, no libtorch — the forward and backward passes for every operation are hand-written CUDA, orchestrated from Rust via cudarc. cuBLAS handles the matmuls; everything else (layernorm, attention, softmax, GELU, cross-entropy, AdamW, dropout) is a custom kernel.

Both implementations train the same two model sizes (~10M and ~85M params) and produce KJV-style text. The Rust trainer runs at speed parity with eager PyTorch (~1.02× at 85M) and reaches within 0.02 val loss of the PyTorch model.

Results

Model	Impl	Params	Best val loss	Wall time
10M	Python	10.75M	1.1030	900s
10M	Rust	10.77M	1.1224	690s
85M	Python	85.21M	1.1195	3792s
85M	Rust	85.31M	1.1337	3832s

Same hyperparameters for both: batch 64, lr 1e-3, cosine schedule, 5000 iters (10M) / 10000 iters (85M), bf16 activations, dropout 0.2, bias=False.

Repository structure

bibleLLM/
├── README.md              ← you are here
├── LICENSE                ← MIT
├── requirements.txt       ← Python deps (numpy, requests, tqdm)
├── model.py               ← GPT model (causal self-attention, MLP, generate())
├── train.py               ← Training loop: AdamW, cosine LR, bf16 autocast
├── sample.py              ← Generate KJV-style text from a checkpoint
├── chat.py                ← Bible companion: RAG + local LLM + optional KJV restyle
├── web_app.py             ← Flask web UI for the Bible companion
├── web_static/
│   └── index.html         ← Chat interface (HTML/CSS/JS)
├── data/
│   ├── prepare.py         ← Download KJV, build char vocab, write train/val .bin
│   ├── build_index.py     ← Parse verses, embed, build FAISS + topical index
│   └── gen_dataset.py     ← Generate Bible Q&A dataset for LoRA fine-tuning
└── rust-cuda/             ← Rust + hand-written CUDA counterpart
    ├── README.md          ← Detailed writeup of the Rust/CUDA implementation
    ├── Cargo.toml
    ├── build.rs           ← Compiles kernels.cu → PTX via nvcc
    └── src/
        ├── main.rs        ← Model, training loop, checkpoint I/O (Rust)
        └── kernels.cu     ← All CUDA kernels (fwd + bwd + AdamW + dropout)

Setup

Python (PyTorch)

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

# PyTorch (installed separately for the right CUDA wheel).
# NVIDIA GB10 / Blackwell / aarch64 / CUDA 13:
pip install torch --index-url https://download.pytorch.org/whl/cu130
# ...or CPU-only:
# pip install torch

Rust (CUDA)

Requires the CUDA toolkit (nvcc on PATH) and Rust (stable).

export PATH=/usr/local/cuda/bin:$PATH
cd rust-cuda
cargo build --release

Usage

1. Prepare data (downloads ~4.3 MB of KJV text)

python data/prepare.py

2. Train

Python (~10M, ~5000 iters):

python train.py
# Custom config:
python train.py --n_layer 12 --n_head 12 --n_embd 768 --max_iters 10000 --out_dir out85m

Rust (~10M, ~5000 iters):

cd rust-cuda
cargo run --release -- train \
    --n_layer 6 --n_head 6 --n_embd 384 --block_size 256 \
    --batch_size 64 --max_iters 5000 --eval_interval 250 \
    --lr 1e-3 --dropout 0.2 --out ckpt.bin

3. Generate text

Python:

python sample.py --start "In the beginning" --num_samples 3 --temperature 0.8

Rust:

cargo run --release -- sample --start "In the beginning" --tokens 400 --temperature 0.8

4. Chat with a Bible companion (RAG + local LLM)

The chatbot uses a local LLM (Qwen 2.5 7B) grounded in Bible verses via hybrid semantic + keyword retrieval. It's a reflective companion — not therapy.

# Build the verse retrieval index (one-time, ~15 seconds)
python data/build_index.py

# Download the LLM (one-time, ~4.5 GB, two-part GGUF)
python -c "from huggingface_hub import hf_hub_download; \
  [hf_hub_download('Qwen/Qwen2.5-7B-Instruct-GGUF', f, local_dir='models') \
   for f in ['qwen2.5-7b-instruct-q4_k_m-00001-of-00002.gguf', \
             'qwen2.5-7b-instruct-q4_k_m-00002-of-00002.gguf']]"

# Start chatting (terminal, with streaming)
python chat.py

# With KJV restyle (blends the LLM's response with char-GPT KJV prose)
python chat.py --kjv

# Web UI (browser interface)
pip install flask
python web_app.py

Features:

• Hybrid retrieval: semantic search (FAISS + bge-base embedder) + keyword search (BM25) + curated topical index (23 life topics with hand-picked verses)
• Topic routing: classifies input (anxiety, grief, forgiveness, parenting, health, etc.) to supplement retrieval with curated verses
• Anti-hallucination: the LLM is instructed to only reference provided passages; a response validator checks for unretrieved verse references
• Reflection + actionable steps: the LLM acknowledges the person's feelings, shares relevant Scripture, offers one practical step, and ends with a question
• Verse memory: retrieved passages from previous turns are available to the LLM for follow-up questions
• Crisis detection: keyword-based with context exceptions; surfaces professional resources for self-harm/abuse mentions
• Streaming: tokens print as they're generated (terminal mode)
• KJV restyle: optional char-GPT continuation in King James English
• Fine-tuning dataset: data/gen_dataset.py generates Bible Q&A pairs for LoRA fine-tuning to further improve response quality

Model / defaults

• Architecture: 6 layers, 6 heads, n_embd=384, block_size=256 → ~10.7M params
• Tokenizer: character-level, vocab ≈ 74 (built from the corpus)
• Data: KJV plain text, ~4.3M characters, 90/10 train/val split
• Expected char-level val loss after a full run: ≈ 1.1–1.2

For the 85M model: 12 layers, 12 heads, n_embd=768, block_size=256.

Notes

• Both trainers auto-select CUDA when available (bf16) and fall back to CPU otherwise (Python only; the Rust trainer requires CUDA).
• The Rust trainer includes a gradient check (cargo run --release -- gradcheck) that validates every backward kernel against finite differences.
• The chatbot (chat.py) uses llama-cpp-python for local LLM inference (no Triton required) and sentence-transformers + FAISS for verse retrieval.
• The --kjv restyle flag uses the trained char-level GPT to continue the LLM's response in KJV English, creating a blended modern/Biblical style.
• Everything is public-domain: the KJV text and this code (MIT licensed).