VisTex-RAG

Production-ready RAG pipeline for technical PDFs — VLM/OCR preprocessing, hybrid dense–sparse retrieval, cross-encoder reranking, and an OpenWebUI chat interface.

Domain-agnostic Retrieval-Augmented Generation framework for engineering specs, regulatory standards, scientific papers, and formula-heavy documents. Ingest scanned PDFs with tables and equations, retrieve with hybrid search, and generate grounded answers via LLM.

Keywords: rag · retrieval-augmented-generation · qdrant · open-webui · fastapi · docker · ocr · vision-language-model · hybrid-search · technical-documentation · llm

---

Features

• VLM/OCR preprocessing — extract text from scanned, tabular, and formula-rich PDFs
• Hybrid retrieval — dense embeddings + sparse BM42 keyword search in Qdrant
• Technical chunking — optimized for tables, identifiers, and structured content
• Cross-encoder reranking — BGE reranker for higher precision on domain queries
• OpenWebUI integration — chat UI with a custom Python tool for the RAG pipeline
• Docker-ready — full stack via Compose, GPU-accelerated inference

---

Architecture

flowchart LR
    Docs["PDF / Images\n(docs/)"] --> VLM["VLM Preprocessor\nQwen3-VL"]
    VLM --> JSON["OCR JSON"]
    JSON --> Chunk["Chunking +\nEmbedding"]
    Chunk --> Qdrant["Qdrant\nHybrid Index"]
    User["User"] --> WebUI["OpenWebUI"]
    WebUI --> RAG["rag-service\nFastAPI"]
    RAG --> Qdrant
    RAG --> Rerank["BGE Reranker"]
    Rerank --> Ollama["Ollama\nQwen2.5"]
    Ollama --> WebUI

Loading

Service	Technology	Role
Ollama	LLM + VLM	Answer generation and OCR
OpenWebUI	Web frontend	Chat UI and Python tools
rag-service	FastAPI + Python	Embeddings, retrieval, reranking
rag-qdrant	Qdrant	Vector + sparse storage

---

Technology Stack

Stage	Model
VLM / OCR	qwen3-vl:4b-q3_k_s
Dense embedding	sentence-transformers/paraphrase-multilingual-mpnet-base-v2
Sparse retrieval	Qdrant/bm42-all-minilm-l6-v2-attentions
Vector DB	Qdrant
Reranking	BAAI/bge-reranker-v2-m3
Answer LLM	qwen2.5:7b-instruct

---

Requirements

• Docker + Docker Compose v2
• NVIDIA Container Toolkit (GPU)
• Ollama installed in WSL2
• PDF / PNG / JPG files in docs/

---

Quick Start

1. Download Hugging Face models

cd rag/
python3 download_models.py

2. First-time setup (build + ingest)

Full pipeline: Ollama, Docker rebuild, OCR, Qdrant ingestion, smoke test.

chmod +x mount.sh
./mount.sh

3. Daily use (start without rebuild)

chmod +x start.sh
./start.sh          # start
./start.sh stop     # stop containers
./start.sh restart  # restart stack
./start.sh restart-rag  # restart backend only
./start.sh status   # show status

Service URLs

Service	URL
OpenWebUI	http://localhost:8080
RAG API (Swagger)	http://localhost:8000/docs
Qdrant Dashboard	http://localhost:6333/dashboard
Ollama API	http://$WSL_IP:11434

---

OpenWebUI Setup

1. Open Workspace → Tools
2. Create a new tool from openwebui/function.py
3. Start a chat → Integrations → Tools → enable the tool
4. Queries run against documents in docs/

---

WSL2 + NVIDIA Setup

Full environment setup (click to expand)

1. Install drivers and verify GPU

Windows: install the latest NVIDIA drivers with WSL2 support.

WSL2:

nvidia-smi

2. Install Ollama in WSL2

curl -fsSL https://ollama.com/install.sh | sh
ollama --version

3. Disable Ollama on Windows (avoid conflicts)

Get-Service ollama
Stop-Service ollama
Set-Service ollama -StartupType Disabled

4. Fix /usr/share/ollama permissions (WSL bug)

sudo rm -rf /usr/share/ollama
sudo mkdir -p /usr/share/ollama
sudo chown $USER:$USER /usr/share/ollama

5. Disable systemd-managed Ollama

sudo systemctl disable ollama
sudo systemctl stop ollama

6. Start Ollama for Docker access

pkill -f ollama
export OLLAMA_HOST=0.0.0.0:11434
nohup ollama serve > ~/.ollama.log 2>&1 &

7. Verify Ollama API

curl http://$(hostname -I | awk '{print $1}'):11434/api/tags

---

Project Structure

VisTex-RAG/
├── docker/              # Dockerfile + docker-compose.yml
├── docs/                # Source documents (PDF, images)
├── openwebui/           # OpenWebUI custom tool
├── rag/
│   ├── config.py        # Models, hosts, flags
│   ├── vlm_preprocessor.py   # VLM OCR → JSON
│   ├── populate_qdrant.py    # Chunk, embed, ingest
│   ├── rag_server.py         # Hybrid RAG API
│   ├── modern_chunking.py
│   └── text_processing.py
├── mount.sh             # First-time build + ingest
└── start.sh             # Fast start/stop/restart

---

Components

vlm_preprocessor.py

Runs VLM-based OCR on PDFs and images, outputs structured Markdown JSON.

populate_qdrant.py

Chunks OCR output, generates dense + sparse vectors, uploads to Qdrant.

rag_server.py

FastAPI server: hybrid search, cross-encoder reranking, parent-chunk context expansion.