RAG Systems: Implementation and Evaluation

This repository contains two components demonstrating Retrieval-Augmented Generation (RAG) systems:

1. A production RAG implementation from a real application
2. A benchmark evaluation comparing different RAG approaches

Repository Structure

rag/
├── rag_implementation/    # RAG module from Compass application
└── rag_evaluation/        # Benchmark comparing LLM vs RAG approaches

---

1. RAG Implementation (rag_implementation/)

The RAG module extracted from Lectura, a study helper application I developed. This implementation provides:

• Multi-format document parsing: PDF, PowerPoint, Excel, CSV, code files, and more
• Semantic chunking: Two-tier strategy preserving document structure
• Vector storage: ChromaDB with cosine similarity search
• Fast embeddings: Sentence Transformers (all-MiniLM-L6-v2, 384 dimensions)

See rag_implementation/README.md for setup and usage instructions.

---

2. RAG Evaluation (rag_evaluation/)

A comparative evaluation of three question-answering approaches on 127 cryptography questions from a graduate-level USF course:

Results

Approach	Accuracy	Avg Time	Tokens
LLM-Only	82.61%	6.08s	69,921
RAG-Naive	90.58%	4.99s	396,383
RAG-Agent	84.78%	14.30s	1,501,852

Key Findings

• RAG-Naive achieves the best accuracy (+7.97% over baseline)
• Simple retrieval outperforms complex agent approaches for Q&A tasks
• RAG is also faster than LLM-only due to efficient parallel embedding lookup

See rag_evaluation/README.md for replication instructions and detailed results.

---

Technologies Used

Component	Technology
Embeddings	Sentence Transformers (all-MiniLM-L6-v2)
Vector Database	ChromaDB
LLM	OpenAI GPT
Document Parsing	PyMuPDF, python-pptx, pandas