Case study · Project 01
Lumen · Local RAG Chatbot
Portfolio-grade, local-only retrieval-augmented chat: every step of embedding, search, and generation is explicit—no LangChain black box— so the demo is interview-explainable and runnable on your machine.
Problem
Many RAG demos hide retrieval inside a framework chain. That makes it hard to show why an answer is grounded, to compare with vs without retrieval, or to debug bad chunks. Interviewers and operators need a path they can walk: index → retrieve → generate, with sources and metrics visible—not a single opaque “chat with your docs” spinner.
Approach
- Explicit pipeline — FastAPI + ChromaDB + bge-small embeddings + Qwen via Ollama; plain HTML/CSS/JS frontend. Each stage is named and inspectable.
- Grounded answers — retrieve from a local Chroma index (or live web results), then generate with numbered sources so claims stay checkable.
- Streaming + metrics — chat UI with tok/s on each completion and one-click Retry so local model performance is honest.
- Comparison mode — same query With RAG vs Without RAG so the value of retrieval is demoable, not assumed.
- Local-first — designed to run on the machine for privacy, offline demos, and cost control; cloud is optional, not required.
Why this design
The product is not “a chatbot”—it is a retrieval system you can explain. Side-by-side comparison and cited sources turn RAG from a buzzword into a measurable behavior. That matches ML literacy (embeddings, grounding, failure modes) and systems thinking (clear data flow over magic chains).
Stack
Python · FastAPI · ChromaDB · bge-small · Ollama / Qwen · HTML / CSS / JS
Role on the build
Architecture and product framing (explicit retrieval, comparison mode, local deployability). Implementation steered with AI coding agents. Clean light chat UI so sources and metrics stay readable in demos.