Local RAG Chatbot: Cited Answers from Your PDF Library

Goal

I built this local RAG chatbot to genuinely understand technical topics by asking questions against a PDF library I trust, and getting answers that come with explicit citations back to the source papers. Right now it is tuned for accretion disks around black holes, especially tilted flows.

---

Why I built this?

This started after a frustrating experience in 2024 when I asked popular web AI tools like ChatGPT and Gemini about research papers on magnetic alignment in Pulsars when the configuration is retrograde. They both produced many citations and convincing summary of those cited papers. However, when I started them reading individually, I realized that most of the studies they cited simply did not exist. Their answers sounded convincing enough to be dangerous, and it made me question about the reliability of these tools. This is expecially alarming for researchers who rely on accurate citations for their work. In academia, arguments should be objective and evidence-based. If you make a claim, you should be able to point to where it came from and verify it. So I built a chatbot that answers strictly from a local, user-controlled dataset of PDFs and attaches citations to the ingested sources.

---

What the project does?

It ingests a folder of PDFs, chunks them, builds embeddings, and creates a FAISS index for fast retrieval. When you ask a question, it pulls the most relevant chunks and generates an answer with citations that point back to the ingested sources. It also includes structured “chapter style” writing modes (intro, methods, current state, future) that help turn a messy topic into something organized and readable. I used it to enhance my understading on accretion disk physics, specifically tilted disks around black holes.

---

Demo

The GIF above shows a typical flow: a question, retrieval of supporting chunks, and a response that stays grounded in what the PDF library actually contains.

---

Final thoughts

For focused questions, it produces reliable, well-cited outputs and can draft decent chapter-style text without inventing references. The best part for me is to create a detailed summary on topics that I read raraely during my doctorate journey, like observations related to tilted disks or summary of an AGN spectra. However, compared to open-source AI platforms, it responses are shorter due to small number of tokens (~8 k), and is considerabely slow because of CPU-only processing. Furthermore, it did not matter how much I tried giving different prompts, its responses are always substandard compared to open-source AI platforms. In the end, I had fun building this chatbot, and now I have something that I can trust. While building it, I realized how much retrieval design and prompt structure control the quality of the result. I started appreciating the technicalties that goes into a llm model, like reducing latency, thought process and making it understand what part of a prompt is relatively important. It is definitely not an easy task to build a chatbot, even a simple one! Looking forward, I want to work on reducing latency by adding GPU layers and improving the refinement workflow so the model follows prompt priorities more consistently.

---

Project Details

Author: Sajal Gupta

Year: 2024-2026

Tech: Python, local LLM, embeddings, FAISS, PDF ingestion, citation formatting