ragkit — a tiny, hackable RAG engine powered by Claude

ragkit is a small Retrieval-Augmented Generation (RAG) library you can read in an afternoon. It ingests your documents, splits them into chunks, embeds them, stores the vectors, retrieves the most relevant passages for a question, and asks Claude to answer using only those passages — with inline citations back to the source chunks.

It is deliberately dependency-light:

• No vector database required. Vectors live in a NumPy array and persist to a single .npz + .json pair on disk.
• No embedding API key required. The default embedder is a deterministic hashing embedder that runs anywhere — so the tests and the CI pipeline work offline. Swap in sentence-transformers with one config flag when you want semantic embeddings.
• One LLM dependency. Generation goes through the official anthropic SDK and defaults to claude-opus-4-8 with adaptive thinking.

┌──────────┐   ┌──────────┐   ┌───────────┐   ┌──────────────┐   ┌─────────┐
│  loaders │──▶│ chunking │──▶│ embeddings│──▶│ vector store │──▶│retriever│──┐
└──────────┘   └──────────┘   └───────────┘   └──────────────┘   └─────────┘  │
                                                                               ▼
                                                            ┌─────────────────────┐
                                                            │  Claude (generation) │
                                                            └─────────────────────┘

Install

git clone https://github.com/aaliboyaci/ragkit.git
cd ragkit
python -m venv .venv && . .venv/bin/activate   # Windows: .venv\Scripts\activate
pip install -e .

Set your Anthropic API key (only needed for the answer-generation step):

cp .env.example .env
# edit .env and set ANTHROPIC_API_KEY=sk-ant-...

Quickstart (CLI)

# 1. Build an index from a folder of .md / .txt files
ragkit index data/sample_docs --store .ragstore

# 2. Ask a question
ragkit ask "What problem does RAG solve?" --store .ragstore

ragkit ask retrieves the top-k chunks, prints which sources it used, and streams Claude's grounded answer.

Quickstart (Python)

from ragkit import RagPipeline

rag = RagPipeline.from_directory("data/sample_docs")
answer = rag.ask("How do embeddings enable semantic search?")
print(answer.text)
for cite in answer.citations:
    print(f"  [{cite.index}] {cite.source} (score={cite.score:.3f})")

Retrieval quality & trust

Two things separate a toy RAG demo from a useful one: retrieving the right passages, and being able to check that the answer stayed faithful to them.

Hybrid retrieval (default). Retrieval fuses semantic search with BM25 lexical search using Reciprocal Rank Fusion. Semantic search catches paraphrase ("car" ≈ "automobile"); BM25 catches exact, rare tokens (names, error codes, API symbols). RRF merges the two rankings by position, so the score scales don't need to be comparable.

LLM re-ranking (optional). Turn it on to over-fetch a wide candidate pool and let Claude reorder it by true relevance — cheap retrieval for recall, the LLM for precision at the top.

from ragkit import RagPipeline
from ragkit.config import Settings

rag = RagPipeline.from_directory(
    "data/sample_docs",
    settings=Settings(use_hybrid=True, rerank=True, rerank_candidates=20, top_k=4),
)
answer = rag.ask("How does RAG reduce hallucination?")

Faithfulness evaluation. A grounded answer is only trustworthy if every claim is supported by a cited passage. The built-in LLM-as-judge breaks an answer into atomic claims and checks each against the context it was given:

answer = rag.ask("What problem does RAG solve?")
report = rag.evaluate_faithfulness("What problem does RAG solve?", answer)
print(f"faithfulness: {report.score:.0%}")
for v in report.unsupported_claims:
    print(f"  UNSUPPORTED: {v.claim} — {v.reason}")

Compare retrieval strategies on the sample corpus (offline, no API key):

python examples/evaluate_rag.py     # hit@k and MRR for semantic vs hybrid

Why this exists

I built ragkit to have a clean, end-to-end reference implementation of the RAG pattern — the kind of thing that's usually buried inside a framework. Every stage is a small, separately testable module so you can see exactly what each step does and replace any of them.

See docs/architecture.md for the design walkthrough.

Roadmap

Ideas I'd like to explore next, roughly in order:

• Hybrid retrieval (combine lexical + semantic scores). — BM25 + RRF
• A re-ranking stage between retrieval and generation. — LLM re-ranker
• Answer-faithfulness evaluation (does the answer stay within its citations?).
• Query rewriting / multi-query expansion before retrieval.
• Streaming citations as they're resolved, not just at the end.
• An optional on-disk approximate index (HNSW) for larger corpora.

Contributions and ideas welcome — see CONTRIBUTING.md.

License

MIT — see LICENSE.