Skip to content

Repository files navigation

PragmaSearch — free local-first semantic search engine, an open-source self-hosted Algolia alternative

MIT License CI status Node 20+ Written in TypeScript One runtime dependency PRs welcome Support on Ko-fi

🔎 Live demo  ·  Quick start  ·  Configuration  ·  Performance  ·  Roadmap

PragmaSearch — local-first semantic search engine

PragmaSearch is a free, self-hosted semantic search engine — an open-source, local-first Algolia alternative. It combines vector (meaning-based) search and BM25 keyword search into one hybrid ranking, with typo tolerance, faceted filtering, and multilingual support — no cloud, no API keys, $0 per search.

Search "something for gaming" and get back an RTX 5090 — even though the word "gaming" appears nowhere in its title. The model runs on your own machine or server, so you replace your hosted-search bill with $0.

Paying $500/mo for hosted search on a 5,000-product shop? A small VPS is more than powerful enough to do it locally, for free. Try the live demo →

✨ Features

  • 🧠 Semantic search — understands meaning via vector embeddings (Transformers.js), not just keywords
  • 🔀 Hybrid ranking — fuses vector + BM25 keyword search with Reciprocal Rank Fusion, plus an exact-match boost for SKUs/brands/part numbers
  • 🔤 Typo tolerance — opple → apple, configurable (length-scaled, à la Algolia)
  • 🎛️ Filtering, faceting & pagination — refine by category / brand / price with live facet counts
  • 🌍 Multilingual — English by default; any language via a multilingual model
  • 📦 Zero infrastructure — one local index file, no database, no service to run; self-host on a $5 VPS
  • 🪶 One runtime dependency — just the model runtime. Own your stack.
  • ⚡ Fast — brute-force cosine + in-memory BM25; no vector database needed for small–mid catalogs

🆚 PragmaSearch vs hosted search

Algolia / hosted PragmaSearch
Cost $ per search / per record $0
Hosting their cloud your machine / VPS
API keys / signup required none
Your data & user queries leave your infrastructure stay on your infrastructure
Semantic (meaning) search paid add-on built in
Typo tolerance ✅ ✅
Filtering · faceting · pagination ✅ ✅
Result highlighting ✅ ✅
Synonyms · ranking rules (boost/bury/pin) ✅ ✅ docs
Analytics (top / zero-result queries) ✅ ✅ docs
UI: widget + React + Vue ✅ ✅ widget · React · Vue

Built for small-to-mid catalogs (≤ ~50k items). For million-SKU real-time catalogs with deep analytics and A/B testing, the hosted platforms still win — see the roadmap.

📊 Relevance, measured (not asserted)

Most search projects claim good relevance. This one ships a benchmark you can rerun:

npm run bench

25 labeled natural-language queries over the 102-product sample catalog. The labels are item-level and intent-based, and every one carries a written rationale in bench/queries.json so you can audit — or disagree with — the ground truth. Queries are written the way a shopper actually types — intent first ("something for gaming", "charge my phone on the go"), and often not sharing the product's vocabulary. Some do share the head noun ("wireless earbuds for running" → …Earbuds), because pretending shoppers never type a product's own word would be its own kind of cheating. What separates the modes is the rest of the query, which is why BM25 alone still reaches 0.859 and still loses.

mode nDCG@10 recall@10 MRR ms/query
hybrid (default) 0.941 0.987 0.928 3.4
vector only 0.907 1.000 0.884 3.0
keyword only (BM25) 0.859 0.969 0.860 0.1

Hybrid wins on ranking quality (nDCG/MRR) — pure vector finds the right items but orders them worse, pure BM25 misses meaning entirely. The run also prints its weakest queries, because a benchmark that only shows wins isn't worth much.

The two semantic rows cost about the same because nearly all of it is embedding the query; the cosine scan over 102×384 is the cheap part. BM25 needs no model at all, hence 0.1 ms.

precision@10 sits near 0.28 for all modes by construction: most queries have only 1–4 relevant items, so 10 slots can never be mostly relevant. Point the benchmark at your own catalog and labels with BENCH_CATALOG / BENCH_QUERIES, or call evaluate() directly.

🚀 Quick start

npm install pragmasearch

# build the index from your catalog (a one-time pass over your own data)
npx pragmasearch index products.json

# …or straight from a spreadsheet / shop-admin CSV export, mapping its column names
npx pragmasearch index export.csv --map "id=SKU,title=Product name"

# search it — try a query whose words aren't in any title
npx pragmasearch search "something for gaming"

Programmatic API:

import { buildIndex, saveIndex, loadIndex, createSearcher } from "pragmasearch";

const index = await buildIndex(products);          // embed your catalog once
await saveIndex("pragmasearch-index.json", index); // one local file

const searcher = await createSearcher(await loadIndex("pragmasearch-index.json"));
const { hits, total, weak } = await searcher.search("something for gaming", 5);
// `weak` is the engine's own verdict: true when nothing actually matched. Semantic search
// always returns the nearest items, so `total` alone never tells you that.

🔍 How it works

  1. Index once (build time): each product is run through a small embedding model that runs in-process, turning it into a vector — a numeric fingerprint of meaning. Saved to a single local file alongside a BM25 keyword index. (The model weights download once from the HF Hub and are cached on disk; point PRAGMA_MODEL_CACHE at a persistent directory to keep that cache, or bake it into your image — see the Dockerfile.)
  2. Search anytime: the query is embedded locally, scored against the vectors (cosine) and the keyword index (BM25), fused with RRF, and the top matches are returned in ~tens of ms.

No registration. No account. No third-party service ever sees your catalog or your users' queries.

🖥️ Demo server

A small server serves a search UI + /api/search, with instant in-browser autocomplete, a faceted refinement sidebar (category / brand / price), pagination, and a Hybrid / Vector / Keyword toggle. Live demo →

npm install
npm run demo            # http://localhost:5173

Point it at any catalog/model by building an index and passing it: npx tsx demo/server.ts your-index.json. The query is embedded on your host — deploy on any persistent host (Railway / Render / Fly / VPS), see DEPLOY.md.

🌐 Models & languages

Default model is Xenova/all-MiniLM-L6-v2 (English, ~23 MB). For other languages, index with --model Xenova/multilingual-e5-small — query/passage prefixes are applied automatically. The model is recorded in the index, so query and document encoders always match. See the model table.

📚 Documentation

🗺️ Roadmap

Done: semantic + keyword hybrid search, typo tolerance, filtering, faceting, pagination, result highlighting, a drop-in widget + native React adapter (components + a headless hook), configurable searchable attributes & field weights, synonyms (multi-way + one-way query expansion), ranking rules (boost / bury / pin), incremental indexing (patch price/stock without re-embedding; upsert embeds only the delta), search analytics (top + zero-result queries, latency), custom ranking, a compact int8 index, signed multi-tenant tokens, language-aware tokenization, a hardened production server (pragmasearch serve), query rules, snippet windows, and a reproducible relevance benchmark — CLI + programmatic API + demo. What's still open is in ROADMAP.md.

💬 Community

Questions, ideas, or a use case we're missing? Open an issue or a discussion. Contributions welcome — see CONTRIBUTING.md.

❤️ Support

PragmaSearch is free and MIT-licensed. If it saves you a search bill, you can support ongoing development:

Support on Ko-fi

Or with crypto:

  • ETH / ERC-20 — 0x6d9344F56E2dfF70751c37Fe2dF6aD12EDd48c11
  • Solana — B6Sm1vzfs44Bzpy8VvncrnRfy7sByHUqrgHsDdKyMP2a

Every bit helps keep this maintained and free. 🙏

📄 License

MIT — free for commercial and personal use.

Releases

Packages

Contributors

Languages