Hybrid code search that understands intent -- not only text or syntax.
v2.6.0 · 15 languages · local-first semantic · critic + two-channel AND · Code Mode (on by default, no API key)
One search tool. Identifiers, natural language, defs/callers, semantic, and patterns — ranked. You do not need a second grep.
For Pi, install the native package directly:
pi install npm:pi-ast-sgrepIt immediately adds asgrep (Code Mode), asgrep_search, asgrep_index, asgrep_status, four /asgrep-* commands, and the ast-sgrep skill. The first search lazily creates .asgrep/; no Rust toolchain, PATH setup, MCP adapter, credential, or runtime download is required. See the complete Pi package guide and Code Mode.
Upgrading to 2.0: this is a breaking semver release. Cloud (--cloud-embed, ASGREP_EMBED_API_KEY) and Ollama (--ollama-embed, ASGREP_OLLAMA_URL) embedding clients are gone, and indexes that still store embed_backend=cloud|ollama fail closed until asgrep reindex. Pi users can update the package normally.
Current embedding engine: a single static (Model2Vec-style) engine compiled into the binary — deterministic 256-d vectors, no model download, no ONNX runtime. The hashed and neural backends are retired; --neural-embed / ASGREP_EMBED_BACKEND now error explicitly, and old indexes re-embed automatically on the next index run. See semantic search.
Standalone CLI binaries are on the v2.6.0 GitHub Release (asgrep, asgrep_darwin_x64, asgrep_linux_arm64, asgrep_linux_x64, asgrep_windows_amd64.exe) once the release is published; until then, build from source. The npm family (ast-sgrep launcher + platform packages) carries the same version. Rust crates.io publication is a separate, explicitly approved step in docs/RELEASING.md and has not happened for prior releases.
To build from source:
git clone https://github.com/AdityaVG13/ast-sgrep
cd ast-sgrep
cargo build --release -p ast-sgrep-cli
./target/release/asgrep --helpStandalone binaries: asgrep and ast-sgrep (aliases).
On Unix, the CLI runs directly unless ASGREP_CPU_LIMIT_PERCENT opts into the
process supervisor. The variable sets the worker process runnable wall-time
fraction in each 10 ms SIGSTOP/CONT cycle; it is not
a machine-wide or one-core CPU percentage, and multi-threaded work may consume several cores
while runnable. On Windows, commands run directly: search, indexing, cancellation, and path
handling are supported, but the duty cycle is not enforced.
For non-Pi clients, use the Agent Plugins package at packages/agent-plugin: plugin.json + skills/ast-sgrep + mcp.json (stdio asgrep-mcp).
Code Mode XOR MCP: Pi → pi install npm:pi-ast-sgrep (Code Mode). MCP hosts → asgrep-mcp / agent-plugin. Do not register both in the same agent.
Paste into your agent:
Clone https://github.com/AdityaVG13/ast-sgrep, cd into it, run `cargo build --release -p ast-sgrep-cli`.
Register target/release/asgrep-mcp as a stdio MCP server named "ast-sgrep" (build with: cargo build --release -p ast-sgrep-mcp).
Verify: run ./target/release/asgrep index . then search for defs: of a symbol in this repo.
2.6.0 is the static-engine release: offline semantic ranking in every default search, with the embedding table compiled into the binary. Full notes: CHANGELOG.
| You can now... | How |
|---|---|
| Search semantically with zero setup | A native static embed engine is compiled in — no model download, no ONNX runtime, no API key, never a network call. The embed channel runs ungated in hybrid fusion, so plain-language queries rank code without any flags. |
| Trust NL→code ranking | On the frozen RepoQA-600 suite: MRR@10 .297 → .719, Hit@1 .190 → .608 (release-paired against published 2.5.4; see the benchmark summary). |
| Upgrade indexes without rebuilding by hand | Indexes record their embedding model identity and re-embed automatically when it changes; retired engines fail closed instead of silently switching. |
| Bound graph chains | max_work caps call-chain exploration with work/truncation diagnostics. |
Honest tradeoffs, measured and disclosed: hybrid/conceptual queries pay the
always-on embed channel (~3–4× fresh-process latency, ~6× warm rounds vs
2.5.4 on the self corpus), index.db grows ~20–30%, and scoped-repo indexing
got faster. Also fixed: the Linux database disk image is malformed failure
after /asgrep-reindex + edit (SQLite last-close no longer deletes shared
WAL sidecars under live sessions), large-integer preservation in native Pi
results, and multilingual structural matching.
For search, one tool is enough. You do not need ripgrep to find a token, ast-grep to find a shape, or Semgrep to find a concept. Index the repo once; unprefixed asgrep ranks the definition, the callers, the structural match, and the semantic near-miss in one list.
Most other code search is either fast text (ripgrep) or pattern matching (ast-grep / Semgrep). Neither answers "where does credential renewal happen?" when those words never appear in the code, and neither ranks Searcher above bench_searcher.
ast-sgrep builds a persistent index: symbols, caller/callee edges, imports, lexical FTS, and symbol-level semantic vectors enriched with call-graph context. Query in natural language or with graph prefixes; get ranked hits with excerpts for humans or agents.
No API key required. Semantic search works out of the box: a static embedding engine is compiled into the binary — no model download, no runtime, never a network call.
| You need... | ast-sgrep gives you... |
|---|---|
| Where is X defined? | defs: + ranked hybrid hits |
| Who calls this? | callers: + call hierarchy (LSP) |
| How does auth refresh work? | NL → symbols + anchors + semantic similarity |
| "credential renewal" (no token overlap) | Semantic hit on auth_refresh |
| Callers of X that match a shape | callers:X AND pattern:fn $NAME($$$) |
| Skip test callers | defs:handle AND NOT callers:test_ |
| Structured JSON for an agent | --json --format agent (why, follow_up_queries) |
| Structural rewrite / codemod | asgrep codemod (indexed native patterns) |
| Agent needs search as a tool (not a subprocess) | asgrep -- in-process, stateful session (Code Mode) |
On an indexed tree, asgrep is the search function. Do not spawn ripgrep, ast-grep, or Semgrep to find code the index already covers.
| Job | Tool |
|---|---|
| Find a name, a shape, a caller, or an idea | ast-sgrep (hybrid / defs: / callers: / pattern: / literal: / semantic) |
| Logs, generated files, or a tree you have not indexed | ripgrep |
| Full-rule rewrites and exotic ast-grep YAML | ast-grep (asgrep codemod covers indexed native patterns) |
| SAST rule packs | Semgrep |
Search quality is the product bar: exact identifiers rank the definition first, conceptual NL prefers code over docs that repeat the query, and vocabulary expansion is a precision tool. CLI process start on a tiny tree can still lose a raw rg race; that is not the contest. See comparison.md.
Index is incremental and lives under the project root at .asgrep/.
cargo build --release -p ast-sgrep-cli
./target/release/asgrep index .
./target/release/asgrep 'defs:auth_refresh' . --limit 3
./target/release/asgrep semantic 'credential renewal' . --limit 3
./target/release/asgrep chain 'auth_refresh' . --limit 3
./target/release/asgrep 'callers:process_request AND pattern:fn $NAME($$$)' .
./target/release/asgrep 'defs:handle AND NOT callers:test_' .
./target/release/asgrep call-path main validate_input .Optional overlays and rewrites:
./target/release/asgrep index . --scip path/to/index.json # JSON SCIP; degrades, never fails
./target/release/asgrep codemod --pattern 'legacy($ARG)' --rewrite 'modern($ARG)' --dry-run .Unprefixed queries run hybrid retrieval. Two-channel AND / AND NOT is recognized only when both sides are prefixed (defs:, callers:, imports:, pattern:, literal:, regex:, word:, or semantic:). See the query grammar.
Getting started → · Architecture → · Docs index →
ast-sgrep embeds symbol chunks (function/method/type with name, kind, callers, callees, excerpt), expanded with code-domain concept groups (auth ↔ credential ↔ token, refresh ↔ renewal, …). Chunks persist per-field vectors; query intent weights those fields instead of concatenating everything into one blob.
Query: "credential renewal"
→ semantic pass ranks auth_refresh (zero token overlap)
Embeddings come from one static engine compiled into the binary (256-d, deterministic). Large repos may use a persisted IVF-ANN sidecar (.asgrep/semantic.ivf). There is no cloud, Ollama, or ONNX embed client.
After fusion, the critic reviews the shortlist in-process. Agent envelopes expose why (including critic: notes) and causal follow_up_queries.
Semantic layer → · Fusion and critic → · Planner →
These are checked-in run summaries, not portable guarantees. Hardware, corpus, cache state, and flags all matter. Status vocabulary: benchmarks/README.md.
Latest: 2.6.0 release-paired measurement, 2026-10-19, against published 2.5.4 on frozen RepoQA-600 / CoSQA-500 corpora: identical harness, one idle host (Apple M5 Max), fresh indexes per arm, 0 errors across 4,400 query runs.
| RepoQA-600 | MRR@10 | Hit@1 | Hit@10 | ≥50%@8k | panel p50 |
|---|---|---|---|---|---|
| asgrep 2.5.4 | .297 | .190 | .563 | .340 | 16.6 ms |
| asgrep 2.6.0 | .719 | .608 | .928 | .790 | 22.2 ms |
| CoSQA-500 | MRR@10 | Hit@1 | Hit@10 | panel p50 |
|---|---|---|---|---|
| asgrep 2.5.4 | .104 | .054 | .240 | 13.8 ms |
| asgrep 2.6.0 | .327 | .234 | .588 | 22.1 ms |
Honest summary: 2.6.0 more than doubles 2.5.4's MRR (2.4×/3.1×) with fully offline operation. RepoQA index gets 37% faster (817 vs 1,296 ms median); the flat CoSQA corpus pays the embed cost (7.4 vs 5.5 s); storage 2.58→3.35 GB.
Self-corpus surfaces (same-day companion on the frozen 740-file bundle corpus):
| Surface | Published 2.5.4 | 2.6.0 |
|---|---|---|
| Pristine index p50 | 18.0 s | 13.5 s |
literal: / pattern: / defs: p50 |
3.8–6.5 ms | 4.2–6.6 ms (flat) |
| Conceptual hybrid NL p50 | 6.3–9.5 ms | 24.9–33.7 ms |
| Warm Code Mode round p50 (240 ids) | 0.67–0.76 ms | 4.27–4.84 ms |
| Self-gold MRR / nDCG (17 q) | .847 / .853 | .882 / .881 |
The 2026-10-11 candidate's conceptual-latency disasters (263+ ms) and ranking regression (.812 MRR) are repaired; the remaining always-on embed-channel cost on conceptual and warm-navigation queries is disclosed, not hidden. On this symbol-heavy self fixture the embed channel alone is mildly negative (no-embed .924 vs hybrid .882); on the external NL suites it is the whole gain. Neither corpus alone describes the product.
Canonical fingerprints, full matrices, raw sample bundles and regeneration
protocol:
baselines.md.
Previous speedups remain dated historical rows.
Foreign-corpus/neural bakeoffs, 23k/100k scale, watch-speed, intent-routing,
prevented-read and idx_big studies could not be refreshed because original
inputs are missing; their records remain explicitly historical/UNREPRODUCIBLE.
| Interface | Build | Use case |
|---|---|---|
| CLI | cargo build --release -p ast-sgrep-cli |
Terminal, scripts, call-path, codemod |
| MCP | cargo build --release -p ast-sgrep-mcp |
AI agents (stdio); structuredContent / outputSchema |
| Code Mode | ast-sgrep-codemode |
Programmatic tool-calling / multi-step plans (Pi) |
| LSP | cargo build --release -p ast-sgrep-lsp |
Editor navigation |
| Library | ast-sgrep-core |
Embed search in Rust tools |
| JSON plugins | --format agent|github|gitlab|agent-capsule |
Agents / CI |
| Doc | Contents |
|---|---|
| docs/README.md | Full documentation index |
| Getting started | Install, index, queries, flags |
| Pi package guide | Pi install, tools, data, security, updates, rollback, uninstall |
| Architecture | Index schema, search pipeline, crates |
| Query grammar | Prefixes, two-channel AND / AND NOT |
| Semantic search | Chunks, providers, IVF-ANN |
| Fusion ranking | RRF, post-fusion critic, why |
| Cascade planner | Retrieval cascade and causal follow-ups |
| Benchmarks | Methodology, reproduction, losses |
| Comparison | vs ripgrep / ast-grep |
| MCP · Code Mode · Use cases · Releasing | Agents, PTC, LSP, release checklist |
| Path | Role |
|---|---|
crates/ast-sgrep-core |
Index, SQLite store, hybrid search, critic, planner |
crates/ast-sgrep-cli |
asgrep / ast-sgrep CLI + supervisor |
crates/ast-sgrep-lang |
Tree-sitter extraction (15 languages) |
crates/ast-sgrep-embed |
Static embedding engine (compiled-in table) |
crates/ast-sgrep-mmap |
Memory-map helpers |
crates/ast-sgrep-lsp |
Language server |
crates/ast-sgrep-mcp |
MCP server |
crates/ast-sgrep-watch |
asgrep-watch helper (watch implementation; owns the notify link) |
crates/ast-sgrep-codemode |
Code Mode / programmatic tool-calling |
crates/ast-sgrep-plugins |
Output formats |
crates/ast-sgrep-testkit |
Shared fixtures for search/index/Pi tests |
tests/ |
Search, index, and Pi behavior tests |
packages/pi/ |
Pi extension, launcher, and native packages |
packages/agent-plugin/ |
Portable Agent Plugins + MCP |
benchmarks/ |
Published results (results/) and studies (studies/) |
docs/ |
User and architecture docs |
v2.5.4. Native and Pi boundary fixes preserve source/path identity, cancellation, scoped lookup, and explicitly returned data. Local-first embeddings, index schema 16, two-channel conjunction, post-fusion critic, causal follow-ups, SCIP overlay, call-path, indexed codemod, and Pi Code Mode (results on the model path) are in place. 15 languages (Dart, MoonBit newest), fusion-normalized ranking, and the hashed semantic layer remain. Search is read-only by default; MCP runs on official rmcp.
GitHub Actions workflows are manual-only (workflow_dispatch) to control Actions minutes. Local quality bar for contributors:
cargo check --workspace -j1
cargo test -p ast-sgrep-core --test parity -j1 -- --test-threads=1
cargo test -p ast-sgrep-cli --test cli_smoke -j1 -- --test-threads=1
cargo build --release -p ast-sgrep-cli -j1
./target/release/asgrep --helpSee CONTRIBUTING.md.
MIT. See LICENSE.