@history-lab/cli

Search ~5M declassified historical documents from the command line. CIA, State Department cables, FRUS diplomatic records, World Bank, NATO, and more.

Data from the FOI Archive by History Lab at Columbia University.

Install

npm install -g @history-lab/cli

Or run directly:

npx @history-lab/cli corpus-search --query "cuban missile crisis" --corpus cia

Quick Start

All commands work immediately with no setup.

# Semantic search using natural language
history-lab search "nuclear weapons testing in the Pacific"

# Search CIA documents about the Cuban Missile Crisis
history-lab corpus-search --query "cuban missile crisis" --corpus cia

# Find Kissinger's cables to Nixon in 1973
history-lab frus-search --sender kissinger --recipient nixon --from 1973-01-01 --to 1974-01-01

# Look up entities
history-lab entities "Castro"

# Get full text of a document
history-lab document CIA-RDP79T00429A001400010019-1

# Archive statistics
history-lab stats

Commands

search <query> — Semantic vector search

Natural language search across all documents.

history-lab search "Soviet influence in Latin America" --limit 20
history-lab search "diplomatic cables about Vietnam" --from 1968 --to 1975

Option	Description
-n, --limit <n>	Number of results (1-100, default 10)
--from <date>	Start date (YYYY, YYYY-MM, or YYYY-MM-DD)
--to <date>	End date

corpus-search — Full-text search

Structured search using PostgreSQL full-text search. No API key needed.

history-lab corpus-search --query "cuba" --corpus cia --classification secret
history-lab corpus-search --query "vietnam war" --corpus cables --from 1968-01-01 --to 1969-01-01

Option	Description
-q, --query <text>	Full-text search query
-t, --title <text>	Title search (case-insensitive)
-c, --corpus <id>	Filter by collection (see corpora command)
--classification <level>	Security classification filter
--from <date>	Start date (ISO format)
--to <date>	End date
--fts-mode <mode>	plfts (default), phfts (exact phrase), wfts (websearch)
-n, --limit <n>	Max results (default 25)
--offset <n>	Pagination offset
--body	Include document body text

frus-search — FRUS diplomatic records

Search the Foreign Relations of the United States collection (312K documents, 1620-1989). Includes sender, recipient, and location metadata.

history-lab frus-search --query "Chile coup" --sender kissinger
history-lab frus-search --sender kissinger --recipient nixon --from 1973-01-01 --to 1974-01-01

Option	Description
-q, --query <text>	Full-text search query
--sender <name>	Sender name
--recipient <name>	Recipient name
--location <place>	Where the document was authored
--from <date>	Start date
--to <date>	End date
--classification <level>	Classification filter
-n, --limit <n>	Max results (default 25)

document <id> — Fetch a document

history-lab document CIA-RDP79T00429A001400010019-1
history-lab document CIA-RDP79T00429A001400010019-1 --enriched  # includes entities and topics

corpora — List collections

history-lab corpora

Shows all available document collections with document counts, date ranges, and word counts.

entities <query> — Entity lookup

history-lab entities "Castro"
history-lab entities "Vietnam" --group LOC

Option	Description
-g, --group <type>	Filter: PERSON, LOC, ORG, or OTHER
-n, --limit <n>	Max results (default 25)

entity-docs <entityId> — Documents for an entity

history-lab entity-docs 12345 --limit 10

topics <corpus> — Topic models

history-lab topics cia                    # list all topics
history-lab topics cia --topic-id 5       # top documents for topic 5

stats — Archive statistics

history-lab stats                     # total docs, pages, words
history-lab stats --decades           # breakdown by decade
history-lab stats --classifications   # breakdown by classification level

Output Formats

All commands support three output formats:

history-lab corpus-search --query "cuba" --corpus cia            # human-readable (default)
history-lab corpus-search --query "cuba" --corpus cia --json     # JSON (for scripting)
history-lab corpus-search --query "cuba" --corpus cia --markdown # Markdown

Use --json to pipe into jq or other tools:

history-lab corpus-search --query "cuba" --corpus cia --json | jq '.[0].title'

Available Corpora

ID	Collection	Documents
cia	CIA CREST	~936K
cables	State Department Cables	~718K
cfpf	Central Foreign Policy Files	~3.2M
frus	Foreign Relations of the US	~312K
clinton	Clinton Emails	~33K
kissinger	Kissinger Transcripts	~3K
briefing	Presidential Daily Briefs	~12K
nato	NATO Archives	~2K
un	UN Archives	~5K
worldbank	World Bank Archives	~9K
cabinet	UK Cabinet Papers	~3K

Configuration

All commands work out of the box with no configuration. Advanced users can override the default API endpoints:

Environment Variable	Description
HISTORYLAB_CORPUS_API_URL	Override corpus API URL
HISTORYLAB_VECTOR_API_URL	Override vector API URL
HISTORYLAB_VECTOR_API_KEY	Override vector API key

MCP Server

For AI tool use (Claude, etc.), this project also provides a remote MCP server:

https://mcp.declassification-engine.org/mcp

See the repository for MCP setup instructions.

License

MIT