Classification: CONFIDENTIAL
Document: MEERU-ARCH-README | Version: 1.0
Prepared for: Enterprise Architecture Review Board
[ COMMING SOON ]
- Executive Summary
- Platform Vision & Mission
- Architecture Overview
- System Architecture
- Intelligence Ontology (MIO)
- Entity Resolution
- Graph Intelligence
- AI Architecture
- Biometric Intelligence
- Security Architecture
- Database Architecture
- Kubernetes & DevOps
- Dark Web & Deep Web Intelligence
- GEOINT Architecture
- Technology Stack
- Deployment Models
- Scalability Envelope
- Roadmap
- Engineering Team Structure
- Document Registry
MEERU is the world's most advanced intelligence, investigation, analytics, and OSINT platform. Designed to operate at national, enterprise, research, investigative, cybersecurity, financial intelligence, and threat intelligence scale, MEERU unifies every intelligence discipline into a single coherent, AI-native platform.
Built for scale: 100B+ entities, multi-trillion relationships, petabyte-scale storage, millions of events per second, and global multi-region deployment.
| Capability | Description |
|---|---|
| Unified Intelligence | OSINT, GEOINT, CTI, Dark Web, Biometrics, and Financial domains in one platform |
| AI-Native | Autonomous collection, correlation, and report generation from the ground up |
| Enterprise Security | Zero Trust, RBAC/ABAC, encryption, air-gapped deployment support |
| Hyperscale Graph | Multi-trillion relationship support with GNN-powered analytics |
| Investigator-Friendly UI | Case management, evidence chains, timeline analysis, collaboration |
| Sovereign Deployment | Multi-cloud, on-premises, and air-gapped environments supported |
Mission: To provide governments, enterprises, and intelligence professionals with an unmatched, ethically governed, and technically superior intelligence capability — fusing open-source intelligence, biometric analysis, cyber threat intelligence, geospatial analytics, dark web monitoring, and AI-driven investigation into a unified operating platform.
| Pillar | Description | Key Technologies |
|---|---|---|
| Intelligence Collection | Distributed crawling of surface, deep, and dark web; real-time news, social, and public records | Scrapy, Playwright, Kafka, Flink |
| Entity Resolution | Probabilistic multi-source identity fusion, alias detection, cross-language matching | Neo4j, Weaviate, custom ML models |
| Graph Intelligence | Multi-trillion relationship graph with GNN, link prediction, community detection | JanusGraph, PyG, DGL |
| AI Investigation Layer | Autonomous AI investigators, analyst co-pilot, multi-agent workflows | vLLM, Llama, Mistral, LangGraph |
| Biometric Intelligence | Face, voice, gait, iris, fingerprint recognition and cross-modal fusion | PyTorch, ArcFace, ECAPA-TDNN |
| GEOINT | Satellite imagery, aircraft/maritime/vehicle tracking, geofencing, route prediction | Deck.gl, MapLibre, GDAL, PostGIS |
| Threat Intelligence | IOC tracking, threat actor monitoring, campaign attribution, attack surface analysis | MISP, OpenCTI, custom CTI models |
| Investigation Platform | Case management, evidence chain, timeline, collaborative workspaces | Next.js, Temporal, PostgreSQL |
MEERU is architected as a cloud-native, microservices-based platform with strict domain separation, event-driven data flows, and a layered API gateway. The platform is divided into six horizontal layers, each independently scalable.
Golden Rule: All services communicate through Kafka event buses or gRPC. No direct database access crosses service boundaries.
| Layer | Services | Responsibility |
|---|---|---|
| Collection Layer | Crawler services, Tor proxy fleet, Social ingestors, Document collectors | Raw data acquisition from all sources |
| Ingestion Layer | Kafka topics, Flink processing, Schema registry, Data normalizer | Event streaming, parsing, normalization |
| Storage Layer | PostgreSQL, ClickHouse, Neo4j, JanusGraph, Weaviate, Redis, Object store | Persistent and analytical data storage |
| Intelligence Layer | Entity resolver, Graph engine, AI models, CTI processor, Biometric engine | Enrichment, analysis, correlation |
| API Layer | REST gateway, GraphQL API, gRPC services, WebSocket feeds | Unified external interface |
| Presentation Layer | Next.js web app, Flutter mobile app, Embeddable widgets | User-facing interfaces |
| Service | Technology | Description |
|---|---|---|
surface-crawler |
Scrapy | Distributed web crawler with 1000+ node cluster |
tor-crawler |
Isolated Tor proxy fleet | Dark/deep web collection |
social-ingestor |
Real-time stream collectors | Social platform stream collection |
news-ingestor |
RSS, news API, NLP | Article collector |
dns-collector |
Passive DNS, WHOIS, CT logs | Certificate transparency log collector |
doc-collector |
PDF, DOCX, image, multimedia | Document ingestion |
pubrecords-collector |
Government database automation | Public records collection |
| Service | Capability |
|---|---|
entity-extractor |
NER, relationship extraction, event detection (multilingual) |
entity-resolver |
Probabilistic identity fusion and deduplication |
graph-builder |
Graph edge creation and relationship inference |
enrichment-engine |
Cross-source attribute enrichment and scoring |
confidence-scorer |
Data quality, source reliability, entity confidence scoring |
translation-service |
100+ language neural machine translation |
ocr-service |
Tesseract + custom ML OCR for document intelligence |
| Service | Capability |
|---|---|
graph-analytics |
GNN inference, link prediction, community detection |
biometric-engine |
Face, voice, gait, iris, fingerprint inference cluster |
geoint-service |
Geospatial analytics, trajectory analysis, geofencing |
cti-service |
Threat intelligence processing, IOC enrichment, attribution |
ai-investigator |
Autonomous AI agent orchestration (LangGraph + vLLM) |
report-generator |
LLM-powered intelligence report generation |
search-service |
Hybrid search federation across OpenSearch + Weaviate |
Collection → Storage Pipeline:
- Collectors emit raw events to Kafka topic:
raw.collection.* - Flink jobs parse, validate, and normalize to canonical schema
- Normalized entities written to PostgreSQL (operational) and ClickHouse (analytical)
- Graph edges published to Kafka topic:
graph.edges.*consumed by JanusGraph writer - Embeddings generated by ML services, indexed in Weaviate
- All events replicated to object store (S3/GCS) for cold storage and replay
Query Flow:
- Client requests hit the API gateway (Kong or custom Go gateway)
- Gateway applies RBAC/ABAC policies, rate limiting, and tenant isolation
- Search queries federated across OpenSearch (keyword) + Weaviate (semantic)
- Graph queries routed to JanusGraph or Neo4j depending on traversal depth
- AI inference requests queued to vLLM serving cluster via gRPC
- Results assembled, confidence-scored, and returned with provenance metadata
| API Type | Use Case | Protocol | Auth |
|---|---|---|---|
| REST | Standard CRUD, search, entity lookup | HTTPS | OAuth2 + JWT |
| GraphQL | Complex nested queries, relationship traversal | HTTPS | OAuth2 + JWT |
| gRPC | High-throughput internal service communication | mTLS | Service accounts |
| WebSocket | Real-time alerts, live collection feeds | WSS | JWT |
| Streaming | Bulk export, continuous intelligence feeds | HTTPS chunked | API Key + JWT |
- Multi-region active-active with latency-based routing
- Circuit breakers on all inter-service calls (Istio)
- Automatic failover for all stateful services (Patroni for PostgreSQL, JanusGraph HA)
- Kafka replication factor >= 3 for all topics
- Zero-downtime deployments via blue-green and canary strategies (ArgoCD)
- 99.99% SLA target for all Tier-1 APIs
- RTO < 5 minutes, RPO < 30 seconds for critical intelligence data
The MEERU Intelligence Ontology (MIO) defines the complete knowledge model for all intelligence domains. It establishes canonical entity types, relationship types, attributes, and semantic reasoning rules that govern all data across the platform.
MIO is versioned, schema-evolved, and reasoned over using OWL 2 DL semantics. All platform data is mapped to MIO before storage.
- Composability: Domain ontologies extend a common base ontology
- Versioning: Every schema change is versioned and backward-compatible
- Semantic richness: Support for OWL 2 class expressions and property chains
- Multilingual: Labels and definitions in 100+ languages
- Provenance: Every attribute carries source, timestamp, and confidence
| Entity Type | Sub-types | Key Attributes |
|---|---|---|
| Person | Individual, Alias, Organization Member, Threat Actor, Victim | Name, DOB, nationality, biometrics, aliases, identifiers |
| Organization | Corporation, NGO, Government, Criminal Group, Threat Group | Name, jurisdiction, registration, ownership, subsidiaries |
| Device | Computer, Mobile, IoT, Network Device, Vehicle | MAC, IP, IMEI, serial, firmware, location history |
| Location | Country, Region, City, Building, Coordinate, Geofence | Coordinates, address, boundary, timezone, metadata |
| Event | Incident, Transaction, Communication, Publication, Attack | Timestamp, participants, location, type, evidence |
| Digital Asset | Domain, IP, Email, Account, Certificate, Malware | Value, registrar, hosting, creation, relationships |
| Financial Asset | Account, Wallet, Transaction, Company, Fund | Value, currency, institution, ownership, transactions |
| Document | Article, Report, Court Filing, Social Post, Image, Video | Content, author, date, source, language, entities |
| Cyber Object | IOC, Vulnerability, Campaign, Infrastructure, Tool | Type, value, severity, attribution, TTPs |
| Relationship | Domain | Range | Properties |
|---|---|---|---|
knows |
Person | Person | strength, context, first_seen, last_seen |
memberOf |
Person | Organization | role, start_date, end_date, status |
controls |
Person/Org | Organization/Asset | type, percentage, mechanism |
locatedAt |
Person/Org/Event | Location | timestamp, duration, frequency |
communicatedWith |
Person/Device | Person/Device | channel, frequency, content_hash |
usedIn |
Device/Tool | Event/Campaign | timestamp, role, confidence |
attributedTo |
Event/Campaign | Person/Org | confidence, evidence, analyst |
relatedTo |
Any | Any | type, strength, confidence, source |
sameAs |
Any | Any | confidence, method, verified |
Cyber Ontology:
- Maps to STIX 2.1 and MITRE ATT&CK framework
- TTP hierarchy: Tactic > Technique > Sub-technique > Procedure
- Campaign → ThreatActor, Infrastructure, Malware, Tool chains
- IOC types: IP, domain, hash, URL, email, certificate, YARA rule
- Kill chain phase mapping for temporal attack sequence analysis
Financial Ontology:
- Legal entity hierarchy: UBO → Company → Subsidiary → Account
- Transaction graph with temporal sequencing and pattern attributes
- Crypto wallet clustering with known exchange attribution
- Regulatory jurisdiction mapping for cross-border analysis
- Sanction and PEP list integration as ontology annotations
Person Ontology:
- Identity graph: canonical person → aliases → accounts → devices
- Biometric node types: FaceEmbedding, VoicePrint, GaitSignature, IrisCode
- Social graph: contacts, associates, family, colleagues, adversaries
- Temporal attributes: known locations by time, activity patterns
- Confidence decay: attribute confidence degrades over time without confirmation
| Version Event | Process | Impact |
|---|---|---|
| Minor attribute addition | Additive schema change, zero downtime migration | Backward compatible, new data enriched |
| Entity type addition | Blue-green migration with dual-write period | New type available after validation |
| Relationship type change | Graph migration with full audit trail | Requires re-indexing period |
| Major restructuring | Versioned namespace fork, gradual migration | Both versions served in parallel |
- OWL 2 DL reasoning via Apache Jena or RDF4J
- Inference rules: transitive closure, inverse properties, property chains
- Example rule:
knows(A,B) ∧ memberOf(B, G) → associatedWith(A, G) - SPARQL 1.1 query support for semantic graph queries
- Reasoning results cached in Neo4j and periodically refreshed
- Contradiction detection: conflicting facts flagged for analyst review
Entity Resolution (ER) is the foundational intelligence capability that determines whether two or more records refer to the same real-world entity. At MEERU scale, this requires a distributed, probabilistic, multi-modal system capable of processing billions of records in near-real-time.
MEERU's Entity Resolution Engine achieves precision >97% and recall >94% across 100B+ entity records through multi-stage blocking, probabilistic scoring, and continuous reconciliation.
| Stage | Technique | Throughput Target |
|---|---|---|
| Blocking | LSH, phonetic blocking, fingerprint hashing | Reduce candidate pairs by 99.9% |
| Candidate generation | Inverted index, embedding similarity (FAISS) | Top-K candidates per record |
| Feature extraction | String similarity, semantic embedding, structured attribute comparison | Per-candidate feature vectors |
| Scoring | Gradient-boosted classifier + calibrated probability | Probability P(match) per pair |
| Clustering | Agglomerative clustering with threshold | Resolved entity clusters |
| Confidence assignment | Ensemble confidence with uncertainty bounds | Per-entity confidence score |
| Reconciliation | Delta-based update on new data | Continuous, near-real-time |
String Similarity Methods:
- Jaro-Winkler similarity for short names and identifiers
- Levenshtein edit distance for typo and OCR error tolerance
- Soundex, Metaphone, and Double Metaphone for phonetic matching
- N-gram TF-IDF cosine similarity for longer text fields
- Embedding-based semantic similarity (multilingual BERT / LaBSE)
Cross-Language & Transliteration:
- ICU transliteration library for script-to-Latin conversion
- 100+ language support: Arabic, Chinese, Cyrillic, Devanagari, etc.
- Back-transliteration models for Arabic-English-Arabic name cycles
- Named entity translation using mBART-50 and Helsinki-NLP models
- Language-specific nicknames and common alias dictionaries
| Signal Type | Weight Range | Reliability Score |
|---|---|---|
| Government ID / biometric | 0.90 – 1.00 | Highest |
| Face embedding similarity | 0.75 – 0.95 | Very High |
| Full name exact match | 0.70 – 0.90 | High |
| Voice biometric match | 0.65 – 0.88 | High |
| Device fingerprint | 0.60 – 0.85 | High |
| Email / phone | 0.55 – 0.80 | Medium-High |
| Username / handle | 0.40 – 0.70 | Medium |
| Location co-occurrence | 0.20 – 0.55 | Medium-Low |
| Behavioral pattern | 0.30 – 0.60 | Medium |
- AKA graph: each canonical entity maintains an alias cluster
- Alias types: name variants, pseudonyms, organizational roles, usernames, handles
- Temporal alias tracking: alias active during specific time windows
- Cross-lingual alias normalization for multilingual identities
- Alias confidence scoring: high-confidence aliases promoted to canonical
Real-Time Update Flow:
- New entity records stream to ER pipeline via Kafka
- Blocking layer generates candidate pairs incrementally
- Only affected entity clusters re-evaluated (delta processing)
- Cluster merges and splits propagated as graph events
- Downstream systems notified via Kafka of entity resolution changes
- Full reconciliation sweep scheduled nightly for consistency check
Conflict Resolution:
- Contradictory attributes flagged with conflict type and confidence
- Source reliability weights determine authoritative value selection
- Temporal priority: more recent data wins within same source tier
- Analyst override: manual resolution locks cluster with audit trail
- Contradiction dashboard: daily review queue for unresolved conflicts
| Component | Technology | Scale |
|---|---|---|
| Blocking index | FAISS + Redis inverted index | 100B+ records, sub-second lookup |
| Feature computation | Python + scikit-learn, distributed Spark | Millions of pairs per hour |
| Scoring model | XGBoost ensemble, retrained weekly | >97% precision at 94% recall |
| Cluster storage | PostgreSQL + Neo4j (cluster-entity links) | Billions of cluster-member edges |
| Streaming pipeline | Kafka + Flink ER job | Millions of new records per hour |
MEERU's graph intelligence platform is the core analytical engine, enabling relationship discovery, influence analysis, community detection, and temporal pattern analysis across multi-trillion edge graphs. The system is built on a federated, distributed graph architecture supporting both transactional and analytical workloads.
The MEERU graph layer targets multi-trillion relationship support through a federated shard architecture, with GNN inference running on dedicated GPU clusters.
| Graph Store | Use Case | Scale Target |
|---|---|---|
| JanusGraph + Cassandra | Primary operational graph — multi-trillion edges, OLTP traversals | 10T+ edges, sub-second 5-hop queries |
| Neo4j AuraDB | Analytical subgraphs, complex Cypher queries, visualization | 1B-10B node subgraphs |
| Weaviate | Embedding-based graph search, semantic similarity traversal | Billions of vector nodes |
| Redis Graph | Ephemeral investigation workspace graphs, session-scoped | Millions of edges per workspace |
| ClickHouse | Graph analytics aggregations, temporal pattern queries | Petabyte-scale edge history |
Node Schema:
- Every node has:
id(UUID),type(ontology class),confidence(0.0-1.0),first_seen,last_seen,source_ids[] - Node attributes stored as typed property maps with provenance metadata
- Node embedding vector (1024-dim) stored in Weaviate for similarity search
- Temporal versioning: attribute history preserved with timestamp ranges
Edge Schema:
- Every edge has:
id,type,source_node,target_node,confidence,weight,first_seen,last_seen,source_ids[] - Directed edges with bidirectional traversal support
- Temporal edges: valid time intervals for historical relationship analysis
- Edge types are ontology-constrained (MIO relationship types)
| Model | Algorithm | Use Case |
|---|---|---|
| RelationshipPredictor | GraphSAGE + edge classifier | Predict hidden/missing relationships between entities |
| CommunityDetector | Graph Transformer + modularity optimization | Identify entity clusters and criminal networks |
| InfluenceRanker | PageRank-GNN hybrid | Score entity influence within networks |
| AnomalyDetector | Variational Graph Autoencoder | Detect unusual graph patterns and emerging threats |
| TemporalEvolution | Temporal Graph Network (TGN) | Model how relationships evolve over time |
| EntityClassifier | GAT (Graph Attention Network) | Classify unknown entities using neighborhood context |
GNN Inference Infrastructure:
- PyTorch Geometric (PyG) for GNN model implementation and training
- DGL (Deep Graph Library) for large-scale distributed graph learning
- GPU cluster: A100 80GB nodes for training, A10G for inference serving
- Model versioning and A/B testing via MLflow
- Batch inference: nightly full-graph GNN sweeps for updated embeddings
- Online inference: sub-200ms per-node GNN inference for real-time queries
| Capability | Algorithm | Output |
|---|---|---|
| Centrality Analysis | Betweenness, Eigenvector, Katz | Key broker and influencer nodes |
| Community Detection | Louvain, Leiden, Label Propagation | Entity clusters and group structures |
| Link Prediction | GNN + Jaccard + Adamic-Adar | Probable hidden connections |
| Path Analysis | Bidirectional BFS, Dijkstra | Shortest and weighted paths between entities |
| Temporal Analysis | Temporal motif mining, event sequences | Activity patterns, behavioral timelines |
| Subgraph Matching | VF2, QuickSI, custom GNN | Pattern matching for known network signatures |
| Influence Spreading | Independent Cascade, Linear Threshold | Information/threat propagation modeling |
- Sigma.js: Canvas/WebGL rendering for up to 500K nodes in browser
- Graphology: In-memory graph data structure for client-side manipulation
- Deck.gl: Geospatial graph overlays on map layer
- Automatic layout: ForceAtlas2, Fruchterman-Reingold, hierarchical
- Progressive loading: Streaming graph expansion from server-side pagination
- Investigation lens: Time-slice, filter by confidence, highlight shortest path
- Export: PNG, SVG, GraphML, GEXF, JSON graph formats
- Multi-shard graph partitioned by entity type and geographic region
- Cross-shard traversal via MEERU Graph Federation Protocol (gRPC-based)
- Federated Cypher / Gremlin query router for transparent cross-shard queries
- Global entity index in Redis for routing any entity ID to its shard
- Shard rebalancing via background migration with zero-downtime
MEERU's AI layer transforms raw data into actionable intelligence. It is built on a multi-agent, multi-modal AI architecture that combines LLMs, computer vision models, graph neural networks, and autonomous investigation agents into a cohesive intelligence amplification system.
All AI inference is served through vLLM, enabling 10x throughput improvement over standard HuggingFace inference. Models run on dedicated A100/A10G GPU clusters with autoscaling.
| AI Domain | Capability | Model Backbone |
|---|---|---|
| LLM Reasoning | Investigation assistance, hypothesis generation, Q&A | Llama 3.1 70B / Mistral Large |
| Report Generation | Automated intelligence reports in analyst format | Llama 3.1 70B + structured prompting |
| NLP - NER | Entity and relationship extraction from text | GLiNER / spaCy + custom fine-tuned |
| NLP - Summarization | Multilingual document summarization | mBART-50 / NLLB-200 |
| NLP - Translation | 100+ language translation | NLLB-200, Helsinki-NLP |
| NLP - Sentiment | Political, financial, threat sentiment analysis | Fine-tuned DeBERTa |
| Computer Vision - OCR | Document text extraction | Tesseract 5 + PaddleOCR |
| Computer Vision - Detection | Object, logo, landmark detection | YOLOv9, CLIP |
| Computer Vision - Face | Face detection, recognition, clustering | RetinaFace + ArcFace |
| Computer Vision - Video | Scene understanding, activity detection | VideoMAE, SlowFast |
| GNN | Link prediction, community detection, anomaly detection | PyG: GraphSAGE, GAT, TGN |
AI Investigator Agent: The AI Investigator is an autonomous LLM-powered agent capable of conducting multi-step intelligence investigations with minimal human guidance.
- Receives investigation task: target, objective, scope, classification
- Autonomously queries MEERU search, graph, OSINT, and dark web modules
- Generates hypothesis chain: proposes leads, ranks by confidence
- Executes collection subtasks and feeds results back into reasoning loop
- Produces structured intelligence report with evidence citations
- Human-in-the-loop checkpoints for high-consequence decisions
Multi-Agent Workflow (LangGraph):
| Agent | Role | LLM |
|---|---|---|
| Orchestrator | Plans investigation, routes subtasks, assembles final report | Llama 70B |
| Collector Agent | Triggers collection tasks, monitors completion, validates data | Mistral 7B |
| Analyst Agent | Interprets collected data, extracts insights, scores confidence | Llama 70B |
| Hypothesis Agent | Generates and ranks investigation hypotheses | Llama 70B |
| Report Agent | Structures and drafts intelligence reports in analyst format | Llama 70B |
| Verification Agent | Fact-checks claims against knowledge base and live search | Mistral 7B |
Text Processing Flow:
- Raw text → Language detection (langdetect + fastText)
- Translation → English normalization for cross-language processing
- NER → Entity extraction with type, span, confidence
- Relationship extraction → Subject-Predicate-Object triples
- Event extraction → Event type, participants, location, time
- Sentiment → Document and entity-level sentiment scores
- Topic modeling → LDA + BERTopic for thematic clustering
- Entities & relationships → published to Kafka for graph ingestion
Multilingual Support:
- 100+ languages via NLLB-200 translation model
- Native NER models for 40+ high-priority languages
- Cross-lingual embeddings: LaBSE, multilingual-E5-large
- RTL support: Arabic, Hebrew, Persian, Urdu
- CJK tokenization: custom tokenizers for Chinese, Japanese, Korean
| Task | Model | Latency (p99) | Hardware |
|---|---|---|---|
| Face detection | RetinaFace | < 50ms | A10G GPU |
| Face recognition | ArcFace R100 | < 80ms | A10G GPU |
| Object detection | YOLOv9 | < 40ms | A10G GPU |
| OCR | PaddleOCR + Tesseract | < 200ms | CPU cluster |
| Image embedding | CLIP ViT-L/14 | < 100ms | A10G GPU |
| Video analysis | VideoMAE | < 500ms/clip | A100 GPU |
| Logo recognition | EfficientNet + custom head | < 60ms | A10G GPU |
| Reverse image search | CLIP + FAISS | < 150ms | A10G + CPU |
- vLLM for LLM serving: PagedAttention, continuous batching, 10x throughput
- Triton Inference Server for CV and embedding models
- ONNX Runtime for CPU-optimized model inference
- Model registry: MLflow for versioning, lineage, and A/B deployment
- Autoscaling: Kubernetes HPA + custom GPU metrics for inference pods
- Quantization: INT8/INT4 for cost optimization on non-critical models
The MEERU Biometric Intelligence Platform (BIP) delivers enterprise-grade biometric recognition across five modalities: facial recognition, voice identification, gait analysis, iris recognition, and fingerprint matching. The platform supports both individual modality search and multi-modal identity fusion.
BIP achieves 1:1 billion facial search in under 500ms using ArcFace embeddings and FAISS approximate nearest neighbor indexing on GPU clusters.
| Modality | EER Target | Acquisition | Range | Covert Collection |
|---|---|---|---|---|
| Face | < 0.1% | Camera, image, video | 0 – 100m | Yes |
| Voice | < 0.5% | Audio recording | 0 – 20m | Yes |
| Gait | < 2.0% | CCTV, radar | 0 – 50m | Yes |
| Iris | < 0.01% | Near-IR camera | 0 – 3m | Partial |
| Fingerprint | < 0.1% | Sensor, latent | Contact / lifted | No |
Face Detection:
- RetinaFace: multi-scale face detection with 5-point landmark localization
- Supports detection in crowds, occlusion, poor lighting, and oblique angles
- Video face tracking: DeepSORT multi-object tracker for face trajectory
- Age and gender estimation as auxiliary attributes (AgeNet, GenderNet)
- Liveness detection: passive liveness check to reject spoofing
Face Recognition & Search:
- ArcFace R100 on MS1MV3: 512-dim embedding, 99.77% LFW accuracy
- FAISS IVF-PQ index: 1:1B search in < 500ms on GPU
- Face clustering: DBSCAN on embeddings for group identity discovery
- Similarity threshold: configurable per use case (0.4 – 0.7 cosine)
- Face relationship graph: co-occurrence, co-location, known association edges
Speaker Identification & Verification:
- ECAPA-TDNN speaker encoder: 192-dim speaker embedding
- Speaker diarization: pyannote.audio for multi-speaker audio segmentation
- Language-independent: 40+ language speaker verification
- Emotion and stress detection as auxiliary signals
- Audio enhancement: SEGAN denoising for degraded recordings
- GaitNet: skeleton-based gait recognition using OpenPose body keypoints
- Silhouette-based backup: GEI (Gait Energy Image) for low-resolution CCTV
- Covariates: clothing, speed, surface — covariate-robust models
- Gait sequence required: minimum 5 complete gait cycles for reliable score
- Gait biometric stored as 256-dim temporal embedding
- IrisCodes: Daugman's algorithm for 2048-bit iris code generation
- Hamming distance matching: < 0.32 threshold for positive match
- NIR illumination required: near-infrared acquisition for accurate segmentation
- Segmentation: UNet-based iris segmentation for off-axis and partially occluded images
- Minutiae extraction: SourceAFIS-compatible extraction pipeline
- Rolled, flat, and latent fingerprint support
- Latent print enhancement: NBIS MINDTCT + custom deep learning enhancement
- AFIS search: NIST NBIS-compatible 1:N search with candidate list ranking
- Palm print: full palm print matching using landmark-guided alignment
| Fusion Strategy | Approach | Use Case |
|---|---|---|
| Score-level fusion | Weighted sum of normalized modality scores | Standard multi-modal match |
| Feature-level fusion | Concatenated embeddings + MLP classifier | High-confidence confirmation |
| Decision-level fusion | Majority vote or AND/OR logic across modalities | Threshold-based verification |
| Bayesian fusion | Probabilistic joint inference with prior | Low-quality input handling |
The unified identity score from multi-modal fusion is written to the MEERU identity graph as a ConfidenceScore edge attribute on the Person node, enabling investigation queries like: find all individuals with face confidence > 0.85 AND voice match > 0.80.
MEERU's security architecture is designed for the highest-sensitivity intelligence environments, implementing Zero Trust principles, defense-in-depth, and continuous compliance verification. The platform is built to pass FedRAMP High, ISO 27001, SOC 2 Type II, and government accreditation requirements.
No implicit trust — every request authenticated, authorized, and logged. Every data access is attributed, timestamped, and auditable to the individual analyst level.
- Zero Trust: Never trust, always verify — identity-centric perimeter
- Least privilege: Minimum access rights for every identity and service
- Defense in depth: Multiple independent security controls at every layer
- Assume breach: Detect and contain, not just prevent
- Immutable audit: Append-only audit log that cannot be modified by any operator
Authentication:
| User Type | Authentication Method | MFA |
|---|---|---|
| Analyst | OIDC via enterprise IdP (Okta, Azure AD) | TOTP or FIDO2 hardware key — required |
| Operator / Admin | Certificate + OTP + hardware key | Hardware FIDO2 — mandatory |
| Service Account | mTLS + short-lived JWT (< 15 min) | N/A — certificate rotation |
| API Consumer | OAuth2 client credentials + API key | Webhook signing secret |
| Air-gapped users | PKI certificate on smart card | PIN + certificate chain |
Authorization: RBAC + ABAC
- RBAC: Role hierarchy — SuperAdmin > TenantAdmin > Analyst > ReadOnly > Guest
- ABAC: Dynamic policies on entity sensitivity, classification, geography, time
- Policy engine: Open Policy Agent (OPA) for real-time policy evaluation
- Attribute sources: user department, clearance, MFA status, time-of-day, IP range
- Example ABAC rule:
allow access IF clearance >= SECRET AND location = headquarters AND MFA_satisfied - Data masking: Automatically redact fields above user clearance level
| Data State | Algorithm | Key Management |
|---|---|---|
| Data at rest (DB) | AES-256-GCM | Vault-managed DEKs, 90-day rotation |
| Data at rest (storage) | AES-256-GCM with customer key option | Tenant-managed KEKs via Vault |
| Data in transit (external) | TLS 1.3 minimum | Auto-renewed Let's Encrypt / PKI |
| Data in transit (internal) | mTLS via Istio service mesh | SPIFFE/SPIRE identity framework |
| Backups | AES-256-GCM + separate backup key | Offline cold key storage |
| Air-gapped environments | AES-256 with HSM | Hardware Security Module (HSM) |
- HashiCorp Vault: Central secrets management with dynamic secret generation
- Database credentials: Dynamic 1-hour TTL credentials per service
- API keys: Vault Agent injection — no secrets in config files or environment variables
- PKI: Vault PKI engine issues short-lived service certificates (24-hour TTL)
- Secrets rotation: Automatic with zero-downtime dual-validation window
- Break-glass: Encrypted emergency access with mandatory audit notification
Audit Log Requirements:
- Every API request logged: identity, action, resource, timestamp, IP, user agent
- Every data access logged: entity accessed, fields returned, classification level
- Every configuration change: before/after state, approver, ticket reference
- Logs written to append-only ClickHouse table (no UPDATE/DELETE permissions)
- Logs replicated to WORM (Write Once Read Many) object storage within 60 seconds
- Log retention: 7 years for all security-relevant events
Audit Query Examples:
- Who accessed entity X in the past 30 days?
- All searches performed by analyst A in case Y?
- Every API call that returned classification CONFIDENTIAL data?
- Configuration changes in the past 90 days with approver?
- Tenant namespace isolation at Kubernetes level (separate namespaces, network policies)
- Database-level row security: PostgreSQL RLS policies enforce tenant isolation
- Graph isolation: JanusGraph graph partitioning by tenant ID
- Encryption key isolation: per-tenant DEKs prevent cross-tenant decryption
- Audit log partitioned by tenant — no cross-tenant log visibility
| Layer | Control | Technology |
|---|---|---|
| Edge | WAF, DDoS protection, geo-filtering | Cloudflare Enterprise or AWS WAF |
| Ingress | API gateway with rate limiting, JWT validation | Kong + custom Go gateway |
| Service mesh | mTLS between all services, traffic policies | Istio + SPIFFE |
| Internal network | Network segmentation, east-west filtering | Kubernetes NetworkPolicies + Calico |
| Database | Private subnet only, no public access, VPC peering | AWS PrivateLink / GCP Private Service Connect |
MEERU employs a purpose-built polyglot persistence architecture, selecting the optimal database technology for each specific data access pattern. No single database technology satisfies all intelligence platform requirements — transactional entity storage, analytical query, graph traversal, vector search, caching, and streaming all demand different architectural optimizations.
The rule: use the right database for the right data. Forcing all data into a single database engine creates unacceptable performance and scalability compromises at MEERU's scale.
| Database | Type | Primary Use Cases | Scale Target |
|---|---|---|---|
| PostgreSQL 16 | RDBMS (OLTP) | Entity records, case management, user data, config, investigations | 10B rows, 50TB, 10K TPS |
| ClickHouse | Columnar OLAP | Analytics queries, event history, audit logs, aggregations, reporting | 100PB, 1M inserts/sec |
| Redis 7 Cluster | In-memory KV | Session cache, hot entity cache, rate limiting, pub/sub, leaderboards | 1TB RAM, 1M ops/sec |
| Neo4j 5 | Native graph | Analytical subgraphs, complex Cypher queries, investigation workspace | 50B nodes, 1T edges |
| JanusGraph | Distributed graph | Primary operational graph — full entity relationship graph | 1T+ nodes, 10T+ edges |
| Weaviate | Vector DB | Embedding search, semantic similarity, multimodal search | 10B vectors, 1M QPS |
| OpenSearch 2 | Search engine | Full-text search, log analytics, faceted search, geo search | 10PB index, 1M QPS |
| Object Store | Blob storage | Raw collected content, documents, images, video, audio, cold data | Exabyte-scale |
Partitioning Strategy:
- Entities table: partitioned by
entity_type(Person, Org, Device, etc.) - Events table: range-partitioned by
created_at(monthly partitions) - Audit log: range-partitioned by
ts(weekly partitions), append-only - Cases: tenant-partitioned by
tenant_idfor isolation
High Availability:
- Patroni: streaming replication with automatic failover (< 30s RTO)
- Replication factor: 1 primary + 2 synchronous replicas + 1 async read replica
- PgBouncer: connection pooling — reduces connection overhead by 90%
- pg_cron: automated maintenance (VACUUM, ANALYZE, partition management)
- Backup: WAL-G continuous archiving to S3, PITR to 30 seconds
Cluster Design:
- Sharded cluster: 8-node shards with 3-replica replication
- ReplicatedMergeTree engine for all analytical tables
- Distributed table layer: transparent query routing across shards
- ZooKeeper ensemble (5 nodes) for cluster coordination
- CollapsingMergeTree for event deduplication at ingestion
Key Table Designs:
entity_timeline:(entity_id, ts, attribute, value)— temporal attribute historygraph_edges_history:(edge_id, source, target, type, ts)— relationship historycollection_events:(source, type, raw_id, ts, confidence)— raw collection logaudit_log:(tenant, user_id, action, resource, ts)— immutable audit trail
- Redis Cluster: 6-node cluster (3 primary + 3 replica) with automatic sharding
- Entity cache: hot entity records with 15-minute TTL, LRU eviction
- Session store: JWT refresh token storage with sliding expiry
- Rate limiting: sliding window counters per API key + user
- Pub/Sub: real-time alert distribution to WebSocket connections
- Sorted sets: investigation priority queues, search ranking caches
- HNSW index: Hierarchical Navigable Small World for sub-millisecond ANN search
- Multi-modal classes: TextChunk, EntityEmbedding, FaceEmbedding, ImageEmbedding, VoicePrint
- Hybrid search: BM25 + vector fusion for best-of-both keyword and semantic retrieval
- Cross-reference links to PostgreSQL entity records for joined queries
- Distributed: 8-node Weaviate cluster with consistent hashing shard routing
| Data Type | Hot Storage | Warm Storage | Cold Archive | Deletion |
|---|---|---|---|---|
| Entity records | Indefinite (PostgreSQL) | N/A | ClickHouse history | On legal hold release |
| Collection events | 90 days (ClickHouse) | 1 year (ClickHouse compressed) | Object store (Parquet) | 7 years then purge |
| Audit logs | 1 year (ClickHouse) | 6 years (ClickHouse) | Object store WORM | 7 years mandatory |
| Intelligence reports | Indefinite (PostgreSQL) | N/A | Object store | On case closure + hold |
| Raw collected content | 30 days (object store) | 1 year (object store Glacier) | 7-year archive | Per data classification policy |
MEERU runs on Kubernetes with a multi-cluster, multi-region deployment model. Each environment tier (development, staging, production) runs independent clusters, with production spanning multiple cloud regions in an active-active configuration.
Production deployment: minimum 3 Kubernetes clusters across 3 cloud regions. Each cluster is self-sufficient and can serve 100% of traffic during regional failover.
| Cluster | Region | Purpose | Nodes |
|---|---|---|---|
| prod-us-east-1 | US East (Primary) | Production — primary user-facing cluster | 200+ mixed (CPU + GPU) |
| prod-eu-west-1 | EU West | Production — GDPR-compliant EU data residency | 150+ mixed |
| prod-ap-south-1 | APAC South | Production — APAC low-latency serving | 100+ mixed |
| staging | US East | Pre-production validation, load testing | 50 CPU nodes |
| dev | US East | Developer sandboxes, feature testing | 20 CPU nodes |
| airgapped-gov | On-premises | Government air-gapped deployment | 100+ mixed |
| Node Pool | Instance Type | Workloads |
|---|---|---|
| system | m7i.xlarge (CPU) | Cluster management: CoreDNS, CNI, logging agents |
| api | c7i.8xlarge (CPU) | API gateway, web services, Go/Python microservices |
| data | r7i.16xlarge (Memory) | Database pods, Kafka brokers, Redis clusters |
| ml-inference | g5.12xlarge (A10G GPU) | vLLM, CV models, Weaviate, biometric inference |
| ml-training | p4d.24xlarge (A100 GPU) | GNN training, LLM fine-tuning (spot instances) |
| crawl | c7i.4xlarge (CPU) | Web crawler pods, Playwright headless browsers |
CI/CD Pipeline:
- Git: GitHub Enterprise (self-hosted) with branch protection and required reviews
- CI: GitHub Actions — unit tests, integration tests, security scans, Docker build
- Container scanning: Trivy for CVE scanning on every image build
- SAST: CodeQL + Semgrep for static analysis on every PR
- CD: ArgoCD GitOps — declarative continuous deployment from Git to Kubernetes
- Deployment strategy: Blue-green for stateless services, canary for stateful (5% → 25% → 100%)
- Rollback: Automatic ArgoCD rollback on health check failure (< 2 minute detection)
Infrastructure as Code:
- Terraform: All cloud resources (VPC, EKS/GKE, RDS, S3, IAM)
- Terragrunt: DRY Terraform configurations across environments
- Helm: Kubernetes application packaging and version management
- Helmfile: Declarative Helm release management across clusters
- Vault: Secrets injected at runtime via Agent Sidecar, never stored in Git
- Module registry: Private Terraform module registry for reusable IaC components
| Signal | Collection | Storage | Visualization |
|---|---|---|---|
| Metrics | Prometheus (pull) + Pushgateway | Thanos (long-term, multi-cluster) | Grafana |
| Logs | Fluent Bit (DaemonSet) | Loki (object-store backend) | Grafana Loki |
| Traces | OpenTelemetry SDK + Collector | Jaeger (Cassandra backend) | Jaeger UI + Grafana Tempo |
| Events | Kubernetes event exporter | ClickHouse events table | Grafana |
| Alerts | Prometheus AlertManager | PagerDuty + Slack integration | AlertManager UI |
SLO Dashboard:
- API availability SLO: 99.99% (4 nines) — error budget tracked in Grafana
- Search latency SLO: p99 < 200ms — histogram tracked per endpoint
- Collection freshness SLO: 95% of sources refreshed within 1 hour
- Alert: PagerDuty P1 on SLO burn rate > 5% per hour
- Istio control plane: Istiod in dedicated system namespace
- mTLS: Automatic mutual TLS for all pod-to-pod communication
- Traffic management: VirtualService, DestinationRule for canary routing
- Observability: Envoy proxy metrics automatically collected by Prometheus
- Circuit breaking: Outlier detection + connection pool limits per service
- Rate limiting: Envoy local rate limiting for high-frequency service endpoints
| Scenario | RTO Target | RPO Target | Recovery Process |
|---|---|---|---|
| Single pod failure | < 30 seconds | 0 (stateless) | Kubernetes self-healing |
| Node failure | < 2 minutes | < 30 seconds | Rescheduling + replica promotion |
| AZ failure | < 5 minutes | < 60 seconds | Multi-AZ routing, replica promotion |
| Region failure | < 15 minutes | < 5 minutes | DNS failover to secondary region |
| Full cluster failure | < 30 minutes | < 30 minutes | Cluster restore from IaC + backup |
MEERU's Dark Web Intelligence module provides comprehensive monitoring, indexing, and analysis of Tor hidden services, dark web marketplaces, forums, and leak sites. The system operates a dedicated, isolated collection infrastructure to prevent attribution and ensure operational security.
Dark web collection infrastructure is completely isolated from production systems — separate network, separate credentials, separate identities. No collection traffic traverses production networks.
| Capability | Description | Update Frequency |
|---|---|---|
| Onion indexing | Full-text indexing of discovered .onion services | Continuous (24/7) |
| Marketplace monitoring | Product listings, vendor profiles, pricing, availability | Every 6 hours |
| Forum intelligence | Threat actor posts, discussions, recruitment, tool sharing | Every 2 hours |
| Leak monitoring | Data breach dumps, credential exposure, document leaks | Hourly |
| Credential exposure | Plaintext and hashed credential monitoring against customer databases | Continuous |
| Threat actor tracking | Alias persistence, infrastructure tracking, activity timelines | Real-time alerts |
| Hidden service mapping | Infrastructure relationship mapping (hosting, shared TLS, IPs) | Daily |
Tor Collection Fleet:
- Isolated crawler cluster: 200+ nodes in dedicated air-gapped network segment
- Each crawler node runs a Tor daemon with separate identity (circuit rotation every 10 minutes)
- Playwright + custom Tor SOCKS5 proxy routing for JavaScript-rendered onion sites
- CAPTCHA solving: third-party solver integration + ML-based CAPTCHA classifier
- Session management: cookie and session persistence for authenticated forum access
- User agent rotation: realistic browser fingerprints to avoid bot detection
- Rate limiting: respectful crawl delays to maintain site access and avoid bans
Source Discovery:
- Seed list: curated list of 50,000+ known onion services
- Passive discovery: link extraction from crawled pages
- Active discovery: .onion address generation and probing (v3 onion prediction research)
- Cross-source discovery: references to new onion sites in clear-web forums and paste sites
- Ahmia and Torch index monitoring for newly listed services
Dark Web NLP:
- Language detection: auto-detect — Russian, English, Chinese, Arabic most common
- Threat actor NER: specialized NER models for handles, tools, tactics, targets
- Sentiment analysis: urgency, hostility, operational intent classification
- Drug/weapons classifier: commodity type detection for marketplace listings
- Cryptocurrency extraction: wallet address detection (BTC, ETH, Monero, etc.)
- PII detection: email, phone, SSN, passport, CC number extraction from leaked data
Leak Processing:
- Decompression: auto-detect and decompress ZIP, 7z, RAR, tar archives
- Format parsing: CSV, JSON, SQL dump, plain text credential formats
- PII classification: identify field types (email, password, name, address)
- Hashing: bcrypt, MD5, SHA1, SHA256 hash type detection
- Deduplication: bloom filter deduplication against existing credential database
- Customer alerting: real-time notification if monitored email/domain appears in new leak
Threat Actor Tracking:
- Alias graph: maintain all known handles, accounts, and personas per threat actor
- Activity timeline: chronological post and transaction history
- Infrastructure tracking: hosting providers, shared infrastructure, operational patterns
- Associate mapping: co-posters, co-vendors, shared referral codes
- Cross-platform correlation: same threat actor appearing on multiple forums/markets
- Authenticated crawling: credential vault for authenticated site access
- Database-backed discovery: pastebin, gist, exposed database UIs (Kibana, Grafana)
- API discovery: exposed internal APIs, Swagger/OpenAPI endpoints
- Document discovery: publicly accessible but unlinked PDF, DOCX, XLS files
- Metadata extraction: EXIF, document properties, authorship data from discovered files
- Collection servers: no direct internet access — Tor-only egress
- No PII of collectors stored on collection servers
- Automated credential rotation: collection accounts rotated every 30 days
- Monitoring: MEERU monitors for collection infrastructure exposure on dark web
- Legal review: all dark web collection activities reviewed by legal team quarterly
MEERU's Geospatial Intelligence (GEOINT) module delivers comprehensive location-based intelligence through satellite imagery analysis, real-time aircraft and maritime tracking, vehicle monitoring, route prediction, terrain analysis, and geofencing alerts.
GEOINT data is processed through a spatially-aware pipeline with PostGIS for geometric operations, Deck.gl for WebGL visualization, and MapLibre for tile-based map rendering.
| Capability | Data Source | Refresh Rate | Resolution |
|---|---|---|---|
| Satellite imagery | Maxar, Planet Labs, Sentinel-2 | Daily to sub-daily | 30cm – 10m |
| Aircraft tracking | ADS-B Exchange, OpenSky Network, Flightradar24 | < 10 seconds | Global coverage |
| Maritime tracking | MarineTraffic AIS, ViMS, Global Fishing Watch | < 5 minutes | Global coverage |
| Vehicle tracking | Operator-submitted GPS, road cameras | Real-time | Intersection-level |
| Ground imagery | Street View APIs, operator CCTV feeds | On-demand | Ground-level |
| Weather | NOAA, Copernicus ERA5 | Hourly | Regional |
Imagery Processing Pipeline:
- Imagery acquisition: API-based tasking and retrieval from commercial satellite operators
- Preprocessing: orthorectification, atmospheric correction, pan-sharpening
- Change detection: multi-temporal differencing for activity monitoring
- Object detection: YOLOv9 trained on satellite imagery (vehicles, aircraft, ships, buildings)
- Structure analysis: building footprint extraction, construction monitoring
- Crowd estimation: density estimation from overhead imagery
Satellite Analysis Models:
| Model | Task | Architecture |
|---|---|---|
| SatDetect | Vehicle, aircraft, ship detection from satellite | YOLOv9 fine-tuned on xView dataset |
| ChangeAlert | Activity change detection between image dates | Siamese CNN + threshold classifier |
| BuildingNet | Building footprint and type classification | SegFormer semantic segmentation |
| ThermalNet | Thermal signature analysis (where IR data available) | ResNet50 + custom head |
- ADS-B ingestion: real-time feed from ADS-B Exchange (1M+ messages/minute)
- Aircraft identity: ICAO hex code, registration, operator, route history
- Flight path reconstruction: trajectory smoothing and gap-filling for sparse ADS-B coverage
- Dark aircraft detection: aircraft with ADS-B off, correlated with radar data
- Interesting flight detection: unusual patterns, sensitive airspace violations, restricted area approaches
- Entity linking: aircraft owner → corporate entity → investigation graph
- AIS ingestion: real-time global AIS feed (Marine Traffic, ViMS)
- Vessel identity: MMSI, IMO, flag state, owner, operator, cargo type
- Dark vessel detection: AIS gap analysis for vessels going dark in sensitive areas
- STS detection: ship-to-ship transfer identification (position correlation)
- Port call history: historical arrival/departure database for 10,000+ ports
- Sanctioned vessel tracking: automatic flag when IMO number matches sanctions list
- Polygon geofences: operator-defined regions of interest with entry/exit alerts
- Circular geofences: radius-based alert zones around points of interest
- Entity-linked geofences: alert when tracked person's device enters defined area
- Historical geofence analysis: retrospective query — who was in area X between time A and B?
- Alert delivery: real-time push via WebSocket to analyst dashboard and mobile
| Component | Technology | Purpose |
|---|---|---|
| Spatial DB | PostgreSQL + PostGIS | Geometric operations, spatial indexing, GeoJSON storage |
| Tile server | MapLibre GL / Martin (Rust) | Vector tile serving for web and mobile clients |
| 3D visualization | Deck.gl + Mapbox GL | WebGL-accelerated geospatial rendering |
| Raster processing | GDAL, Rasterio, OpenCV | Satellite imagery preprocessing and analysis |
| Routing | OSRM, Valhalla | Route prediction, travel time estimation |
| Geocoding | Nominatim (OSM), Google Maps API | Address to coordinate conversion and reverse |
| Layer | Technologies |
|---|---|
| Backend Services | Go, Python, Rust |
| Frontend | Next.js, TypeScript, Sigma.js, Graphology, Deck.gl, MapLibre |
| Mobile | Flutter |
| Databases | PostgreSQL, ClickHouse, Redis, Neo4j, JanusGraph, Weaviate |
| Search | OpenSearch (full-text + hybrid) |
| Streaming | Apache Kafka, Apache Flink |
| Workflow Orchestration | Temporal |
| Infrastructure | Kubernetes, Istio, Terraform, ArgoCD, Vault, OpenTelemetry |
| AI/ML | PyTorch, Hugging Face, vLLM, Llama, Mistral, GNN frameworks |
| Security | Zero Trust, RBAC, ABAC, multi-tenancy, E2E encryption, Vault |
| Environment | Description |
|---|---|
| Public Cloud | AWS, GCP, Azure — multi-region active-active |
| Private Cloud | On-premises Kubernetes, VMware, OpenStack |
| Air-Gapped | Fully offline deployment with local model serving |
| Sovereign | Government environments with data residency controls |
| Hybrid | Mixed cloud/on-premises with encrypted federation |
| Metric | Target | Architecture Support |
|---|---|---|
| Entities | 100B+ | Distributed JanusGraph + PostgreSQL partitioned |
| Relationships | Multi-trillion | Graph federation across shards |
| Storage | Petabyte-scale | Object store + columnar ClickHouse |
| Event Throughput | Millions/second | Kafka + Flink streaming pipeline |
| Search QPS | Millions/second | OpenSearch + Weaviate distributed cluster |
| API Latency (p99) | < 200ms | Istio service mesh, edge caching, Redis |
Objectives:
- Operational OSINT collection platform with 10+ source types
- Entity extraction, resolution, and basic graph construction
- Search platform: full-text + basic semantic search
- Investigation workspace: case management, entity timeline, evidence management
- Basic analytics: entity profile, relationship explorer, simple graph visualization
- Security foundation: RBAC, audit logging, mTLS, basic multi-tenancy
- API: REST API with authentication for external integrations
MVP Sprint Plan:
| Sprint | Month | Deliverable |
|---|---|---|
| Sprint 1-2 | Month 1 | Infrastructure: Kubernetes cluster, CI/CD, PostgreSQL, Kafka, Redis, OpenSearch |
| Sprint 3-4 | Month 2 | Collection: Surface web crawler, news ingestor, DNS/WHOIS collector |
| Sprint 5-6 | Month 3 | Intelligence: NER pipeline, entity extraction, basic entity resolution |
| Sprint 7-8 | Month 4 | Graph: Basic entity-relationship graph (Neo4j), graph API |
| Sprint 9-10 | Month 5 | Search + UI: OpenSearch integration, basic Next.js investigation dashboard |
| Sprint 11-12 | Month 6 | Investigation: Case management, evidence management, basic timeline, first customer demo |
MVP Technology Choices:
| Component | MVP Choice | Rationale |
|---|---|---|
| Entity storage | PostgreSQL | Operational simplicity, strong ACID, JSONB for flexible attributes |
| Graph | Neo4j (single node) | Fastest path to working graph UI, upgrade to JanusGraph in Phase 2 |
| Search | OpenSearch (3 nodes) | Full-text + geo search covers MVP needs |
| Vector search | pgvector on PostgreSQL | Avoid Weaviate complexity until Phase 2 |
| AI/NLP | spaCy + cloud LLM API | Avoid GPU infrastructure until Phase 2 |
| Collection | Scrapy + Kafka | Battle-tested, sufficient for MVP scale |
Objectives:
- Dark web and deep web collection modules
- Biometric intelligence: facial recognition + voice identification
- GEOINT: aircraft and maritime tracking, satellite imagery integration
- Cyber threat intelligence: IOC processing, MISP integration, threat actor tracking
- AI investigation layer: LLM-powered analyst co-pilot and report generation
- Weaviate deployment and semantic search upgrade
- JanusGraph migration for hyperscale graph
- Full multi-tenancy, ABAC policies, and government-grade security controls
Enterprise Milestones:
| Milestone | Month | Capability Unlocked |
|---|---|---|
| Dark Web Alpha | Month 9 | Tor crawler live, onion indexing, basic marketplace monitoring |
| Biometric Beta | Month 11 | Face recognition operational, search across 100M+ face index |
| CTI Module v1 | Month 12 | IOC ingestion, MISP sync, threat actor graph active |
| GEOINT v1 | Month 13 | Aircraft + maritime live tracking, geofencing alerts |
| AI Co-Pilot Beta | Month 15 | LLM analyst assistant integrated into investigation workspace |
| Enterprise GA | Month 18 | Full enterprise feature set, SLA contracts, first government customer |
Objectives:
- Multi-trillion edge graph with federated JanusGraph cluster
- GNN platform: link prediction, community detection, anomaly detection at full scale
- Autonomous AI investigators: fully autonomous multi-step investigation agents
- Full biometric suite: gait, iris, fingerprint, and multi-modal fusion
- Satellite imagery analysis pipeline at scale
- Sovereign deployment: air-gapped government deployment packages
- Global federation: cross-deployment graph federation for allied organizations
Hyperscale Milestones:
| Milestone | Month | Capability Unlocked |
|---|---|---|
| Graph Hyperscale | Month 21 | JanusGraph federation: 1T+ edges operational |
| GNN Platform GA | Month 24 | Full GNN model suite in production |
| AI Investigator GA | Month 27 | Autonomous multi-agent investigation pipeline |
| Full Biometrics GA | Month 30 | All 5 modalities + multi-modal fusion operational |
| Sovereign Package | Month 33 | Air-gapped deployment certified for government |
| Hyperscale GA | Month 36 | 100B+ entities, global federation, full platform GA |
| Team | Focus | Phase 1 Size | Phase 2 Size | Phase 3 Size |
|---|---|---|---|---|
| Platform/Infra | K8s, CI/CD, Observability, Security | 4 engineers | 6 engineers | 8 engineers |
| Collection | Crawlers, ingestors, dark web | 4 engineers | 8 engineers | 10 engineers |
| Data/Graph | Databases, graph, entity resolution | 4 engineers | 8 engineers | 12 engineers |
| AI/ML | NLP, CV, GNN, LLM, biometrics | 3 engineers | 10 engineers | 15 engineers |
| Backend API | Go/Python services, API gateway | 4 engineers | 8 engineers | 10 engineers |
| Frontend | Next.js, graph viz, GEOINT map | 3 engineers | 6 engineers | 8 engineers |
| Security | AppSec, compliance, audit | 2 engineers | 4 engineers | 6 engineers |
| Document ID | Title | Classification |
|---|---|---|
| MEERU-ARCH-001 | Executive Architecture Overview | CONFIDENTIAL |
| MEERU-ARCH-002 | System Architecture | CONFIDENTIAL |
| MEERU-ARCH-003 | Intelligence Ontology Architecture | CONFIDENTIAL |
| MEERU-ARCH-004 | Entity Resolution Architecture | CONFIDENTIAL |
| MEERU-ARCH-005 | Graph Intelligence Architecture | CONFIDENTIAL |
| MEERU-ARCH-006 | AI Architecture | CONFIDENTIAL |
| MEERU-ARCH-007 | Biometric Intelligence Architecture | CONFIDENTIAL |
| MEERU-ARCH-008 | Security Architecture | CONFIDENTIAL |
| MEERU-ARCH-009 | Database Architecture | CONFIDENTIAL |
| MEERU-ARCH-010 | Kubernetes & DevOps Architecture | CONFIDENTIAL |
| MEERU-ARCH-011 | Dark Web & Deep Web Architecture | CONFIDENTIAL |
| MEERU-ARCH-012 | GEOINT Architecture | CONFIDENTIAL |
| MEERU-ARCH-013 | MVP, Enterprise & Hyperscale Roadmap | CONFIDENTIAL |
| MEERU-ARCH-README | Comprehensive README | CONFIDENTIAL |
© MEERU Global Intelligence Platform. All rights reserved.
This document contains CONFIDENTIAL information. Distribution is restricted to authorized personnel only. | Biometric Intelligence Architecture | Face, voice, gait, iris, fingerprint & multi-modal fusion | | MEERU-ARCH-008 | Security Architecture | Zero Trust, RBAC/ABAC, encryption, secrets & compliance | | MEERU-ARCH-009 | Database Architecture | Multi-model storage, partitioning, replication & data access patterns | | MEERU-ARCH-010 | Kubernetes & DevOps Architecture | Cluster design, GitOps, CI/CD, observability & infrastructure as code | | MEERU-ARCH-011 | Dark Web & Deep Web Architecture | Tor intelligence, onion indexing, hidden services & threat monitoring | | MEERU-ARCH-012 | GEOINT Architecture | Satellite imagery, aircraft & maritime tracking, geospatial analytics | | MEERU-ARCH-013 | MVP, Enterprise & Hyperscale Roadmap | Phased delivery plan, milestones & engineering priorities |
- Architecture Review Board: architecture@meeru.platform
- Security Team: security@meeru.platform
- DevOps On-Call: devops-oncall@meeru.platform
- Documentation: https://docs.meeru.platform
- Status Page: https://status.meeru.platform
Document compiled from MEERU Architecture Specification v1.0 © 2026 MEERU Platform. All rights reserved.