feat: add Milvus vector index provider

[zc277584121]

Summary

Adds a Milvus-backed external vector index provider for the in-memory metadata store.

This fresh branch replaces #404, which was based on an older storage/vector-index layout and now conflicts with current main.

What changed

• Add VectorIndex and a MilvusVectorIndex implementation using pymilvus.MilvusClient
• Add vector_index.provider="milvus" config with URI, token, collection name, and dimension options
• Wire Milvus into the inmemory metadata backend for create, update, delete, clear, and similarity search
• Reject milvus with non-inmemory metadata stores for now so the configured external index is not silently ignored
• Add docs, an ADR, and a runnable Milvus Lite example
• Cover filter-expression safety, ID escaping, dimension mismatch handling, and Milvus Lite round trips

Validation

• uv run ruff check src/memu/app/settings.py src/memu/database/factory.py src/memu/database/inmemory src/memu/database/vector_index tests/test_milvus_vector_index.py examples/milvus_vector_index.py
• uv run ruff format --check src/memu/app/settings.py src/memu/database/factory.py src/memu/database/inmemory src/memu/database/vector_index tests/test_milvus_vector_index.py examples/milvus_vector_index.py
• uv run mypy src/memu/app/settings.py src/memu/database/factory.py src/memu/database/inmemory src/memu/database/vector_index
• uv run python -m pytest tests/test_milvus_vector_index.py tests/test_inmemory.py tests/test_backend_conformance.py

[Copilot]

Pull request overview

Adds an optional external vector-index abstraction to memU and ships a Milvus-backed implementation, wired into the inmemory metadata backend so similarity search can be offloaded to Milvus / Milvus Lite / Zilliz Cloud while keeping metadata in-memory.

Changes:

• Introduces a VectorIndex protocol plus MilvusVectorIndex (pymilvus MilvusClient) and a small factory to construct it from DatabaseConfig.
• Wires the external index into the inmemory item repository for upsert/delete/clear mirroring and for non-salience similarity search routing.
• Adds a milvus optional dependency group, docs/ADR, an end-to-end example, and Milvus Lite-backed tests.

Reviewed changes

Copilot reviewed 16 out of 17 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
uv.lock	Locks new optional Milvus-related dependencies (pymilvus, milvus-lite, grpcio, etc.).
tests/test_milvus_vector_index.py	Adds Milvus Lite round-trip tests plus filter-expression and dimension mismatch coverage.
src/memu/database/vector_index/milvus.py	Implements MilvusVectorIndex, including safe filter-expression building and collection bootstrap.
src/memu/database/vector_index/interfaces.py	Defines the new VectorIndex protocol contract.
src/memu/database/vector_index/init.py	Adds build_vector_index to construct providers from VectorIndexConfig.
src/memu/database/inmemory/repositories/memory_item_repo.py	Mirrors item mutations to the external index and routes similarity search through it (except salience).
src/memu/database/inmemory/repo.py	Plumbs vector_index into the inmemory store and closes it on shutdown.
src/memu/database/inmemory/init.py	Extends the inmemory database builder to accept and pass through a vector_index.
src/memu/database/factory.py	Constructs Milvus index for inmemory and rejects milvus config for non-inmemory stores.
src/memu/app/settings.py	Extends VectorIndexConfig to include milvus options (uri/token/collection/dim) and defaulting.
README.md	Documents how to run the Milvus Lite example via uv sync --extra milvus.
pyproject.toml	Adds [project.optional-dependencies].milvus and mypy override for pymilvus.*.
examples/milvus_vector_index.py	Adds a runnable MemoryService + Milvus Lite end-to-end example.
docs/milvus.md	Adds a user guide for configuring Milvus Lite/server/Zilliz Cloud.
docs/architecture.md	Updates architecture docs to describe database_config.vector_index provider options.
docs/adr/README.md	Adds the new ADR entry to the ADR index.
docs/adr/0006-external-vector-index.md	Adds ADR documenting the external vector index design and rollout constraints.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[zc277584121]

Addressed the review items in 6d40560:

• Fixed scalar scope extraction by using a runtime-compatible isinstance tuple.
• Added the missing import to the Zilliz Cloud configuration snippet.
• Tightened Milvus filter compatibility for None and __in filters, and mirrored scalar core metadata such as memory_type/resource_id for consistent repository filtering.
• Added optional db_name and consistency_level configuration for Milvus deployments.

Validation:

• uv run ruff check src/memu/app/settings.py src/memu/database/factory.py src/memu/database/inmemory/repositories/memory_item_repo.py src/memu/database/vector_index tests/test_milvus_vector_index.py examples/milvus_vector_index.py
• uv run ruff format --check src/memu/app/settings.py src/memu/database/factory.py src/memu/database/inmemory/repositories/memory_item_repo.py src/memu/database/vector_index tests/test_milvus_vector_index.py examples/milvus_vector_index.py
• uv run mypy src/memu/database/inmemory src/memu/database/vector_index src/memu/app/settings.py src/memu/database/factory.py
• uv run python -m pytest tests/test_milvus_vector_index.py tests/test_inmemory.py tests/test_backend_conformance.py

[zc277584121]

@evan-ak could you take a look when you have a chance? This replaces the stale #404 branch and is ready for review.