-
Notifications
You must be signed in to change notification settings - Fork 15
feat(qdrant): add Qdrant vector database chart #310
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
3ce6741
6525d29
1f18326
32c7f71
339658b
715630a
29efec9
2d33b2e
487a965
a02906e
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| apiVersion: v2 | ||
| # Keep in step with values.yaml `version` (the deployed image tag, which carries | ||
| # a `v` prefix). appVersion is what shows on helm.zop.dev and in `helm list`. | ||
| appVersion: "1.19.0" | ||
| description: High-performance vector database for AI search, recommendations and RAG | ||
| name: qdrant | ||
| version: 0.0.1 | ||
| icon: "https://storage.googleapis.com/zopdev-blog-resources/1/files/originals/20260813/a95c34e7-1888-4a94-8476-0ea91d710f53-qdrant.png" | ||
| maintainers: | ||
| - name: ZopDev | ||
| url: zop.dev | ||
| annotations: | ||
| type: datasource |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,131 @@ | ||
| # Qdrant Helm Chart | ||
|
|
||
| The Qdrant Helm chart provides an easy way to deploy and manage a [Qdrant](https://qdrant.tech/) vector database in your Kubernetes environment. Qdrant is a high-performance vector search engine for AI applications — semantic search, recommendations, and retrieval-augmented generation (RAG). This chart includes persistence, resource management, built-in Prometheus metrics, and API-key authentication. | ||
|
|
||
| > **Single node.** This chart deploys one Qdrant node (`replicas: 1`, distributed mode off) — there is no high-availability or clustering. It is intended as an application datastore rather than a multi-node Qdrant cluster. | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| - Kubernetes 1.19+ | ||
| - Helm 3+ | ||
| - **Prometheus Operator CRDs** (`ServiceMonitor`, `PrometheusRule`) — required, not optional: the chart renders both unconditionally, so `helm install` fails on a cluster without them (`no matches for kind "ServiceMonitor" in version "monitoring.coreos.com/v1"`). | ||
| - [Stakater Reloader](https://github.com/stakater/Reloader) (optional, recommended) — required only for the pod to restart automatically when `customConfig` changes. | ||
|
|
||
| ## Add Helm Repository | ||
|
|
||
| ```bash | ||
| helm repo add zopdev https://helm.zop.dev | ||
| helm repo update | ||
| ``` | ||
|
|
||
| ## Install Helm Chart | ||
|
|
||
| ```bash | ||
| helm install [RELEASE_NAME] zopdev/qdrant | ||
| ``` | ||
|
|
||
| For example: | ||
|
|
||
| ```bash | ||
| helm install my-qdrant zopdev/qdrant | ||
| ``` | ||
|
|
||
| You can customize the installation by providing a custom `values.yaml` file or overriding values via the command line. | ||
|
|
||
| --- | ||
|
|
||
| ## Configuration | ||
|
|
||
| | Parameter | Description | Default | | ||
| |-----------|-------------|---------| | ||
| | `version` | Qdrant image tag (`qdrant/qdrant:<version>`) | `v1.19.0` | | ||
| | `diskSize` | Size of the PVC backing `/qdrant/storage` | `10Gi` | | ||
| | `resources.requests.cpu` | CPU request | `250m` | | ||
| | `resources.requests.memory` | Memory request | `512Mi` | | ||
| | `resources.limits.cpu` | CPU limit | `1000m` | | ||
| | `resources.limits.memory` | Memory limit | `2Gi` | | ||
| | `customConfig` | Extra Qdrant config (YAML), mounted at `/qdrant/config/local.yaml` | `""` | | ||
|
|
||
| --- | ||
|
|
||
| ## Ports | ||
|
|
||
| | Port | Purpose | | ||
| |------|---------| | ||
| | `6333` | REST / HTTP API (also serves `/metrics` and the health endpoints) | | ||
| | `6334` | gRPC API | | ||
|
|
||
| --- | ||
|
|
||
| ## Authentication | ||
|
|
||
| The chart generates a strong API key on install and stores it in a Kubernetes Secret named `<release-name>-qdrant-apikey-secret` under the key `api-key`. Qdrant is started with `QDRANT__SERVICE__API_KEY` set to this value, so all data requests must be authenticated — **including `/metrics`** (401 without the key). Only the health endpoints (`/livez`, `/readyz`, `/healthz`) remain open, so the probes work without the key; the ServiceMonitor authenticates with a Bearer token (see [Monitoring](#monitoring)). | ||
|
|
||
| The API key is **preserved across upgrades** — it is only generated once and re-used on subsequent `helm upgrade` runs. | ||
|
|
||
| ### Retrieving the API key | ||
|
|
||
| ```bash | ||
| kubectl get secret <release-name>-qdrant-apikey-secret \ | ||
| -o jsonpath='{.data.api-key}' | base64 --decode | ||
| ``` | ||
|
|
||
| ### Connecting from an application | ||
|
|
||
| Applications need three things, published in the ConfigMap `<release-name>-qdrant-configmap`: | ||
|
|
||
| | Key | Value | | ||
| |-----|-------| | ||
| | `QDRANT_HOST` | `<release-name>-qdrant` | | ||
| | `QDRANT_HTTP_PORT` | `6333` | | ||
| | `QDRANT_GRPC_PORT` | `6334` | | ||
|
|
||
| Pass the API key in the `api-key` request header (or `Authorization: Bearer <key>`): | ||
|
|
||
| ```bash | ||
| curl -H "api-key: <key>" http://<release-name>-qdrant:6333/collections | ||
| ``` | ||
|
|
||
| Qdrant has no databases or users — applications create **collections** at runtime through the REST or gRPC API, so no per-database provisioning is required. | ||
|
|
||
| --- | ||
|
|
||
| ## Custom configuration | ||
|
|
||
| Provide extra Qdrant settings as a YAML document via `customConfig`. It is mounted at `/qdrant/config/local.yaml`, which Qdrant layers on top of its defaults: | ||
|
|
||
| ```yaml | ||
| customConfig: | | ||
| log_level: INFO | ||
| storage: | ||
| optimizers: | ||
| default_segment_number: 4 | ||
| ``` | ||
|
|
||
| > Settings injected by the chart through `QDRANT__*` environment variables (such as the API key) always take precedence over `customConfig`. | ||
|
|
||
| --- | ||
|
|
||
| ## Monitoring | ||
|
|
||
| Qdrant exposes a built-in Prometheus endpoint on the HTTP port (`6333`) at `/metrics` — no exporter sidecar is required. When the API key is enabled, `/metrics` requires authentication (only the health endpoints stay open), so the shipped `ServiceMonitor` scrapes it with the key as a Bearer token, read from the generated Secret. The chart also ships a `PrometheusRule` with two alerts: | ||
|
|
||
| - **`QdrantDown`** (critical) — no metric scrapes succeed (`up == 0`). | ||
| - **`QdrantInRecoveryMode`** (critical) — the node booted into recovery mode, typically after an OOM. This is a **silent-degradation** state worth understanding before you meet it in an incident: the pod reports `Ready` and stays in the Service (so `QdrantDown` does **not** fire), but every read and write against existing collections fails. Recovery mode is Qdrant's designed remedy — with the pod reachable, drop a collection to get back under the memory limit, or raise `resources.limits.memory`. | ||
|
|
||
| --- | ||
|
|
||
| ## Uninstall | ||
|
|
||
| ```bash | ||
| helm uninstall [RELEASE_NAME] | ||
| ``` | ||
|
|
||
| `helm uninstall` **deletes the API-key Secret but keeps the PersistentVolumeClaim** created by the StatefulSet. Two consequences: | ||
|
|
||
| - Reinstalling against the retained volume comes up with a **new** API key (the old data is intact, but any client still using the old key must be updated). | ||
| - If you want to reclaim the storage, delete the PVC manually: | ||
|
|
||
| ```bash | ||
| kubectl delete pvc -l app=<release-name>-qdrant -n <namespace> | ||
| ``` | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,27 @@ | ||
| Qdrant is installed as release {{ .Release.Name }} in namespace {{ .Release.Namespace }}. | ||
|
|
||
| ──────────────────────────────────────────────────────────────── | ||
| API KEY (generated for you, required on every data request) | ||
|
|
||
| kubectl get secret {{ include "qdrant.secretName" . }} \ | ||
| -n {{ .Release.Namespace }} \ | ||
| -o jsonpath='{.data.api-key}' | base64 --decode | ||
|
|
||
| Pass it in the "api-key" header (or "Authorization: Bearer <key>"). | ||
| Only the health endpoints (/livez, /readyz, /healthz) are open — every | ||
| other request, including /metrics, is rejected without the key. | ||
| ──────────────────────────────────────────────────────────────── | ||
|
|
||
| Connect from another pod in the cluster (values also in ConfigMap | ||
| {{ .Release.Name }}-qdrant-configmap): | ||
|
|
||
| REST : http://{{ .Release.Name }}-qdrant.{{ .Release.Namespace }}.svc:6333 | ||
| gRPC : {{ .Release.Name }}-qdrant.{{ .Release.Namespace }}.svc:6334 | ||
|
|
||
| Quick check (from a pod that has the key in $KEY): | ||
|
|
||
| curl -H "api-key: $KEY" http://{{ .Release.Name }}-qdrant:6333/collections | ||
|
|
||
| Note: `helm uninstall {{ .Release.Name }}` deletes the API-key Secret but keeps | ||
| the PersistentVolumeClaim, so a reinstall against the retained volume comes up | ||
| with a new key. Delete the PVC manually to reclaim storage. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,22 @@ | ||
| {{/* | ||
| Fully qualified app name: <release>-qdrant, capped at 63 chars for DNS. | ||
| */}} | ||
| {{- define "qdrant.fullname" -}} | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Note — this helper is defined and then never used; every template hardcodes Worth wiring up, because using it in The helper set is also missing |
||
| {{- printf "%s-qdrant" .Release.Name | trunc 63 | trimSuffix "-" -}} | ||
| {{- end -}} | ||
|
|
||
| {{/* | ||
| Name of the Secret holding the generated API key. | ||
| */}} | ||
| {{- define "qdrant.secretName" -}} | ||
| {{- printf "%s-qdrant-apikey-secret" .Release.Name | trunc 63 | trimSuffix "-" -}} | ||
| {{- end -}} | ||
|
|
||
| {{/* | ||
| Selector labels. This feeds the StatefulSet's immutable spec.selector, so it is | ||
| kept to the single flat `app` label the repo's datastore charts use — no | ||
| app.kubernetes.io/* keys mixed in. | ||
| */}} | ||
| {{- define "qdrant.selectorLabels" -}} | ||
| app: {{ .Release.Name }}-qdrant | ||
| {{- end -}} | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,49 @@ | ||
| apiVersion: monitoring.coreos.com/v1 | ||
| kind: PrometheusRule | ||
| metadata: | ||
| # Named after the chart (not the release) so it does not collide with the | ||
| # PrometheusRule of sibling datastore charts pulled into the same umbrella | ||
| # release (postgres, redis, mysql, ...), which would fail the install. | ||
| name: {{ include "qdrant.fullname" . }} | ||
| namespace: {{ .Release.Namespace }} | ||
| generation: 1 | ||
| labels: | ||
| app: kube-prometheus-stack | ||
| heritage: Helm | ||
| release: prometheus | ||
| spec: | ||
| groups: | ||
| - name: {{ .Release.Namespace }}.{{ .Release.Name }}-qdrant.rules | ||
| rules: | ||
| - alert: QdrantDown | ||
| expr: up{namespace="{{ .Release.Namespace }}", service="{{ .Release.Name }}-qdrant"} == 0 | ||
| # 2m avoids a false page during the short not-ready window of a normal | ||
| # rolling upgrade. | ||
| for: 2m | ||
| labels: | ||
| severity: critical | ||
| servicealert: "true" | ||
| namespace: {{ .Release.Namespace }} | ||
| service: {{ .Release.Name }}-qdrant | ||
| annotations: | ||
| summary: 'Qdrant instance {{ .Release.Name }} is down' | ||
| description: 'Qdrant instance {{ .Release.Name }} is not responding to metric scrapes' | ||
|
|
||
| - alert: QdrantInRecoveryMode | ||
| # app_status_recovery_mode is a real gauge from Qdrant's /metrics; it | ||
| # flips to 1 when the node boots into recovery (e.g. after an OOM or | ||
| # WAL corruption). Critical, not warning: in recovery mode the pod | ||
| # reports Ready and stays in the Service, but every read and write | ||
| # against existing collections fails — an up-but-broken state, which | ||
| # this repo pages as critical (cf. KafkaOfflinePartitions, | ||
| # MariaDBReplicationNotRunning). QdrantDown does NOT fire here (up == 1). | ||
| expr: app_status_recovery_mode{namespace="{{ .Release.Namespace }}", service="{{ .Release.Name }}-qdrant"} == 1 | ||
| for: 2m | ||
| labels: | ||
| severity: critical | ||
| servicealert: "true" | ||
| namespace: {{ .Release.Namespace }} | ||
| service: {{ .Release.Name }}-qdrant | ||
| annotations: | ||
| summary: 'Qdrant instance {{ .Release.Name }} is in recovery mode' | ||
| description: 'Qdrant instance {{ .Release.Name }} booted into recovery mode. It reports Ready and stays in the Service, but every read and write against existing collections fails. Drop a collection to recover, or raise the memory limit.' |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| apiVersion: v1 | ||
| kind: ConfigMap | ||
| metadata: | ||
| name: {{ .Release.Name }}-qdrant-configmap | ||
| namespace: {{ .Release.Namespace }} | ||
| labels: | ||
| app: {{ .Release.Name }}-qdrant | ||
| data: | ||
| # Connection details for applications. The matching API key is stored in the | ||
| # Secret {{ include "qdrant.secretName" . }} under the "api-key" field; pass it | ||
| # to Qdrant via the "api-key" request header (or "Authorization: Bearer <key>"). | ||
| QDRANT_HOST: "{{ .Release.Name }}-qdrant" | ||
| QDRANT_HTTP_PORT: "6333" | ||
| QDRANT_GRPC_PORT: "6334" |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| {{- if .Values.customConfig }} | ||
| apiVersion: v1 | ||
| kind: ConfigMap | ||
| metadata: | ||
| name: {{ .Release.Name }}-qdrant-custom-config | ||
| namespace: {{ .Release.Namespace }} | ||
| data: | ||
| local.yaml: |- | ||
| {{- .Values.customConfig | nindent 4 }} | ||
| {{- end }} |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,24 @@ | ||
| {{- $secretName := include "qdrant.secretName" . }} | ||
|
|
||
| {{/* Generate a strong API key only if one is not already stored in the cluster. */}} | ||
| {{- $generatedKey := printf "%s%s%s" (randAlpha 8) (randNumeric 8) (randAlpha 8 | upper) }} | ||
| {{- $apiKey := $generatedKey }} | ||
|
|
||
| {{- $secret := lookup "v1" "Secret" .Release.Namespace $secretName }} | ||
| --- | ||
| apiVersion: v1 | ||
| kind: Secret | ||
| metadata: | ||
| name: {{ $secretName }} | ||
| namespace: {{ .Release.Namespace }} | ||
| type: Opaque | ||
| data: | ||
| {{/* Install: generate fresh. Upgrade: re-emit the stored key so it never | ||
| rotates out from under connected clients. Fallback: generate. */}} | ||
| {{- if .Release.IsInstall }} | ||
| api-key: {{ $apiKey | b64enc }} | ||
| {{- else if $secret }} | ||
| api-key: {{ index $secret.data "api-key" }} | ||
| {{- else }} | ||
| api-key: {{ $apiKey | b64enc }} | ||
| {{- end }} |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,27 @@ | ||
| apiVersion: v1 | ||
| kind: Service | ||
| metadata: | ||
| name: {{ .Release.Name }}-qdrant-headless | ||
| namespace: {{ .Release.Namespace }} | ||
| labels: | ||
| # Distinct from the ClusterIP Service's label so the ServiceMonitor selects | ||
| # only that one — otherwise every metric is scraped twice, and the not-ready | ||
| # pod exposed here (publishNotReadyAddresses) would flap QdrantDown on every | ||
| # rollout. The pod selector below is unchanged, so DNS identity still works. | ||
| app: {{ .Release.Name }}-qdrant-headless | ||
| spec: | ||
| # Headless service backing the StatefulSet — gives the pod a stable DNS name | ||
| # and is required for correct per-pod network identity. | ||
| clusterIP: None | ||
| publishNotReadyAddresses: true | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Correct for a StatefulSet — but combined with the ServiceMonitor selector it has a side effect worth knowing about. Because this Service carries the same See the ServiceMonitor comment; changing the label here is the cleaner of the two fixes. |
||
| ports: | ||
| - port: 6333 | ||
| targetPort: 6333 | ||
| protocol: TCP | ||
| name: http | ||
| - port: 6334 | ||
| targetPort: 6334 | ||
| protocol: TCP | ||
| name: grpc | ||
| selector: | ||
| app: {{ .Release.Name }}-qdrant | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,22 @@ | ||
| apiVersion: v1 | ||
| kind: Service | ||
| metadata: | ||
| name: {{ .Release.Name }}-qdrant | ||
| namespace: {{ .Release.Namespace }} | ||
| labels: | ||
| app: {{ .Release.Name }}-qdrant | ||
| spec: | ||
| ports: | ||
| # The HTTP port (6333) also serves /metrics and the health endpoints, so the | ||
| # ServiceMonitor scrapes this same named port — no separate metrics port. | ||
| - port: 6333 | ||
| targetPort: 6333 | ||
| protocol: TCP | ||
| name: http | ||
| - port: 6334 | ||
| targetPort: 6334 | ||
| protocol: TCP | ||
| name: grpc | ||
| selector: | ||
| app: {{ .Release.Name }}-qdrant | ||
| type: ClusterIP |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,24 @@ | ||
| apiVersion: monitoring.coreos.com/v1 | ||
| kind: ServiceMonitor | ||
| metadata: | ||
| name: {{ .Release.Name }}-qdrant | ||
| labels: | ||
| app: {{ .Release.Name }}-qdrant | ||
| release: prometheus | ||
| spec: | ||
| selector: | ||
| matchLabels: | ||
| app: {{ .Release.Name }}-qdrant | ||
|
Comment on lines
+9
to
+11
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Should fix — this selector matches both Services, so every metric is scraped twice.
Two consequences:
Fix: give the headless Service a distinguishing label (e.g. |
||
| endpoints: | ||
| # Qdrant exposes a built-in Prometheus endpoint on the HTTP port (6333) at | ||
| # /metrics — no exporter sidecar. With an API key set, /metrics requires | ||
| # authentication (only the health endpoints stay open), so the scrape sends | ||
| # the key as a Bearer token from the generated Secret. | ||
| - port: http | ||
| interval: 30s | ||
| path: /metrics | ||
| authorization: | ||
| type: Bearer | ||
| credentials: | ||
| name: {{ include "qdrant.secretName" . }} | ||
| key: api-key | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Blocking — the Prometheus Operator CRDs are a hard prerequisite and are not listed.
alerts.yamlandserviceMonitor.yamlare ungated, so every install emits aServiceMonitorand aPrometheusRule. On a cluster without kube-prometheus-stack,helm installfails with:Ungating matches the datastore family (
postgres,mysql,clickhouse,kafkaare all unconditional), so I am not asking for ametrics.enabledflag — just state the requirement so nobody walks into a failed install:Caveat: reasoned from the templates, not verified — the minikube I tested on already had the CRDs installed.