Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,7 @@ Below is a list of available charts along with their links:
| **JupyterHub** | [helm.zop.dev/jupyterhub](https://helm.zop.dev/src/readme.html?id=jupyterhub) | <a href="https://zop.dev/zopday/app/deploy?install=jupyterhub"><img src="https://zop.dev/deploytozopday-inkhard.svg" alt="Deploy to Zopday" height="28"></a> |
| **LiteLLM** | [helm.zop.dev/litellm](https://helm.zop.dev/src/readme.html?id=litellm) | <a href="https://zop.dev/zopday/app/deploy?install=litellm"><img src="https://zop.dev/deploytozopday-inkhard.svg" alt="Deploy to Zopday" height="28"></a> |
| **LocalAI** | [helm.zop.dev/localai](https://helm.zop.dev/src/readme.html?id=localai) | <a href="https://zop.dev/zopday/app/deploy?install=localai"><img src="https://zop.dev/deploytozopday-inkhard.svg" alt="Deploy to Zopday" height="28"></a> |
| **Ollama** | [helm.zop.dev/ollama](https://helm.zop.dev/src/readme.html?id=ollama) | <a href="https://zop.dev/zopday/app/deploy?install=ollama"><img src="https://zop.dev/deploytozopday-inkhard.svg" alt="Deploy to Zopday" height="28"></a> |
| **Outline** | [helm.zop.dev/outline](https://helm.zop.dev/src/readme.html?id=outline) | <a href="https://zop.dev/zopday/app/deploy?install=outline"><img src="https://zop.dev/deploytozopday-inkhard.svg" alt="Deploy to Zopday" height="28"></a> |
| **Superset** | [helm.zop.dev/superset](https://helm.zop.dev/src/readme.html?id=superset) | <a href="https://zop.dev/zopday/app/deploy?install=superset"><img src="https://zop.dev/deploytozopday-inkhard.svg" alt="Deploy to Zopday" height="28"></a> |
| **WordPress** | [helm.zop.dev/wordpress](https://helm.zop.dev/src/readme.html?id=wordpress) | <a href="https://zop.dev/zopday/app/deploy?install=wordpress"><img src="https://zop.dev/deploytozopday-inkhard.svg" alt="Deploy to Zopday" height="28"></a> |
Expand Down
12 changes: 12 additions & 0 deletions charts/ollama/Chart.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
apiVersion: v2
appVersion: "0.32.9"
description: Helm chart for deploying Ollama, a self-hosted LLM inference server
name: ollama
version: 0.0.1
type: application
icon: "https://storage.googleapis.com/zopdev-blog-resources/1/files/originals/20260818/c8e9597d-0ced-4880-9335-f4f8a78f9f31-ollama.png"
maintainers:
- name: ZopDev
url: zop.dev
annotations:
type: application
211 changes: 211 additions & 0 deletions charts/ollama/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,211 @@
# Ollama Helm Chart

[Ollama](https://github.com/ollama/ollama) runs LLMs (Llama 3, Mistral,
etc.) locally and serves them over a REST API. This chart deploys a
single Ollama server backed by a persistent volume for pulled models.

---

## Prerequisites

- Kubernetes 1.19+
- Helm 3+

---

## Add Helm Repository

```bash
helm repo add zopdev https://helm.zop.dev
helm repo update
```

---

## Install Helm Chart

```bash
helm install my-ollama zopdev/ollama
```

To pull one or more models automatically on first start:

```bash
helm install my-ollama zopdev/ollama \
--set models={llama3,mistral}
```

Without `models` set, the server comes up with no models installed —
pull one afterwards with:

```bash
kubectl exec my-ollama-ollama-0 -- ollama pull llama3
```

---

## Uninstall Helm Chart

```bash
helm uninstall my-ollama
```

The PersistentVolumeClaim created from the volume template outlives the
release. Delete it separately to reclaim the disk:

```bash
kubectl delete pvc models-my-ollama-ollama-0
```

---

## Configuration

| **Input** | **Type** | **Description** | **Default** |
|---|---|---|---|
| `image.repository` | `string` | Container image. | `ollama/ollama` |
| `image.tag` | `string` | Image tag. | `0.32.9` |
| `image.pullPolicy` | `string` | Image pull policy. | `IfNotPresent` |
| `service.type` | `string` | Service type for the API. | `ClusterIP` |
| `service.port` | `int` | Service port for the API. | `11434` |
| `diskSize` | `string` | Size of the models volume. | `"20Gi"` |
| `resources` | `object` | CPU and memory for the pod. | 500m/2Gi – 2000m/4Gi |
| `models` | `list` | Models to pull automatically on first start. See *Model pulling* below. | `[]` |
| `nodeSelector` | `object` | Node labels to constrain scheduling. | `{}` |
| `tolerations` | `list` | Tolerations to allow scheduling onto tainted nodes. | `[]` |
| `env` | `object` | Extra environment variables. | `{}` |
| `ingress.enabled` | `bool` | Create an Ingress for the API. | `false` |
| `ingress.className` | `string` | IngressClass name. | `""` |
| `ingress.host` | `string` | Hostname. Required when the ingress is enabled. | `""` |
| `ingress.annotations` | `object` | Ingress annotations. | `{}` |
| `ingress.tlsSecretName` | `string` | Existing TLS secret for the host. | `""` |

### Model pulling

`models` is a plain list of model names, e.g. `["llama3", "mistral"]`.
When it's non-empty, an init container starts a throwaway Ollama server
against the same volume the main container uses, pulls each model in
order, then exits — the main container only starts once every model is
already on disk. The init container fails (rather than reporting
success) if the server doesn't come up within 60s, or if any model
name is invalid or fails to pull — so a typo in `models` shows up as a
failed release instead of a healthy-looking pod silently serving no
model. Pulling several large models can still take a while; only the
server's own startup is time-bounded, not the pull itself, so a slow
but working pull is not killed.

Changing `models` on an existing release and running `helm upgrade`
pulls any newly-added models but does not remove ones taken off the
list — they stay on the volume until deleted manually.

### Running on GPU nodes

This chart adds no Ollama-specific GPU field. Target a GPU node pool the
same way as any other workload, through the generic `nodeSelector` and
`tolerations` passthrough, and request the GPU itself through
`resources.limits`:

```bash
helm install my-ollama zopdev/ollama \
--set resources.limits."nvidia\.com/gpu"=1 \
--set nodeSelector.cloud\\.google\\.com/gke-accelerator=nvidia-tesla-t4
```

Requires the appropriate device plugin (e.g. NVIDIA's) already installed
on the cluster.

### Security

Ollama has no built-in authentication — upstream ships no auth
mechanism at all, so `GET /api/tags` and every other endpoint answer
any caller with no credentials. That's fine reachable only inside the
cluster, but turning on `ingress.enabled` publishes an inference API
(and `POST /api/pull`, which downloads arbitrary models onto the
volume) to anyone who can resolve the host. Pair an ingress with
authentication at the proxy/gateway layer, or restrict it to trusted
networks.

The container also runs as root, matching the upstream image's own
default — `/root/.ollama` is where it expects to write, so this is a
deliberate fit rather than an oversight.

---

## Example `values.yaml`

```yaml
models:
- llama3
- mistral

resources:
requests:
cpu: "2000m"
memory: "8Gi"
limits:
cpu: "4000m"
memory: "16Gi"

diskSize: "50Gi"

ingress:
enabled: true
className: nginx
host: ollama.example.com
```

```bash
helm install my-ollama zopdev/ollama -f values.yaml
```

---

## Features

- Persistent model storage, so pulled models survive pod restarts
- Optional automatic model pulling on first start via `models`
- Generic `nodeSelector`/`tolerations` passthrough for targeting GPU nodes
- Ingress with optional TLS
- Configurations that cannot work (a bad ingress setup) are rejected when
the chart renders, naming the cause, rather than installing and
crash-looping

---

## Connection Config

The API is served on `service.port` (11434) of the `<release>-ollama`
service.

```bash
kubectl port-forward svc/my-ollama-ollama 11434:11434
curl http://localhost:11434/api/generate \
-d '{"model": "llama3", "prompt": "hello"}'
```

- **`/api/generate`**, **`/api/chat`**, **`/api/embeddings`** — inference endpoints.
- **`/api/pull`** — pull a model over the API instead of `kubectl exec`.
- **`/api/tags`** — list models currently pulled.
- **`/`** — used for readiness/liveness; returns `Ollama is running`.

---

## Contributing

We welcome contributions to improve this Helm chart. Please refer to the
[CONTRIBUTING.md](../../CONTRIBUTING.md) file for contribution
guidelines.

---

## Code of Conduct

To maintain a healthy and collaborative community, please adhere to our
[Code of Conduct](../../CODE_OF_CONDUCT.md).

---

## License

This project is licensed under the [LICENSE](../../LICENSE). Please
review it for terms of use.
27 changes: 27 additions & 0 deletions charts/ollama/templates/NOTES.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
Ollama is installed as release {{ .Release.Name }}.

Reach the API:

kubectl port-forward -n {{ .Release.Namespace }} \
svc/{{ include "ollama.fullname" . }} 11434:{{ .Values.service.port }}

curl http://localhost:11434/api/generate \
-d '{"model": "<model>", "prompt": "hello"}'
{{- if .Values.ingress.enabled }}

Or over the ingress at http{{ if .Values.ingress.tlsSecretName }}s{{ end }}://{{ .Values.ingress.host }}
{{- end }}

{{- if .Values.models }}

These models were pulled automatically on first start:
{{- range .Values.models }}
- {{ . }}
{{- end }}
{{- else }}

No models are installed by default. Pull one after install:

kubectl exec -n {{ .Release.Namespace }} \
{{ include "ollama.fullname" . }}-0 -- ollama pull llama3
{{- end }}
38 changes: 38 additions & 0 deletions charts/ollama/templates/_helpers.tpl
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
{{/*
Expand the name of the chart.
*/}}
{{- define "ollama.name" -}}
{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }}
{{- end }}

{{/*
Fully qualified app name, used for the Service and workload names.
*/}}
{{- define "ollama.fullname" -}}
{{- if .Values.fullnameOverride }}
{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }}
{{- else }}
{{- printf "%s-ollama" .Release.Name | trunc 63 | trimSuffix "-" }}
{{- end }}
{{- end }}

{{- define "ollama.chart" -}}
{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }}
{{- end }}

{{/*
Common labels.
*/}}
{{- define "ollama.labels" -}}
helm.sh/chart: {{ include "ollama.chart" . }}
{{ include "ollama.selectorLabels" . }}
{{- if .Chart.AppVersion }}
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
{{- end }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
{{- end }}

{{- define "ollama.selectorLabels" -}}
app.kubernetes.io/name: {{ include "ollama.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
{{- end }}
23 changes: 23 additions & 0 deletions charts/ollama/templates/headless-service.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
{{/*
Governing service for the StatefulSet. A StatefulSet's serviceName is
meant to name a headless service -- that is what gives the pod a stable
DNS record. The field is immutable once the object exists, so it is far
cheaper to get right before release than after.
*/}}
apiVersion: v1
kind: Service
metadata:
name: {{ include "ollama.fullname" . }}-headless
labels:
{{- include "ollama.labels" . | nindent 4 }}
spec:
type: ClusterIP
clusterIP: None
publishNotReadyAddresses: true
ports:
- port: {{ .Values.service.port }}
targetPort: http
protocol: TCP
name: http
selector:
{{- include "ollama.selectorLabels" . | nindent 4 }}
36 changes: 36 additions & 0 deletions charts/ollama/templates/ingress.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
{{- if .Values.ingress.enabled }}
{{- if not .Values.ingress.host }}
{{- fail "ingress.enabled requires ingress.host" }}
{{- end }}
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: {{ include "ollama.fullname" . }}
labels:
{{- include "ollama.labels" . | nindent 4 }}
{{- with .Values.ingress.annotations }}
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}
spec:
{{- if .Values.ingress.className }}
ingressClassName: {{ .Values.ingress.className }}
{{- end }}
{{- if .Values.ingress.tlsSecretName }}
tls:
- hosts:
- {{ .Values.ingress.host | quote }}
secretName: {{ .Values.ingress.tlsSecretName }}
{{- end }}
rules:
- host: {{ .Values.ingress.host | quote }}
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: {{ include "ollama.fullname" . }}
port:
number: {{ .Values.service.port }}
{{- end }}
15 changes: 15 additions & 0 deletions charts/ollama/templates/service.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
apiVersion: v1
kind: Service
metadata:
name: {{ include "ollama.fullname" . }}
labels:
{{- include "ollama.labels" . | nindent 4 }}
spec:
type: {{ .Values.service.type }}
ports:
- port: {{ .Values.service.port }}
targetPort: http
protocol: TCP
name: http
selector:
{{- include "ollama.selectorLabels" . | nindent 4 }}
Loading
Loading