feat(karpenter-gcp): update chart to upstream v0.6.0

charts/karpenter-gcp/Chart.yaml

@@ -1,8 +1,8 @@
 apiVersion: v2
-appVersion: "1.0"
+appVersion: "v0.6.0"
 description: A Helm chart for deploying Karpenter in gke cluster
 name: karpenter-gcp
-version: 0.0.3
+version: 0.0.4
 maintainers:
   - name: ZopDev
-    url: zop.dev
+    url: zop.dev

charts/karpenter-gcp/README.md

@@ -6,17 +6,40 @@ This Helm chart deploys Karpenter on a GKE (Google Kubernetes Engine) cluster us
 
 ## Prerequisites
 
-- Kubernetes v1.23+
+- Kubernetes v1.26+
 - Helm 3.0+
 - Required GCP APIs enabled:
     - `compute.googleapis.com`
     - `container.googleapis.com`
-- A GCP service account with the following roles:
-    - Compute Admin
-    - Kubernetes Engine Admin
-    - Monitoring Admin
-    - Service Account User
-- A Kubernetes secret containing the GCP service account key (JSON)
+- A GCP service account for the controller, granted a **minimal custom IAM role** rather than
+  the broad predefined roles. Upstream publishes the role definition:
+
+  ```bash
+  export PROJECT_ID=<your-project-id>
+  export GSA_NAME=karpenter-gsa
+
+  curl -fsSL https://raw.githubusercontent.com/cloudpilot-ai/karpenter-provider-gcp/main/deploy/iam/karpenter-controller-role.yaml \
+      -o karpenter-controller-role.yaml
+  gcloud iam roles create karpenter_controller --project=$PROJECT_ID --file=karpenter-controller-role.yaml 2>/dev/null || \
+  gcloud iam roles update karpenter_controller --project=$PROJECT_ID --file=karpenter-controller-role.yaml
+
+  gcloud projects add-iam-policy-binding $PROJECT_ID \
+      --member="serviceAccount:$GSA_NAME@$PROJECT_ID.iam.gserviceaccount.com" \
+      --role="projects/$PROJECT_ID/roles/karpenter_controller"
+  ```
+
+  Grant `roles/iam.serviceAccountUser` **scoped to the node service account** the controller
+  attaches to VMs, not project-wide:
+
+  ```bash
+  export NODE_SA_EMAIL=<your-node-sa>@$PROJECT_ID.iam.gserviceaccount.com
+  gcloud iam service-accounts add-iam-policy-binding $NODE_SA_EMAIL \
+      --role roles/iam.serviceAccountUser \
+      --member "serviceAccount:$GSA_NAME@$PROJECT_ID.iam.gserviceaccount.com" \
+      --project $PROJECT_ID
+  ```
+- A Kubernetes secret containing the GCP service account key (JSON), unless you use Workload
+  Identity — in which case annotate the chart's service account and set `credentials.enabled=false`.
 
 ---
 
@@ -101,10 +124,20 @@ metadata:
   name: default-example
 spec:
   serviceAccount: "<service_account_email_created_before>"
+  # Follow the GKE release channel the cluster is enrolled in. Pin instead with
+  # `version: "125.19216.104.126"` if you want to control when image updates roll out.
   imageSelectorTerms:
-    - alias: ContainerOptimizedOS@latest
-  tags:
+    - family: ContainerOptimizedOS
+      channel: cluster
+  labels:
     env: dev
+  # Required in practice. The CRD does not mark `disks` as required and has no
+  # default, but GCE rejects every instance create without it:
+  #   Error 400: Invalid value for field 'resource.disks': ''. No disks are specified.
+  disks:
+    - boot: true
+      category: pd-balanced
+      sizeGiB: 100
 ```
 #### NodePool
 
@@ -125,9 +158,10 @@ spec:
         - key: "karpenter.sh/capacity-type"
           operator: In
           values: ["on-demand", "spot"]
+        # Machine type prefix only -- "n4-standard" no longer matches.
         - key: "karpenter.k8s.gcp/instance-family"
           operator: In
-          values: ["n4-standard", "n2-standard", "e2"]
+          values: ["n4", "n2", "e2"]
         - key: "kubernetes.io/arch"
           operator: In
           values: ["amd64"]
@@ -136,6 +170,67 @@ spec:
           values: ["us-central1-c", "us-central1-a", "us-central1-f", "us-central1-b"]
 ```
 
+---
+
+## Upgrading from chart 0.0.3
+
+Chart 0.0.4 moves the controller from Karpenter GCP v0.0.1 to v0.6.0. That spans several
+upstream breaking changes, so read this before upgrading a live cluster. Upstream's
+[MIGRATION.md](https://github.com/cloudpilot-ai/karpenter-provider-gcp/blob/main/MIGRATION.md)
+has the full detail.
+
+**Apply the CRDs by hand first.** Helm installs everything under `crds/` on first install and
+never touches it again on upgrade. This release changes all three existing CRDs and adds
+NodeOverlay, so apply them yourself before `helm upgrade`:
+
+```bash
+helm pull zopdev/karpenter-gcp --untar
+kubectl apply -f karpenter-gcp/crds/
+```
+
+**Values renamed.** `controller.settings.location` is now `controller.settings.clusterLocation`,
+with an optional `controller.settings.nodeLocation` for the exact GKE API location. Nothing
+carries over automatically — a stale `location` key is silently ignored and the controller
+starts with no location.
+
+**NodePool `instance-family` values.** Requirements must use the machine type prefix, not
+family plus shape. `n4-standard` no longer matches anything; use `n4`. Audit every NodePool
+before upgrading or Karpenter stops finding instance types.
+
+**GCENodeClass `tags` is now `labels`.** Rename the field on existing node classes.
+
+**`imageSelectorTerms[].alias` is deprecated** in favour of structured `family` with `channel`
+or `version`. Existing aliases keep working, but migrate them.
+
+**Instance labels removed.** `karpenter.k8s.gcp/instance-category` and
+`karpenter.k8s.gcp/instance-cpu-model` no longer exist. Remove them from NodePool
+requirements and workload scheduling constraints.
+
+**Reserved `GCENodeClass.spec.metadata` keys are rejected**, including `kube-env`,
+`startup-script`, `cluster-name`, and others GKE bootstrap owns. Audit existing node classes.
+
+**`kubeletConfiguration` fields now take effect.** Fields the CRD previously accepted and
+dropped — `systemReserved`, `kubeReserved`, the `eviction*` family, and others — are now
+applied. Values already set will change node behaviour on the next provision.
+
+**IAM.** Replace the broad `roles/compute.admin` and `roles/container.admin` bindings with
+the minimal custom role in [Prerequisites](#prerequisites), and scope
+`roles/iam.serviceAccountUser` to the node service account. The controller also needs
+`container.clusters.get` and, if you use `channel:` image terms, `container.clusters.list`.
+
+**Legacy bootstrap pools.** Karpenter no longer creates `karpenter-default`,
+`karpenter-ubuntu`, `karpenter-cos-arm64`, or `karpenter-ubuntu-arm64`. It discovers an
+existing RUNNING pool instead. Once provisioning works, delete the legacy pools at your pace.
+
+**`disks` is now effectively mandatory on GCENodeClass.** The CRD neither requires it
+nor supplies a default, but GCE rejects instance creation without it
+(`Error 400: Invalid value for field 'resource.disks': ''`). Add a boot disk to every
+node class before upgrading, or provisioning fails silently at the NodeClaim level while
+the NodePool still reports Ready.
+
+**Node rotation.** The GCENodeClass hash version bumps, so Karpenter triggers one controlled
+rolling replacement of every node it manages after the upgrade.
+
 ## Uninstall Helm Chart
 
 To remove the Karpenter Helm chart and associated resources, run:
@@ -163,11 +258,29 @@ This Helm chart includes several configurable parameters to adapt deployment to
 | **Input** | **Type** | **Description** | **Default** |
 |----------|----------|------------------|-------------|
 | `controller.settings.projectID` | `string` | GCP project ID | `""` |
-| `controller.settings.region` | `string` | GCP region | `""` |
+| `controller.settings.clusterLocation` | `string` | Region or zone used for instance type discovery, e.g. `us-central1` | `""` |
+| `controller.settings.nodeLocation` | `string` | Exact cluster location for GKE API calls: `us-central1-a` for a zonal cluster, `us-central1` for a regional one. Falls back to `clusterLocation` | `""` |
 | `controller.settings.clusterName` | `string` | GKE cluster name | `""` |
 | `controller.settings.vmMemoryOverheadPercent` | `float` | Overhead multiplier for VM memory | `0.065` |
 | `controller.settings.batchMaxDuration` | `string` | Max batching delay | `"10s"` |
 | `controller.settings.batchIdleDuration` | `string` | Idle batching delay | `"1s"` |
+| `controller.settings.ignoreDRARequests` | `bool` | Ignore pods' Dynamic Resource Allocation requests while simulating scheduling | `true` |
+| `controller.settings.defaultNodePoolTemplateName` | `string` | Pin the GKE node pool read for bootstrap metadata. Empty means automatic discovery | `""` |
+| `controller.settings.defaultNodepoolServiceAccount` | `string` | GCP service account email attached to provisioned nodes, overriding the Compute Engine default | `""` |
+
+### Feature Gates
+
+Everything except `spotToSpotConsolidation` is ALPHA upstream and off by default.
+
+| **Input** | **Type** | **Description** | **Default** |
+|----------|----------|------------------|-------------|
+| `controller.featureGates.spotToSpotConsolidation` | `bool` | Spot replacement consolidation, single and multi-node | `true` |
+| `controller.featureGates.nodeOverlay` | `bool` | Let NodeOverlay resources influence scheduling decisions | `false` |
+| `controller.featureGates.staticCapacity` | `bool` | A NodePool with `spec.replicas` holds a fixed node count regardless of pod demand | `false` |
+| `controller.featureGates.nodeRepair` | `bool` | Replace nodes failing GKE Node Problem Detector health conditions | `false` |
+| `controller.featureGates.reservedCapacity` | `bool` | Scheduling onto reserved GCP capacity. The provider does not implement GCE reservations yet | `false` |
+| `controller.featureGates.capacityBuffer` | `bool` | CapacityBuffer pre-provisioning. The CRD is not shipped here — GKE provides it from `1.35.2-gke.1842000`, and enabling this without it fails at render time | `false` |
+| `controller.disableControllerWarmup` | `bool` | Whether watches/informers start before leader election is won. `false` speeds up leader failover | `true` |
 
 ### Logging
 
@@ -182,10 +295,37 @@ This Helm chart includes several configurable parameters to adapt deployment to
 | **Input** | **Type** | **Description** | **Default** |
 |----------|----------|------------------|-------------|
 | `controller.replicaCount` | `integer` | Number of controller replicas | `2` |
+| `controller.image.tag` | `string` | Controller image tag | `"v0.6.0"` |
+| `controller.priorityClassName` | `string` | Priority class for the controller pods. `gmp-critical` is created by Google Managed Prometheus. Set to `""` on clusters without it — naming a class that does not exist blocks pod creation entirely | `"gmp-critical"` |
 | `controller.image.repository` | `string` | Controller image repository | `"public.ecr.aws/cloudpilotai/gcp/karpenter"` |
-| `controller.image.tag` | `string` | Controller image tag | `"v0.0.1"` |
+| `controller.image.pullPolicy` | `string` | Image pull policy | `"IfNotPresent"` |
+| `imagePullSecrets` | `list` | Existing pull secrets, for a private registry mirror | `[]` |
+| `controller.strategy` | `object` | Deployment update strategy | `{ rollingUpdate: { maxUnavailable: 1 } }` |
+| `controller.terminationGracePeriodSeconds` | `integer` | Grace period for controller shutdown | `30` |
+| `controller.healthProbe.port` | `integer` | Port for the health and readiness probes. Upstream defaults to `8081`; this chart keeps `8001` so the port does not move on upgrade | `8001` |
+| `controller.affinity` | `object` | Overrides the default pod anti-affinity, which spreads replicas one per node using the chart's selector labels. Leave empty to keep that default | `{}` |
+| `controller.tolerations` | `list` | Tolerations for the controller pods | `[]` |
 | `controller.resources` | `object` | CPU and memory requests | `{ cpu: 500m, memory: 500Mi }` |
 | `controller.metrics.port` | `integer` | Metrics port | `8080` |
+| `podSecurityContext` | `object` | Security context applied at the pod level | `{ runAsNonRoot: true, seccompProfile: { type: RuntimeDefault } }` |
+| `securityContext` | `object` | Security context applied to the karpenter container | see `values.yaml` |
+| `podDisruptionBudget.minAvailable` | `integer` | Minimum available pods, used only when `maxUnavailable` is null | `1` |
+| `podDisruptionBudget.maxUnavailable` | `integer` | Maximum unavailable pods. Takes precedence over `minAvailable` when set | `null` |
+
+### Metrics
+
+`serviceMonitor` renders a Prometheus Operator `ServiceMonitor`, and only when the
+`monitoring.coreos.com/v1` API is present on the cluster.
+
+| **Input** | **Type** | **Description** | **Default** |
+|----------|----------|------------------|-------------|
+| `serviceMonitor.enabled` | `bool` | Create a ServiceMonitor for the metrics endpoint | `false` |
+| `serviceMonitor.additionalLabels` | `map` | Extra labels on the ServiceMonitor | `{}` |
+| `serviceMonitor.interval` | `string` | Scrape interval. Empty uses the operator default | `""` |
+| `serviceMonitor.scrapeTimeout` | `string` | Scrape timeout. Empty uses the operator default | `""` |
+| `serviceMonitor.relabelings` | `list` | Relabelings for the metrics endpoint | `[]` |
+| `serviceMonitor.metricRelabelings` | `list` | Metric relabelings for the metrics endpoint | `[]` |
+| `serviceMonitor.sampleLimit` | `integer` | Per-scrape cap on accepted samples. `null` means no limit | `null` |
 
 ### Service Account
 
@@ -227,12 +367,16 @@ podLabels: {}
 
 podDisruptionBudget:
   minAvailable: 1
+  maxUnavailable: null
+
+serviceMonitor:
+  enabled: false
 
 controller:
   replicaCount: 2
   revisionHistoryLimit: 10
   image:
-    tag: "v0.0.1"
+    tag: "v0.6.0"
 
   env: []
 
@@ -244,10 +388,13 @@ controller:
   metrics:
     port: 8080
 
+  featureGates:
+    spotToSpotConsolidation: true
+
   settings:
     projectID: "<project-id>"
-    region: "<zone>"
-    clusterName: "<cluster-name"
+    clusterLocation: "<region-or-zone>"
+    clusterName: "<cluster-name>"
     vmMemoryOverheadPercent: 0.065
     batchMaxDuration: 10s
     batchIdleDuration: 1s
@@ -277,8 +424,8 @@ The karpenter deployment includes:
 - ClusterRoleBinding for controller access to Karpenter CRDs
 - ClusterRoles for managing Karpenter and core Kubernetes resources
 - PodDisruptionBudget for controller
-- Service for exposing metrics
-- CRDs: GCENodeClass, NodePool, NodeClaim
+- Service for exposing metrics, and an optional ServiceMonitor for the Prometheus Operator
+- CRDs: GCENodeClass, NodePool, NodeClaim, NodeOverlay
 
 ---