feat(karpenter-gcp): update chart to upstream v0.6.0 - #318
Open
jatintalgotra-zd wants to merge 3 commits into
Open
feat(karpenter-gcp): update chart to upstream v0.6.0#318jatintalgotra-zd wants to merge 3 commits into
jatintalgotra-zd wants to merge 3 commits into
Conversation
The chart tracked karpenter-provider-gcp v0.0.1 and was 11 releases behind. The controller env var LOCATION was renamed upstream to CLUSTER_LOCATION, so recent images started with no location configured. Sync CRDs to v0.6.0, add the NodeOverlay CRD and its RBAC, expose the six feature gates and the new controller settings, and add the pod and container security contexts, ServiceMonitor, and PDB maxUnavailable. Also fixes two defects in the local fork: priorityClassName was hardcoded to gmp-critical, which blocks pod creation on any cluster without Google Managed Prometheus, and the pod anti-affinity selector matched a label the chart never sets, so replicas could stack on one node.
Live GKE testing showed every instance create failing with "Error 400: Invalid value for field 'resource.disks': ''. No disks are specified." The CRD neither requires disks nor defaults it, so the NodePool reports Ready while provisioning fails at the NodeClaim level.
…urable The two defects found in this chart both came from the fork freezing a value upstream exposes: priorityClassName blocked pod creation off-GKE, and the anti-affinity selector named a label the renamed chart never sets. The same shape of risk remained in image.repository, pullPolicy, healthProbe.port, strategy, terminationGracePeriodSeconds, tolerations and imagePullSecrets. Restore all of them as values, keeping every current default so rendered output is unchanged. healthProbe.port stays 8001 rather than adopting upstream's 8081 so the port does not move under existing installs. The anti-affinity default stays in the template using selectorLabels instead of moving to values as a literal, so it cannot drift from the chart name again. Missing values keys vs upstream drop from 15 to 4.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The chart was forked from karpenter-provider-gcp around October 2025 and tracked app version
v0.0.1. Upstream is now atv0.6.0— 11 releases ahead. This syncs it.Why this matters
Upstream renamed the controller's
LOCATIONenv var toCLUSTER_LOCATIONand addedNODE_LOCATION. The chart still set the old name, so any recent image would start with no location configured. That is the headline fix.What changed
CRDs — the largest part of the diff. GCENodeClass grew 413 → 681 lines. Previously missing:
networkConfig(private nodes, additional interfaces),shieldedInstanceConfig,confidentialInstanceType, disk IOPS/throughput,subnetRangeName,autoGPUTaint,gpuDriverVersion, KMS keys, secondary boot disks, structuredimageSelectorTerms, and the kubelet configuration fields. NodePool/NodeClaim move from karpenter-core v1.7 to v1.14. The NodeOverlay CRD did not exist here at all and is added with its RBAC.Controller config — location rename, plus
IGNORE_DRA_REQUESTS,DEFAULT_NODEPOOL_TEMPLATE_NAME,DEFAULT_NODEPOOL_SERVICE_ACCOUNT,DISABLE_CONTROLLER_WARMUP. All six feature gates are now configurable; onlySpotToSpotConsolidationwas being passed.Templates — pod and container security contexts, ServiceMonitor (off by default, gated on the Prometheus Operator API), PDB
maxUnavailable, explicit namespaces, and a render-time guard that fails clearly ifcapacityBufferis enabled without the CRD.Two fork defects fixed
priorityClassNamewas hardcoded togmp-critical. On a cluster without Google Managed Prometheus the API server refuses every pod, and because the failure sits at the ReplicaSet levelhelm installstill reports success while the Deployment sits at 0/2. Now a value, defaulting togmp-critical, so GKE behaviour is unchanged.app.kubernetes.io/name: karpenter, but this chart's name iskarpenter-gcp, so it matched nothing and both replicas could land on the same node. Now uses theselectorLabelshelper.Reviewing the diff
Most of the CRD line count is a reformat: the fork had re-indented upstream's controller-gen output, and this takes it verbatim so future syncs are a plain copy.
docs/index.yamlchurns broadly becausehelm repo indexregenerates every entry — that is the existing release process, not a mistake.Testing
Verified on a 2-node minikube cluster:
us-central1→ "location is a region",us-central1-b→ "location is a zone, extracting region" — failing only on absent GCP credentials, which is correct off-GCPgmp-criticalon the cluster brings both pods up, confirming the priority class fixspec.tagsrejected as an unknown field, malformedaliasrejected by CEL, reservedmetadata.kube-envrejectedreadOnlyRootFilesystem: truenodeoverlays; CapacityBuffer guard fails without the CRD and renders with ithelm lintcleanNot tested: real node provisioning. That needs a live GKE cluster with IAM. Config plumbing, CRD schemas, RBAC and scheduling are verified; the GCP path beyond credential acquisition is not.
Upgrade notes
This is breaking for existing users, documented in the chart README. Briefly:
settings.location→settings.clusterLocation; NodePoolinstance-familyvalues must drop the shape suffix (n4-standard→n4); GCENodeClasstags→labels;imageSelectorTerms[].aliasdeprecated; the broadcompute.admin/container.adminIAM roles should be replaced with upstream's minimal custom role.Note that Helm never upgrades CRDs in
crds/, so existing installs must apply them manually beforehelm upgrade. The README documents this.