Add support for cdk8s

.dev/dev-storage-classes.yaml

@@ -0,0 +1,21 @@
+# Apply these when running the cluster in a dev environment
+apiVersion: storage.k8s.io/v1
+kind: StorageClass
+metadata:
+  name: node-local-zfs
+provisioner: rancher.io/local-path
+volumeBindingMode: WaitForFirstConsumer
+---
+apiVersion: storage.k8s.io/v1
+kind: StorageClass
+metadata:
+  name: memory-hdd
+provisioner: rancher.io/local-path
+volumeBindingMode: WaitForFirstConsumer
+---
+apiVersion: storage.k8s.io/v1
+kind: StorageClass
+metadata:
+  name: memory-ssd
+provisioner: rancher.io/local-path
+volumeBindingMode: WaitForFirstConsumer

.gitignore

@@ -1,2 +1,7 @@
 .idea
-**/charts
+**/charts
+.junie
+
+imports
+dist/
+node_modules

PLAN.md

@@ -0,0 +1,23 @@
+we are working on a comprehensive GitOps setup.
+
+the current version uses ArgoCD with each ArgoCD Application stored in a top-level directory in the repo,
+as either classic K8s .yaml files or a kustomize.yaml.
+
+i want to add support for cdk8s-based apps, with automatic update PRs using Renovate.
+additionally i want to be able to deploy a cluster locally for testing.
+
+## plan
+
+- [x] keep the same top-level setup and add support for cdk8s apps using a `app.ts` file as the entry-point
+      (done via an ArgoCD Config Management Plugin sidecar, see `argocd/kustomization.yaml`; sample in `demo/`)
+- [ ] the current setup uses some Renovate magic to open PRs from other repositories from GitHub Actions.
+  see kir-dev/k8s and kir-dev/StartSCH (both already in ~/src).
+  create a script that can be run from the CI/CD pipelines of other repositories that opens an update PR for that specific app.
+
+## rules
+- top-level directories other than the ones starting with a `.` are ArgoCD Applications.
+  keep non-Application files either in a dir like `.cdk8s` (or something else) or at the top-level.
+- use bun. don't add dependencies unless necessary.
+
+## notes
+- there is a very basic cdk8s sample in ~/Desktop/f using bun

README.md

@@ -1,3 +1,6 @@
+
+TODO: Update README
+
 # Kir-Dev Kubernetes configuration
 
 ## Bootstrapping
@@ -39,10 +42,35 @@ Create a new directory containing
     [`helmCharts:`](https://kubectl.docs.kubernetes.io/references/kustomize/builtins/#_helmchartinflationgenerator_)
     to install Helm charts. Set values either using `valuesInline:` or by creating a `values.yaml` and
     referencing it using `valuesFile:`.
+- a cdk8s app: a `cdk8s.yaml` + `app.ts` (see below).
 
 ArgoCD checks each directory (except the ones starting with a `.`). If it sees `kustomization.yaml`, it `kubectl apply --kustomize`s it, otherwise it applies
 `.yaml` files using `kubectl apply`.
 
+### cdk8s apps
+
+Directories containing an `app.ts` are synthesized with cdk8s by an ArgoCD
+[Config Management Plugin](https://argo-cd.readthedocs.io/en/stable/operator-manual/config-management-plugins/)
+(sidecar on the repo-server, see `argocd/kustomization.yaml`). There is a single
+cdk8s project for the whole repo: the shared `cdk8s.yaml` (imports) and
+`package.json` (deps) live at the top level, and each cdk8s app is just a
+directory with an `app.ts`:
+
+```
+<repo root>/
+  cdk8s.yaml        # language: typescript + imports (shared)
+  package.json      # deps: cdk8s, constructs; devDeps: cdk8s-cli (shared)
+  imports/          # generated (gitignored)
+  my-app/
+    app.ts          # the cdk8s app entrypoint (Chart(s) + app.synth())
+```
+
+`app.ts` imports from `../imports/k8s`. The CMP runs from the repo root:
+`bun install`, `cdk8s import`, then synthesizes *only that app's* `app.ts` into a
+per-app `dist/<app>/` and feeds `dist/<app>/*.k8s.yaml` to ArgoCD. The sidecar
+image is `nixery.dev/bun/kubernetes-helm/bash/coreutils/cacert/nodejs` (nixery
+bundles `bun`, `helm`, `node`, and a shell). See `demo/` for a minimal example.
+
 ## Documentation
 
 - https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/

argocd/kustomization.yaml

@@ -22,6 +22,40 @@ helmCharts:
           # https://argo-cd.readthedocs.io/en/stable/user-guide/kustomize/#kustomizing-helm-charts
           kustomize.buildOptions: "--enable-helm"
 
+        # Sidecar Config Management Plugin for cdk8s apps.
+        # Top-level directories containing an `app.ts` are synthesized with cdk8s.
+        # The shared `cdk8s.yaml` + `package.json` live at the repo root, so the
+        # generate step cd's to the root, installs deps, (re)generates `imports/`,
+        # then synthesizes only this app's `app.ts` into a per-app output dir.
+        # https://argo-cd.readthedocs.io/en/stable/operator-manual/config-management-plugins/
+        cmp:
+          create: true
+          plugins:
+            cdk8s:
+              discover:
+                fileName: "app.ts"
+              generate:
+                command: [bash, -c]
+                args:
+                  - |
+                    set -e
+                    app_dir="$(pwd)"
+                    root_dir="$(cd .. && pwd)"
+                    out_dir="$root_dir/dist/$(basename "$app_dir")"
+                    cd "$root_dir"
+                    # copyfile backend -> real node_modules (no symlinks into bun's
+                    # global cache, which breaks module resolution from nested dirs)
+                    if [ -f bun.lock ]; then bun install --frozen-lockfile --backend=copyfile >&2; else bun install --backend=copyfile >&2; fi
+                    bun run import >&2
+                    # materialize the imports symlink (import script caches via symlink)
+                    if [ -L imports ]; then
+                      target="$(readlink imports)"
+                      rm imports
+                      cp -rL "$target" imports
+                    fi
+                    ./node_modules/.bin/cdk8s synth --app "bun $app_dir/app.ts" --output "$out_dir" >&2
+                    cat "$out_dir"/*.k8s.yaml
+
       global:
         networkPolicy:
           create: true
@@ -109,6 +143,42 @@ helmCharts:
               cpu: 50m
               memory: 32Mi
               ephemeral-storage: 50Mi
+
+        # Sidecar running the `cdk8s` CMP. Needs `bun` (run/synth cdk8s apps),
+        # `nodejs` (cdk8s-cli bin is a node script), `helm` (cdk8s import pulls
+        # helm charts), plus a shell + coreutils and ca-certificates.
+        # Provided by nixery: https://nixery.dev/
+        extraContainers:
+          - name: cmp-cdk8s
+            command: ["/var/run/argocd/argocd-cmp-server"]
+            image: nixery.dev/bun/kubernetes-helm/bash/coreutils/cacert/nodejs
+            imagePullPolicy: IfNotPresent
+            env:
+              # bun needs a writable HOME for its install cache; /tmp is the cmp-tmp emptyDir.
+              - name: HOME
+                value: /tmp
+            securityContext:
+              runAsNonRoot: true
+              runAsUser: 999
+            volumeMounts:
+              - mountPath: /var/run/argocd
+                name: var-files
+              - mountPath: /home/argocd/cmp-server/plugins
+                name: plugins
+              - mountPath: /home/argocd/cmp-server/config/plugin.yaml
+                subPath: cdk8s.yaml
+                name: argocd-cmp-cm
+              # Do not share the repo-server's /tmp volume (path traversal mitigation).
+              - mountPath: /tmp
+                name: cmp-tmp
+
+        volumes:
+          - name: argocd-cmp-cm
+            configMap:
+              name: argocd-cmp-cm
+          - name: cmp-tmp
+            emptyDir: {}
+
         resources:
           limits:
             cpu: 700m