Skip to content

Latest commit

 

History

History
495 lines (401 loc) · 24.9 KB

quickstart-kubernetes.md

File metadata and controls

495 lines (401 loc) · 24.9 KB

Quickstart: Kubernetes

This path installs the EDK enterprise services on Kubernetes using the Helm chart under helm/edk-enterprise/. The chart renders deployments, services, ingress (or a single-port Gateway API front door), NetworkPolicies, and hardened security defaults.

Use the public Enterprise Development Kit Deployment repository: https://github.com/Sphereon-Opensource/Enterprise-Development-Kit-Deployment.

The chart reference, including every value and its default, is in helm/edk-enterprise/README.md.

Prerequisites

  • A Kubernetes cluster and kubectl configured against it.
  • Helm 3 or Helm 4.
  • Nexus credentials for the published nexus.sphereon.com/edk-docker/enterprise-* and nexus.sphereon.com/edk-docker/admin-console images for the selected global.imageTag.
  • Keep global.imageRegistry=nexus.sphereon.com/edk-docker. Do not use sphereon or docker.io/sphereon; those values point Kubernetes at public Docker Hub.
  • Reachable PostgreSQL 15+ databases for the platform control plane and tenant workload data plane. The chart does not deploy Postgres. Use managed databases, operator-managed databases, or environment-owned Postgres releases, then point database.platform.* and database.tenant.* at them. These must be two separate logical databases; never put platform and tenant state in the same database, even with separate schemas.
  • A protected license bundle ZIP from the license issuer provided through your EDK distribution channel, ready to import during onboarding.
  • TLS material for the operator and tenant hosts. Use one wildcard certificate for *.<base-domain>, or individual certificates for every deployed host. For Let's Encrypt wildcard certificates, use cert-manager with DNS-01 validation.
  • An existing symmetric AWS KMS key plus a workload identity that permits only the platform pod to attest, encrypt/decrypt, create, tag, resolve, and revoke the server-derived KMS bindings. Set secretManagement.bootstrap.kek.identity, secretManagement.bootstrap.kek.awsRegion, and the cloud workload annotation under serviceAccount.platform.annotations. Do not grant that identity to the shared satellite ServiceAccount.

Recommended one-go install or upgrade

From the repository root on Linux or macOS, use the release-independent wrapper. Select the immutable image tag named by the release you are installing:

export TARGET_IMAGE_TAG='<approved-release-tag>'

bash ./scripts/upgrade-helm.sh \
  --release sphereon-edk-enterprise \
  --namespace edk \
  --values ./customer-values.yaml \
  --image-tag "$TARGET_IMAGE_TAG" \
  --tenant-host abc.example.com

--release and --namespace are yours to choose. The values shown are the wrapper defaults; pass the release name and namespace this install actually uses.

The wrapper preserves existing cryptographic Secrets, backs up the installed release, validates the candidate manifests, quiesces the complete release, upgrades without automatic rollback, waits for all Deployments, and optionally verifies the tenant DID document. It reads the installed global.imageTag and selects the known ordered migration path automatically. Add --migration-values <path> only for an additional overlay that is not already known to the wrapper.

Upgrading from 0.25.0-RC1 to 0.25.0-RC2

0.25.0-RC1 shipped Helm defaults that left /.well-known off the DID service's anonymous path list. That path serves the tenant DID document, and with it behind bearer-token auth the document could not be fetched anonymously, so tenant creation could not complete. 0.25.0-RC2 corrects the defaults so the DID document and the other public discovery paths resolve without a token.

A values file exported from an RC1 install can still carry the old, empty anonymous-path settings, and those would override the corrected RC2 defaults. The overlay at helm/edk-enterprise/examples/upgrades/0.25.0-rc1-to-0.25.0-rc2-values.yaml re-asserts the correct public paths. Pass it after your own values file so it takes precedence. Set --release and --namespace to the release name and namespace of your existing RC1 install, not the placeholders below. If you did not override them when you installed, the wrapper defaults are sphereon-edk-enterprise and edk.

bash ./scripts/upgrade-helm.sh \
  --release <your-release> \
  --namespace <your-namespace> \
  --values ./customer-values.yaml \
  --image-tag 0.25.0-RC2

The wrapper detects RC1 from the installed Helm values and applies the bundled RC1-to-RC2 overlay automatically. The overlay sets only serviceIdentity.anonymousPathPrefixes and holds no Secret values.

After the upgrade completes, create a tenant from the admin console. This now succeeds because the DID document resolves without a token. Confirm it by fetching https://<tenant-host>/.well-known/did.json along with the tenant metadata paths listed in onboarding.md. Once a tenant exists, you can also re-run the wrapper with --tenant-host <tenant-host> so it checks the DID document for you.

Upgrading from 0.25.0-RC2 to 0.25.0-RC3

RC3 adds the tenant trust-domain management route and changes tenant KMS configuration to use the stable logical provider id default. An RC2 values export can pin the old ingress route list, so apply the bundled transition overlay after your maintained values:

bash ./scripts/upgrade-helm.sh \
  --release <your-release> \
  --namespace <your-namespace> \
  --values ./customer-values.yaml \
  --image-tag 0.25.0-RC3 \
  --tenant-host <existing-rc2-tenant-host>

The wrapper detects RC2 and applies both compatibility overlays cumulatively: RC1-to-RC2 first and RC2-to-RC3 second. Keeping the earlier overlay is intentional; it makes the result independent of whether the maintained values file originated from RC1 or RC2.

RC3 also switches the backend Deployments to the Recreate strategy so a single schema generation reaches the database during the migration. A release installed before RC3 carries the RollingUpdate default, which Kubernetes refuses to combine with Recreate, so an unconverted release fails the upgrade with spec.strategy.rollingUpdate: Forbidden. The wrapper converts every backend Deployment while the release is stopped. If you drive helm upgrade yourself, apply the one-time conversion documented under "Upgrading without the wrapper" in the README before the upgrade.

Secret management is a greenfield cutover. It does not import, adopt, backfill, or dual-read provider state from an earlier release. Before the rollout, take a database snapshot and remove obsolete secret-provider and secret-migration state. If live legacy state is detected, platform startup stops with a reset diagnostic. The wrapper stops every release workload; restore both database snapshots before running an explicit Helm rollback. That restoration is the only supported way to run the previous binary again.

After the rollout, verify both an existing RC2 tenant and a newly created tenant:

  • https://<existing-rc2-tenant-host>/.well-known/did.json
  • https://<existing-rc2-tenant-host>/.well-known/openid-credential-issuer
  • create a new tenant and verify the same two endpoints for its host

The wrapper owns these release overlays. Rerunning the same RC3 command reapplies the same cumulative pair, so it cannot undo an earlier compatibility correction. Do not copy the transition values into the maintained site values file.

Upgrading directly from 0.25.0-RC1 to 0.25.0-RC3

Use the same one-go command and name only the final target image:

bash ./scripts/upgrade-helm.sh \
  --release <your-release> \
  --namespace <your-namespace> \
  --values ./customer-values.yaml \
  --image-tag 0.25.0-RC3 \
  --tenant-host <existing-rc1-tenant-host>

The wrapper detects the installed RC1 tag and performs the complete ordered upgrade itself:

  1. Render and validate RC2 with the RC1-to-RC2 overlay.
  2. Quiesce the release, upgrade to the official RC2 images without automatic rollback, and wait for every Deployment.
  3. Render RC3 with the RC1-to-RC2 and RC2-to-RC3 overlays in that order.
  4. Quiesce the release again and perform the final Helm upgrade to RC3. Platform startup runs the idempotent registration/KMS data migration before readiness.
  5. Verify the existing tenant DID document when --tenant-host is supplied.

The intermediate and final candidate manifests are stored in the upgrade backup directory. If either stage fails, the script scales the release to zero and stops without rolling application pods back. Restore both database snapshots before running an explicit Helm rollback or retry.

Upgrading from 0.25.0-RC3 to 0.25.0-RC4

RC4 replaces the single serviceIdentity.internalClientExistingSecret key with one client-secret key per satellite role, adds a dedicated federationSessionEncryption key, and adds a third secret-management database role. It also corrects the public ingress path lists for platform, did, issuer, and tenant-as, and removes the /api/trust-domain/v1 path that the RC2-to-RC3 overlay advertised on tenant-as without a serving module behind it. Provision the two manual prerequisites below before running the wrapper.

bash ./scripts/upgrade-helm.sh \
  --release <your-release> \
  --namespace <your-namespace> \
  --values ./customer-values.yaml \
  --image-tag 0.25.0-RC4 \
  --release-set-evidence ./enterprise-image-set.json \
  --tenant-host <existing-rc3-tenant-host>

--release-set-evidence is required: the wrapper refuses immutable RC3 and RC4 tags without it (--release-set-evidence is required for immutable RC3 and RC4 upgrades.). Obtain enterprise-image-set.json with the RC4 release and pass the file as delivered; it must bind the requested tag to all seven default images with immutable content and provenance.

The wrapper detects the installed RC3 tag and applies the bundled 0.25.0-rc3-to-0.25.0-rc4-values.yaml overlay after your maintained values, on top of the earlier RC1-to-RC2 and RC2-to-RC3 overlays applied cumulatively. Keeping the earlier overlays is intentional, for the same reason as the RC3 upgrade above. Do not copy the transition values into the maintained site values file.

Runtime Secret keys

Recreate or update the runtime Secret with one key per satellite role instead of the single internal-client-secret key. The chart no longer reads internal-client-secret; Helm fails if internalClientSecretKey is still set. The default serviceIdentity.clientSecretKeys values are:

  • kms-service-client-secret
  • tenant-as-service-client-secret
  • did-service-client-secret
  • blob-service-client-secret
  • issuer-service-client-secret
  • verifier-service-client-secret
  • trust-domain-service-client-secret
  • as-signing-service-client-secret

wallet-unit-service-client-secret and wallet-interaction-service-client-secret are needed only when those optional workloads are enabled.

Add a federation-session-encryption-key entry for federationSessionEncryption.key. Nothing creates these keys on an upgrade: add them to the existing Secret yourself. Generate every value independently; none of them may reuse the retired internal-client-secret value, and the key names above may all be renamed in your values overlay as long as serviceIdentity.clientSecretKeys and federationSessionEncryption.key point at the keys you chose.

Secret-management database role

RC4 adds a runtime-password key to the secret-management database Secret and requires the secret_management_runtime PostgreSQL role in both the platform database and the tenant database. The database init script that creates this role only runs against an empty data directory, so an existing installation must create the role by hand before upgrading, in both databases, as the bootstrap superuser:

CREATE ROLE secret_management_runtime LOGIN NOSUPERUSER NOCREATEDB NOCREATEROLE NOINHERIT NOBYPASSRLS PASSWORD '<runtime-password>';
GRANT CONNECT ON DATABASE <db> TO secret_management_runtime;
GRANT USAGE ON SCHEMA public TO secret_management_runtime;

Use the same generated password on both databases and store it under the runtime-password key of the Secret referenced by database.secretManagement.existingSecret, alongside the existing admin-password and tenant-password keys. Grant all three roles USAGE on the target schema only; do not grant CREATE, ownership, role membership, superuser, or BYPASSRLS.

Secret-authority public keys

The platform mounts one assertion public key per workload listed under secretAuthority.keys.assertionPublicKeys, whether or not that workload is deployed. RC4 lists service-blob and service-wallet-onboarding, which an RC3 platform Secret does not carry, so the platform pod fails to mount its secret-authority volume with references non-existent secret key until they exist. Generate a key pair for each missing workload with scripts/generate-secret-authority-keys.sh and add the public key to the Secret named by secretAuthority.existingSecrets.platform under the key the chart lists (for example service-wallet-onboarding-assertion.pub.pem). Keep the private half only if you deploy that workload.

Business wallet

The chart does not deploy the business wallet by default. Its image, vdx-business-wallet-web, is a wallet image: it is not part of the EDK enterprise image set and is not distributed through the EDK registry. To deploy the business wallet, set services.business-wallet.enabled: true and set global.walletImageRegistry to the registry that holds the wallet images.

OID4VCI issuer identifier

Each tenant's OID4VCI credential issuer identifier becomes https://<tenant-host>/oid4vci/<tenant>, not the bare tenant origin. Existing credential configuration carries over automatically at the first platform start after the upgrade. Wallets and relying parties that hold the old identifier must be onboarded again with a new credential offer.

Authorization-server migration resume

If the platform refuses to start with a message beginning Authorization-server migration source previously failed, set platform.authorizationServerMigrationResumeFailed: true in your values for one upgrade, then set it back to false. Do not change oauth2.servers.* configuration between the failed start and the retry. A migration that fails only during planning retries on its own and does not need the flag.

After the rollout, verify both an existing RC3 tenant and a newly created tenant:

  • https://<existing-rc3-tenant-host>/.well-known/openid-credential-issuer returns 200 with credential_issuer ending in /oid4vci/<tenant>
  • https://<existing-rc3-tenant-host>/.well-known/did.json
  • create a new tenant and verify the same two endpoints for its host

1. Create the namespace and pull secret

kubectl create namespace edk

kubectl -n edk create secret docker-registry sphereon-nexus \
  --docker-server=nexus.sphereon.com \
  --docker-username=<username> \
  --docker-password=<password>

Reference the pull secret from your values with global.imagePullSecrets.

2. Create the database credentials secrets

The chart reads database usernames and passwords from existing Secrets named by database.platform.existingSecret and database.tenant.existingSecret. Create separate Secrets for the platform database and tenant workload database:

kubectl -n edk create secret generic edk-platform-postgres \
  --from-literal=username=edk_platform \
  --from-literal=password=<platform-db-password>

kubectl -n edk create secret generic edk-tenant-postgres \
  --from-literal=username=edk_tenant \
  --from-literal=password=<tenant-db-password>

kubectl -n edk create secret generic edk-secret-management-database \
  --from-literal=admin-password=<secret-management-admin-password> \
  --from-literal=tenant-password=<secret-management-tenant-password> \
  --from-literal=runtime-password=<secret-management-runtime-password>

Before installing, provision fixed PostgreSQL login roles secret_management_admin and secret_management_tenant_serving in the platform database with the matching passwords. Both roles must be NOSUPERUSER NOBYPASSRLS and need USAGE on the target schema. Neither role may own or create schema objects. The platform database owner is used only by the deployment-time secret-management-migrator route; do not reuse it for either runtime pool. See Secret management for the exact trust boundary.

Point the chart at the control-plane database through database.platform.host, database.platform.port, and database.platform.name, and at the tenant workload database through database.tenant.host, database.tenant.port, and database.tenant.name. The two databases may live on the same PostgreSQL server only if they are separate database names with separate credentials. Do not reuse one database or one shared credential for both endpoints in an enterprise deployment. Tenant schemas are created inside the tenant workload database; they are not a replacement for the platform/tenant database split.

3. Create the runtime Secret

The chart does not generate the per-client STS secrets used for east-west service tokens or the password protecting the software PKCS#12 keystores. Create those values in the release namespace before installing. edk-runtime-secrets is an example Kubernetes Secret name, not an image or prepackaged file. Each satellite still reads the env name EDK_INTERNAL_CLIENT_SECRET; Helm mounts only that role's serviceIdentity.clientSecretKeys.<role> entry into it:

The corresponding Helm parameters are serviceIdentity.internalClientExistingSecret and keystore.existingSecret.

kubectl -n edk create secret generic edk-runtime-secrets \
  --from-literal=kms-service-client-secret='<independent-secret>' \
  --from-literal=tenant-as-service-client-secret='<independent-secret>' \
  --from-literal=did-service-client-secret='<independent-secret>' \
  --from-literal=blob-service-client-secret='<independent-secret>' \
  --from-literal=issuer-service-client-secret='<independent-secret>' \
  --from-literal=verifier-service-client-secret='<independent-secret>' \
  --from-literal=wallet-unit-service-client-secret='<independent-secret>' \
  --from-literal=wallet-interaction-service-client-secret='<independent-secret>' \
  --from-literal=trust-domain-service-client-secret='<independent-secret>' \
  --from-literal=as-signing-service-client-secret='<independent-secret>' \
  --from-literal=admin-console-portal-bff-secret='<independent-long-random-portal-bff-secret>' \
  --from-literal=keystore-password='<long-random-pkcs12-password>'

Generate each value independently with at least 32 random bytes. For production, have the cluster's secret-management mechanism create this Secret; do not commit Secret manifests containing plaintext values.

Reference the Secret name and keys from the values overlay:

serviceIdentity:
  internalClientExistingSecret: edk-runtime-secrets
  clientSecretKeys:
    tenant-kms: kms-service-client-secret
    tenant-as: tenant-as-service-client-secret
    did: did-service-client-secret
    blob: blob-service-client-secret
    issuer: issuer-service-client-secret
    verifier: verifier-service-client-secret
    wallet-unit: wallet-unit-service-client-secret
    wallet-interaction: wallet-interaction-service-client-secret
    trust-domain-identifier: trust-domain-service-client-secret
    as-signing-management: as-signing-service-client-secret
keystore:
  existingSecret: edk-runtime-secrets
  passwordKey: keystore-password
portalBff:
  existingSecret: edk-runtime-secrets
  clientSecretKey: admin-console-portal-bff-secret

Each clientSecretKeys.<role> value is a distinct key used by that satellite confidential client. admin-console-portal-bff-secret belongs only to the dedicated portal BFF confidential client. keystore-password protects the platform and tenant-KMS software keystores. Changing Secret data under the same name requires restarting the platform and the satellite whose key changed because these values are read when containers start.

4. Pick a values overlay

The examples/ directory holds ready-to-copy overlays. Start from one and adjust:

File Purpose
shared-postgres-values.yaml Environment-owned in-cluster Postgres endpoints for separate platform and tenant databases, JWT auth wired to the tenant AS
external-managed-postgres-values.yaml Managed external Postgres with an egress NetworkPolicy
local-docker-desktop-values.yaml Port-forwarded Docker Desktop evaluation without Gateway API or Ingress resources
service-jwt-auth-values.yaml Require JWT on each service's admin REST
secret-backed-credentials-values.yaml Pull signing and provider credentials from Secrets
admin-console-values.yaml Configure the platform console and isolated issuer/verifier testing-console routes
gateway-cilium-values.yaml Single-port multi-tenant front door via Cilium Gateway API
gateway-aws-alb-values.yaml, gateway-gke-values.yaml, gateway-azure-agic-values.yaml Cloud gateway variants
mesh-mtls-values.yaml Service-mesh mTLS for inter-service traffic
opentelemetry-values.yaml OTLP exporter wiring

Set global.platformBaseDomain to the customer-controlled base domain for the installation. The platform host is platform.<base-domain> and tenant hosts are <tenant-slug>.<base-domain>. Point DNS for platform.<base-domain> and *.<base-domain> at the gateway address. Do not publish runtime probes or backing service hosts as customer endpoints.

5. Render and install

Render first so you can review the manifests:

helm template edk ./helm/edk-enterprise \
  -n edk \
  -f ./helm/edk-enterprise/examples/shared-postgres-values.yaml

Then install or upgrade:

helm upgrade --install edk ./helm/edk-enterprise \
  -n edk \
  -f ./helm/edk-enterprise/examples/shared-postgres-values.yaml

Rendering validates that both runtime Secret references are configured. Helm cannot verify Secret objects or keys at render time; use the checks in troubleshooting.md if pods enter CreateContainerConfigError after installation.

6. Ingress and gateway

Use the single-port Gateway API front door. The chart defaults to gateway.enabled: true and ingress.legacy.enabled: false, so one HTTPS listener fronts the platform host and all tenant hosts. HTTPRoutes fan out by host and path to each backing service. The operator host is {gateway.operatorHost}.{baseDomain} (default platform.<baseDomain>), and each tenant is reached at <slug>.<baseDomain>. Provide wildcard TLS material with gateway.tls.mode: secret and an existing Secret, or gateway.tls.mode: certManager with a cert-manager ClusterIssuer. For Let's Encrypt wildcard certificates, configure the ClusterIssuer for DNS-01, not HTTP-01. Individual certificates per host are possible, but each tenant host certificate must exist before that tenant host goes live. The cloud gateway example overlays cover AWS, GKE, Azure, and Cilium.

Whichever model you choose, the customer-visible contract is the platform host and tenant gateway hosts. DID resolver, OAuth/OIDC, OID4VCI, OID4VP, and any selected operator/admin API paths are host/path routes through that front door. Customers do not call pods or containers directly. Keep every /api/.../v1 path internal or protected by operator/tenant authentication, and never publish runtime probes as customer-facing routes.

Admin console and testing console. services.admin-console serves the platform persona at platform.<base-domain>/admin-console; services.admin-console-tenant serves the tenant persona at <tenant>.<base-domain>/admin-console from the same image without a platform BFF credential. The wildcard route is only a transport rule: the application must resolve an active registered tenant before OAuth. The tenant runtime also keeps the /testing-console/{kind}/{instanceId} page and its existing page-only rewrite and support routes. See examples/admin-console-values.yaml and tls-and-gateway.md.

7. Security defaults

The chart runs every service as non-root UID/GID 10001 with a read-only root filesystem, dropped Linux capabilities, the runtime default seccomp profile, REST auth enabled, JWT auth wiring enabled, and NetworkPolicies enabled. For production, set non-empty auth.jwt.issuer, auth.jwt.jwksUri, and auth.jwt.audience. The example overlays point the JWT issuer and JWKS URI at the tenant AS.

8. First-run setup and tenant onboarding

With the release running, open https://platform.<base-domain>/setup-license or https://platform.<base-domain>/admin-console and complete first-run setup. Setup generates the license request, imports the protected license bundle, and bootstraps the first platform operator account. After setup closes the anonymous setup gate, sign in at https://platform.<base-domain>/admin-console and create tenants from the admin console or platform admin API.

The scripts/provision helper and Postman collection are optional validation and automation tools that call the same APIs against the running platform. See onboarding.md.