feat: migrate to RHDH 2.1 chart with new schema by Jdubrick · Pull Request #461 · redhat-ai-dev/ai-rolling-demo-gitops

Jdubrick · 2026-10-02T19:50:52Z

What does this PR do?

Updates to the latest RHDH Chart for 2.1, which is 3.4.5
- Updates to the rhel10 image (listed for 2.1) and the matching 2.1 plugin catalog index
Set nameOverride: backstage to preserve the existing Route, Service, and Deployment names. Setup URLs, OIDC callbacks, and CI references depend on those names
- This was already present, just preserving it
Move application configuration, dynamic plugins, authentication, secret references, volumes, and PostgreSQL settings from the old chart keys to the new chart keys. Continue using the existing development PostgreSQL secret
Keep OKP in this repository’s separate subchart and preserve its Lightspeed retrieval configuration and the BYOK data setup
- Will handle removing the OKP subchart in favour of the builtin OKP handling via the Chart in a follow up PR
Remove the custom rhdh-profile.py and ConfigMap. We do not currently maintain that profile and can use the profile generated by the chart.
Replace the PostSync Job that patched sidecars into the Deployment with chart-managed containers, init containers, volumes, and mounts. Remove the Job’s deployment-patching RBAC and extra rollout.
- Add a targeted egress NetworkPolicy for the feedback harvester. The new chart’s default-deny policy covers its pod, while its database is in the separate lightspeed-postgres namespace. The allowance is limited to PostgreSQL pods there on TCP 5432.
Pass the custom router domain through the new chart key so Routes and the OKP URL continue to use it.
Adapt Kind CI and Helm tests to the new chart layout, and carry forward the latest development branch’s Lightspeed resource limits and notebook chunking settings.
Update the image updater for the RHEL 10 repository and the migrated image-tag path.
- This change will be opened against main after this PR merges so that it properly updates the new location. If I change it beforehand it will just be broken during this PRs review

Which issue(s) does this PR fix

https://redhat.atlassian.net/browse/RHIDP-16650

How to test changes / Special notes to the reviewer

qodo-code-review · 2026-10-06T13:33:42Z

ⓘ Qodo reviews are paused because the subscription is no longer active. Ask your workspace admin to reactivate the subscription to resume reviews. Manage billing

JslYoon

Generally lgtm, are you also updating rhdh-intelligent-assistant-configs to stop the sync process?

Jdubrick · 2026-10-06T19:43:11Z

Generally lgtm, are you also updating rhdh-intelligent-assistant-configs to stop the sync process?

The sync was updated to support the new schema redhat-developer/rhdh-intelligent-assistant-configs#10 but I'm not sure I follow about your comment re: stopping the sync? Are you referring to the rhdh-profile.py? If so I plan on changing that after this PR lands to avoid blocking any syncs while this is in review @JslYoon

maysunfaisal · 2026-10-06T22:34:57Z

Do we want to continue with the approach where OKP is hacked in via the rolling demo repo rather than enabling it in rhdh-chart?

I added this piece to the descripion:

Keep OKP in this repository’s separate subchart and preserve its Lightspeed retrieval configuration and the BYOK data setup Will handle removing the OKP subchart in favour of the builtin OKP handling via the Chart in a follow up PR

I think it'll be cleaner / easier to review/test if I separate this into a separate PR where it is just removing the old OKP and enabling via the Chart, wdyt @maysunfaisal

gabemontero · 2026-10-07T20:08:10Z

@Jdubrick and I talked and I would like to do a final validation of this after @HusneShabbir 's PR to enable kserve is merged , we rebase this on top of that, and we can bring up this PR locally and confirm the changes there are good.

The RHDH 2.1 chart reads openshift.clusterRouterBase under the redhat-developer-hub dependency. The installer still set the legacy global.clusterRouterBase key, so custom router domains were ignored. Update both installer paths to set the value used by the Route and Lightspeed OKP URL. The new chart also applies default-deny egress to the RHDH pod. Its PostgreSQL allowance selects only the chart's same-namespace database, while the feedback-harvester sidecar connects to lightspeed-postgres in a separate namespace. Add an egress allowance restricted to pods labeled app=postgres in that namespace on TCP 5432. Omit it from Kind CI, where that database is absent, and assert the exact selectors and port in the Helm render test. Validated with Helm lint, all three Helm tests, shell syntax checks, and git diff --check.

Keep the KServe test's original LEGACY_SIDECARS identifier and the quoted CI PostgreSQL image tag; neither needs a presentation change for the new chart. Restore concise comments that explain the Backstage name override, Kind's Ingress and Fedora PostgreSQL overrides, and why the vendored OKP image is omitted from Kind. The name override preserves the existing Route, Service, and Deployment names used by setup URLs, OIDC callbacks, and CI references. These edits do not change the rendered workloads or network behavior.

Signed-off-by: Jordan Dubrick <jdubrick@redhat.com>

The upstream sync opens a pull request, where any generated profile change can be reviewed when it arrives. Remove the instruction that made upstream workflow cleanup a prerequisite for accepting migration updates. Signed-off-by: Jordan Dubrick <jdubrick@redhat.com>

The migration replaces the legacy backstage/lightspeed values tree, so rebasing over upstream/development's new sidecar resource limits and notebook chunking settings cannot retain them at their old paths. Apply the same values to intelligentAssistant.core.resources and appConfig.intelligent-assistant.notebooks.chunkingStrategy in the 2.1 chart configuration.

Signed-off-by: Jordan Dubrick <jdubrick@redhat.com>

The RHDH 2.1 chart migration timed out during helm install --wait, but the CI artifact contained only a PodInitializing error from an init container that had not started. Capture pod descriptions, events, and logs for each container so a failed run identifies the blocking pod and container. Reduce the Helm wait from 40 to 15 minutes; the last successful Kind install took about three minutes, and this leaves time for diagnosis within the 45-minute job limit.

The new PostgreSQL subchart enables a read-only root filesystem and runs with group 1001 when the CI container security context is enabled. The Fedora PostgreSQL image used by Kind exits while creating /var/lib/pgsql, leaving the RHDH wait-for-db init container blocked until Helm times out. Restore the writable root filesystem and group 0 from the last successful CI render, and assert those settings in the CI values test.

This reverts commit 99fe081.

The RHDH 2.1 chart consumes postgresql.auth.existingSecret as a literal in the Backstage pod, so values.yaml uses the development secret name. Setup creates a PostgreSQL secret from ARGOCD_APP_NAME, which would not match that value for another release. Pass the matching secret name as an ArgoCD Helm parameter in both install paths. This mirrors the CI Helm override and keeps make install and make install-no-rhoai release-name safe.

Signed-off-by: Jordan Dubrick <jdubrick@redhat.com>

The RHDH chart default-deny policy selects the Backstage pod, while its HTTPS allowance matches only TCP 443. On OpenShift, kubernetes.default.svc:443 forwards to an API endpoint on TCP 6443 and OVN evaluates this egress policy after load balancing. The KServe informer therefore times out before TLS or authorization. Allow TCP 6443 from only the release Backstage pod in non-CI deployments. The connector plugin also starts when rhoai.enabled is false, so the policy must cover lightweight installs as well.

gabemontero

it is working with the k8s egress network policy

undo if needed that okp disablement for the lack of self server certs @Jdubrick and as we discussed you'll add in the f/ups you already have planned the stuff in that space to make it sufficiently configurable etc.

Restore the OKP RAG endpoint configuration and okp retrieval source. The original commit was a temporary test-only change and must not remain in the migration PR. Signed-off-by: Jordan Dubrick <jdubrick@redhat.com>

Jdubrick force-pushed the codex/rhdh-2-1-chart-migration branch 4 times, most recently from 444dd4c to 0b32fe7 Compare October 6, 2026 13:23

Jdubrick marked this pull request as ready for review October 6, 2026 13:33

Jdubrick requested review from a team, maysunfaisal and yangcao77 as code owners October 6, 2026 13:33

Jdubrick requested review from JslYoon, gabemontero and michael-valdron October 6, 2026 13:33

Jdubrick force-pushed the codex/rhdh-2-1-chart-migration branch 2 times, most recently from 3a64bca to 5a352cc Compare October 6, 2026 16:32

Jdubrick changed the title ~~(WIP): feat: migrate to RHDH 2.1 chart with new schema~~ feat: migrate to RHDH 2.1 chart with new schema Oct 6, 2026

JslYoon reviewed Oct 6, 2026

View reviewed changes

maysunfaisal reviewed Oct 6, 2026

View reviewed changes

maysunfaisal approved these changes Oct 7, 2026

View reviewed changes

Jdubrick force-pushed the codex/rhdh-2-1-chart-migration branch from 5a352cc to e7f92ff Compare October 7, 2026 19:24

Jdubrick requested a review from JslYoon October 7, 2026 19:25

Jdubrick added 8 commits October 8, 2026 09:47

chore(rhdh): vendor 2.1 community chart for migration

c19f740

feat(rhdh): translate values for 2.1 community chart

a1749a4

feat(rhdh): restore OKP retrieval and BYOK init

f0f7f31

feat(rhdh): move pod additions from PostSync job into chart

c06cb27

test(rhdh): align CI with the 2.1 chart values

40d5a6a

chore: use rhel10 rhdh image instead of rhel9 for 2.x

e062e0b

Signed-off-by: Jordan Dubrick <jdubrick@redhat.com>

Jdubrick added 6 commits October 8, 2026 09:48

bump rhdh chart to 3.4.5

65a3d23

Signed-off-by: Jordan Dubrick <jdubrick@redhat.com>

Revert "ci: capture pod diagnostics when Kind deployment stalls"

f61c5db

This reverts commit 99fe081.

Jdubrick force-pushed the codex/rhdh-2-1-chart-migration branch from e7f92ff to f61c5db Compare October 8, 2026 13:50

JslYoon approved these changes Oct 8, 2026

View reviewed changes

Jdubrick added 3 commits October 8, 2026 14:40

tmp okp fix no merge

cd231fe

Signed-off-by: Jordan Dubrick <jdubrick@redhat.com>

gabemontero approved these changes Oct 9, 2026

View reviewed changes

Revert "tmp okp fix no merge"

6d2ab7c

Restore the OKP RAG endpoint configuration and okp retrieval source. The original commit was a temporary test-only change and must not remain in the migration PR. Signed-off-by: Jordan Dubrick <jdubrick@redhat.com>

gabemontero approved these changes Oct 9, 2026

View reviewed changes

Jdubrick merged commit e3dc5ce into development Oct 9, 2026
2 checks passed

Navigation Menu

Repository navigation

feat: migrate to RHDH 2.1 chart with new schema - #461

feat: migrate to RHDH 2.1 chart with new schema#461
Jdubrick merged 18 commits into
developmentfrom
codex/rhdh-2-1-chart-migration

Jdubrick commented Oct 2, 2026 •

edited

Loading

Uh oh!

qodo-code-review Bot commented Oct 6, 2026

Uh oh!

JslYoon left a comment

Uh oh!

Jdubrick commented Oct 6, 2026

Uh oh!

maysunfaisal Oct 6, 2026

Uh oh!

Jdubrick Oct 7, 2026

Uh oh!

gabemontero commented Oct 7, 2026

Uh oh!

gabemontero left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

Jdubrick commented Oct 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Which issue(s) does this PR fix

How to test changes / Special notes to the reviewer

Uh oh!

qodo-code-review Bot commented Oct 6, 2026

Uh oh!

JslYoon left a comment

Choose a reason for hiding this comment

Uh oh!

Jdubrick commented Oct 6, 2026

Uh oh!

maysunfaisal Oct 6, 2026

Choose a reason for hiding this comment

Uh oh!

Jdubrick Oct 7, 2026

Choose a reason for hiding this comment

Uh oh!

gabemontero commented Oct 7, 2026

Uh oh!

gabemontero left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Jdubrick commented Oct 2, 2026 •

edited

Loading