Feature request | Render base and target branches with different Argo CD versions

[fouadchamoun]

Problem

Today, diff preview could be misleading in case ArgoCD itself is being updated.

For example, the recent ArgoCD Helm chart update from 10.2.2 to 10.2.3 (seemingly patch) upgraded ArgoCD from 3.4.6 to 3.5.0, which brought with it an upgrade from Helm 3 to Helm 4.

The Helm upgrade has a hidden breaking change that affects "null" overrides in values, and thus resulted in unwanted rendering changes in some of my apps.

Those changes were invisible in the ArgoCD diff because the base and target apps were rendered by the same ArgoCD version (the new one).

Proposed solution

Add the ability to render the base and target apps with different ArgoCD versions.

We could add a --argocd-chart-version-base.

The original --argocd-chart-version argument would keep functioning as the "target" version and would be the default for the "base" version when not specified.

Internally, the tool would:

1. install base version of ArgoCD
2. render base apps
3. if target ArgoCD version different than base version, then upgrade the helm chart installation (no need to uninstall/reinstall i think ?)
4. render target apps

Limits

• this would only work in Ephemeral Cluster Mode
• i have not yet checked what other internal logic would be impacted

Contributing

I would be happy to try to contribute this, in case you think it's a good idea :)

Current workaround

I run the tool twice and extract the output manifests for base and target from the corresponding run.
I then have to rebuild the diff.md from scratch, and thus lose the useful built-in diff formatting.
This feels very clunky and hacky.
Workaround code: https://github.com/fouadchamoun/homelab/blob/05f5377bcfec8ccae3076fa28498fa32d45fe491/.github/workflows/generate-diff.yml
Example output from workaround: fouadchamoun/homelab#409 (comment)
(you can see the unwanted diff in one app, caused by the hidden Helm update)

[dag-andersen]

Hi @fouadchamoun, thanks for the detailed feature request and for sharing your workaround. It is an interesting use case!

The project has been around for almost two years, and this is the first request we have received for this behavior. I am therefore not yet sure how broadly it is needed. However, I fully understand that there may be cases where this would be useful.

Let's see if this issue gets any upvotes :)

The Helm upgrade has a hidden breaking change that affects "null" overrides in values, and thus resulted in unwanted rendering changes in some of my apps.

Would you mind showing me an Argo CD application that broke? We upgraded to Argo CD 3.5 recently and did not see any issues at all, so I am wondering what triggered it on your side :)

i have not yet checked what other internal logic would be impacted

Currently, the implementation renders both the base and target branches at the same time to speed things up, instead of processing the branches sequentially. Unfortunately, I think supporting this would require quite a big rewrite of the sequential steps in the code... however, it can be done.

Thank you again for sharing your pipeline. It is always valuable to see how people use the tool and work around its current limitations.

[fouadchamoun]

Hi @dag-andersen, thanks for taking the time to write back!

Let's see if this issue gets any upvotes :)

You're right, it would only make sense to work on it if it affects more people :)

Would you mind showing me an Argo CD application that broke?

here fouadchamoun/homelab#409 (comment) you can see the diff comment mentioning the homebridge application suddenly getting extra securityContext fields.
that's because it's based on a "base-app" local chart that sets these fields by default, and the values.yaml for homebridge overrides them with null values.

This is the issue mentioning the breaking change on the ArgoCD side: argoproj/argo-cd#29068 .

Unfortunately, I think supporting this would require quite a big rewrite

maybe it would be easier to find a way to make the current workaround (double invocation) less "hacky".

[dag-andersen]

@fouadchamoun Ah thank you - this issue now makes sense to me :)

i am surprised that you get all these:

       annotations:
-        checksum/cmd-params: 9b9656c9eabf2b9dd562dd3f413aa3a092f27e4f936896932b10e45c10dcd86b
+        checksum/cmd-params: 274768187b3760826ce94e0b36235386eb8aa2d70771a187b75a8b84509fb27e
       labels:

Because those kind of lines should be ignored by default 😮

[fouadchamoun]

i am surprised that you get all these:
Because those kind of lines should be ignored by default 😮

probably because i'm just doing a regular diff : https://github.com/fouadchamoun/homelab/blob/05f5377bcfec8ccae3076fa28498fa32d45fe491/.github/workflows/generate-diff.yml#L128