Summary
When an Application renders custom resources whose CRD is not installed in the ephemeral cluster, those resources lose their metadata.namespace. If the Application contains several same-named CRs in different namespaces, they collapse into a single resource — and the extra copies are silently dropped from the diff.
The result is a false negative: a change confined to one of the dropped copies produces no diff at all.
This is easy to miss because it degrades gracefully — you get a plausible-looking diff, just an incomplete one.
Root cause
In pkg/extract/extract.go:
// IsNamespaced returns true if the given GroupKind is namespaced
func (p *resourceInfoProvider) IsNamespaced(gk schema.GroupKind) (bool, error) {
return p.namespacedByGk[gk], nil
}
namespacedByGk is populated by argocd.K8sClient.GetListOfNamespacedScopedResources(), which discovers kinds from the ephemeral cluster's API server. That cluster only ever has Argo CD installed — the tool renders manifests, it never syncs them, so no application CRDs are ever registered.
For any CRD-backed kind, the map lookup misses and Go returns the zero value false — meaning "cluster-scoped". controller.DeduplicateTargetObjects then clears the namespace and deduplicates on group/kind//name, merging resources that are genuinely distinct.
The mismatch is between where the scope information comes from (the empty ephemeral cluster) and where the resources are destined (a real cluster that has the CRDs).
Reproduction
An Application that renders three ExternalSecret resources, all named my-secret, in namespaces ns-a, ns-b, ns-c:
Duplicate resource warning: Resource external-secrets.io/ExternalSecret//my-secret appeared 3 times among application resources.
Note the empty namespace in ExternalSecret//my-secret.
With --output-app-manifests, only one copy is written, with metadata.namespace absent. Changing only the copy that gets discarded yields No changes found.
Impact
- False negatives. Changes to a dropped copy are invisible. For a tool used as a PR review gate, a missing diff is worse than a noisy one — reviewers trust the empty result.
- Ambiguous attribution. Even when a change is caught, the diff header reads
ExternalSecret: my-secret with no namespace, so a reviewer can't tell which instance changed.
- Scales with CRD usage. Built-in kinds are unaffected (always present in discovery). Estates leaning on ExternalSecrets, cert-manager Certificates, Traefik middlewares, KEDA ScaledObjects, etc. are the exposed ones.
Verified workaround
Because everything in --secrets-folder is kubectl apply'd to the cluster before rendering, dropping the relevant CRDs in there registers the kinds and fixes it completely:
|
Without CRD |
With CRD in secrets folder |
| Duplicate warnings |
6 |
0 |
| Resources extracted for the app |
15 |
18 |
| Same-named CRs preserved |
1 of 3 |
3 of 3, namespaces intact |
| Diff header |
ExternalSecret: my-secret |
ExternalSecret: ns-c/my-secret |
| Runtime |
49s |
45s |
No measurable cost. It works, but it relies on a side effect of the secrets folder, and it requires knowing in advance which CRDs matter.
Suggested fixes
Roughly in order of preference:
- Trust the manifest. If a resource carries an explicit
metadata.namespace, preserve it rather than letting an unknown-kind lookup clear it. Discovery should fill in a missing namespace, not override a declared one.
- Don't silently default unknown kinds to cluster-scoped.
IsNamespaced could return an error, or the caller could treat unknown kinds as namespaced (the safer default — the overwhelming majority of CRDs are namespaced).
- Warn loudly. The existing warning says "duplicate", which reads like a user misconfiguration. It would be clearer as something like "CRD not installed in the preview cluster; namespace information for
<kind> was discarded and N resources were merged — this diff may be incomplete."
- Document it. A first-class flag or documented convention for pre-loading CRDs would beat relying on the secrets folder.
Option 1 alone would resolve the false negative for anything that sets its namespace explicitly, which is the common case for raw manifests.
Environment
argocd-diff-preview v0.2.12 (binary, Darwin-aarch64)
- Argo CD Helm chart 10.2.3 (app v3.5.0),
--cluster kind, kind v0.32.0
- Default
--render-method server-api
Happy to test a patch — this reproduces reliably for me.
Summary
When an Application renders custom resources whose CRD is not installed in the ephemeral cluster, those resources lose their
metadata.namespace. If the Application contains several same-named CRs in different namespaces, they collapse into a single resource — and the extra copies are silently dropped from the diff.The result is a false negative: a change confined to one of the dropped copies produces no diff at all.
This is easy to miss because it degrades gracefully — you get a plausible-looking diff, just an incomplete one.
Root cause
In
pkg/extract/extract.go:namespacedByGkis populated byargocd.K8sClient.GetListOfNamespacedScopedResources(), which discovers kinds from the ephemeral cluster's API server. That cluster only ever has Argo CD installed — the tool renders manifests, it never syncs them, so no application CRDs are ever registered.For any CRD-backed kind, the map lookup misses and Go returns the zero value
false— meaning "cluster-scoped".controller.DeduplicateTargetObjectsthen clears the namespace and deduplicates ongroup/kind//name, merging resources that are genuinely distinct.The mismatch is between where the scope information comes from (the empty ephemeral cluster) and where the resources are destined (a real cluster that has the CRDs).
Reproduction
An Application that renders three
ExternalSecretresources, all namedmy-secret, in namespacesns-a,ns-b,ns-c:Note the empty namespace in
ExternalSecret//my-secret.With
--output-app-manifests, only one copy is written, withmetadata.namespaceabsent. Changing only the copy that gets discarded yieldsNo changes found.Impact
ExternalSecret: my-secretwith no namespace, so a reviewer can't tell which instance changed.Verified workaround
Because everything in
--secrets-folderiskubectl apply'd to the cluster before rendering, dropping the relevant CRDs in there registers the kinds and fixes it completely:ExternalSecret: my-secretExternalSecret: ns-c/my-secretNo measurable cost. It works, but it relies on a side effect of the secrets folder, and it requires knowing in advance which CRDs matter.
Suggested fixes
Roughly in order of preference:
metadata.namespace, preserve it rather than letting an unknown-kind lookup clear it. Discovery should fill in a missing namespace, not override a declared one.IsNamespacedcould return an error, or the caller could treat unknown kinds as namespaced (the safer default — the overwhelming majority of CRDs are namespaced).<kind>was discarded and N resources were merged — this diff may be incomplete."Option 1 alone would resolve the false negative for anything that sets its namespace explicitly, which is the common case for raw manifests.
Environment
argocd-diff-previewv0.2.12 (binary,Darwin-aarch64)--cluster kind, kind v0.32.0--render-method server-apiHappy to test a patch — this reproduces reliably for me.