Repository navigation
Conversation
…orbidden
The startup probe against Secrets("default").List() only assigned
controllerIndexers.secretIndexer on the success path. Whenever the
probe failed with a non-401 StatusError (e.g. 403 Forbidden, which is
exactly what happens when useSecretsForPrivateRepositories is set to
false and the ClusterRole grants no access to secrets), secretIndexer
was left nil.
The secrets informer is started by informerFactory.Start() regardless
of this probe, so nothing prevents GetImagePullSecrets from later
calling ci.secretIndexer.GetByKey() on any workload that has an
imagePullSecret configured, causing a SIGSEGV.
Assign secretIndexer unconditionally: the underlying store just stays
permanently empty when RBAC denies access, which is the degraded mode
the anonymous/RBAC-restricted setup was already intended to support.
Fixes deckhouse#343
Signed-off-by: joris <344303+jorisd@users.noreply.github.com>
jorisd
force-pushed
the
fix-nil-secretindexer-panic
branch
from
September 23, 2026 18:00
95bb284 to
e696892
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #343.
Root cause
In
NewChecker(pkg/registry/checker.go), the startup probe againstSecrets("default").List()only assignscontrollerIndexers.secretIndexeron the success branch:When
useSecretsForPrivateRepositories: falseis set, the ClusterRole grants no access to secrets, so the probe fails with a 403StatusError— which falls into theisStatusbranch, not the 401 sub-case, sosecretIndexeris never assigned and staysnil.The secrets informer is still started unconditionally by
informerFactory.Start(stopCh)a few lines below, regardless of this probe's outcome. Nothing else guards againstsecretIndexerbeing nil, so as soon as any watched workload has animagePullSecretconfigured,GetImagePullSecrets(pkg/registry/indexers.go) callsci.secretIndexer.GetByKey(ref)on a nilcache.Indexerand panics with a SIGSEGV. The pod then crash-loops permanently, since the same workload set is reconciled on every restart.Fix
Assign
secretIndexerunconditionally, right after the probe, instead of only on the success path. The underlying store simply stays permanently empty when RBAC denies access — which is the degraded/anonymous behavior theuseSecretsForPrivateRepositories: falsesetting is already meant to support (per the existing401warning message), it just never got wired up for the "informer running but never synced" case.This reproduces reliably and is confirmed by two independent reports on #343 (v0.13.0 and v0.14.0, both with
useSecretsForPrivateRepositories: false). Credit to @ex-trim for identifying that settinguseSecretsForPrivateRepositories: trueworks around it, and for the initial idea of moving the indexer assignment — this PR keeps the original probe/logging behavior intact and only changes wheresecretIndexergets set.Testing
Manually verified against our own deployment: with
useSecretsForPrivateRepositories: falseand at least one workload carrying animagePullSecret, the unpatched binary panics within a few reconcile cycles; same workload set with this change applied no longer panics (secret lookups just resolve to "not found", as expected for the RBAC-restricted mode).