deploy: use single worker thread in csi-provisioner for NVMe-oF - #6538
Conversation
nixpanic
added
the
ci/skip/multi-arch-build
There was a problem hiding this comment.
🟢 Approval recommended
The change is a minimal, targeted manifest flag adjustment with no apparent side effects outside NVMe-oF provisioning concurrency.
Pull request overview
Updates the NVMe-oF Kubernetes deployment to serialize provisioning operations by limiting the external-provisioner to a single worker thread, aligning with the NVMe-oF gateway’s inability to process configuration/provisioning requests in parallel.
Changes:
- Configure
csi-provisionerwith--worker-threads=1in the NVMe-oF provisioner manifest.
File summaries
| File | Description |
|---|---|
| deploy/nvmeof/kubernetes/csi-nvmeofplugin-provisioner.yaml | Sets csi-provisioner worker concurrency to 1 to avoid parallel CreateVolume requests for NVMe-oF. |
Review details
- Files reviewed: 1/1 changed files
- Comments generated: 0
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Merge Protections🟢 All 2 merge protections satisfied — ready to merge. Show 2 satisfied protections🟢 ⛓️ Depends-On RequirementsRequirement based on the presence of
🟢 🚦 Auto-queueWhen all merge protections are satisfied, this pull request will be queued automatically. |
|
Wait for #6536 to be merged. |
|
/queue |
|
@Mergifyio rebase |
🛑 The pull request rule doesn't match anymoreDetailsThis action has been cancelled. |
The NVMe-oF gateway does not handle configuration requests like volume creation in parallel. This causes multi volume creation to receive failures, and Kubernetes retries with some backoff delays. Instead of the failures returned by the CSI-driver, it is more efficient to allow only a single CreateVolume at the time. This can be configured by passing --worker-threads=1 to the external-provisioner. With a single worker thread, volume creation is very stable, and performance improves a lot. Because of the serialization, multi-volume creation does take a while, but this is a limitation of the NVMe-oF gateway. See-also: ceph/ceph-csi-operator#600 Signed-off-by: Niels de Vos <ndevos@ibm.com>
ce4266c to
5831f90
Compare
ceph-csi-bot
added
ok-to-test
|
/test ci/centos/upgrade-tests-cephfs |
|
/test ci/centos/k8s-e2e-external-storage/1.34 |
|
/test ci/centos/k8s-e2e-external-storage/1.35 |
|
/test ci/centos/upgrade-tests-rbd |
|
/test ci/centos/k8s-e2e-external-storage/1.36 |
|
/test ci/centos/mini-e2e-helm/k8s-1.34 |
|
/test ci/centos/mini-e2e-helm/k8s-1.35 |
|
/test ci/centos/mini-e2e-helm/k8s-1.36 |
|
/test ci/centos/mini-e2e/k8s-1.34 |
|
/test ci/centos/mini-e2e/k8s-1.35 |
|
/test ci/centos/mini-e2e/k8s-1.36 |
ceph-csi-bot
added
ci/in-progress/e2e
mergify
Bot
removed
the
ci/in-progress/e2e
|
Tick the box to add this pull request to the merge queue (same as
|
🛑 This pull request comes from a fork and cannot be rebased on behalf of a userDetailsGitHub refuses an OAuth token on its rebase API for a fork, so rebasing a fork's pull request on behalf of a GitHub user means impersonating that user to force-push the contributor's branch. Mergify does not do that. Unset |
The NVMe-oF gateway does not handle configuration requests like volume
creation in parallel. This causes multi volume creation to receive
failures, and Kubernetes retries with some backoff delays.
Instead of the failures returned by the CSI-driver, it is more efficient
to allow only a single CreateVolume at the time. This can be configured
by passing --worker-threads=1 to the external-provisioner.
With a single worker thread, volume creation is very stable, and
performance improves a lot. Because of the serialization, multi-volume
creation does take a while, but this is a limitation of the NVMe-oF
gateway.
See-also: ceph/ceph-csi-operator#600
CI job ordering
Depends-on: #6536