From d64edfd43623d425b4bc3b612dc6f58b3e613385 Mon Sep 17 00:00:00 2001 From: Jay Date: Fri, 30 Jan 2026 12:18:16 -0500 Subject: [PATCH 1/3] docs: enhance network volumes documentation with S3 upload instructions and path configuration details - Added important notes on using relative paths for network storage to avoid model detection issues. - Included a section on uploading files via the S3-compatible API, with detailed command examples and explanations. - Clarified the expected directory structure and provided troubleshooting tips for network volume access during testing. --- docs/network-volumes.md | 34 ++++++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+) diff --git a/docs/network-volumes.md b/docs/network-volumes.md index c9e634ce5..98afd3012 100644 --- a/docs/network-volumes.md +++ b/docs/network-volumes.md @@ -24,6 +24,10 @@ If you use the S3-compatible API, the same paths map as: - Pod: `/workspace/my-folder/file.txt` - S3 API: `s3:///my-folder/file.txt` +> **Important: Path on the network storage** +> +> When you upload or configure paths **on the network storage itself** (e.g. via S3), use paths **relative to the volume root**. For example, put models at `/models/[loras, vae, diffusion_models, etc]/file.ext`, **not** `/runpod-volume/models/...`. The worker mounts the volume at `/runpod-volume`, so if you use `/runpod-volume` in the path on the storage, the container will see `runpod-volume/runpod-volume/models/...` and models will not be found. + ## Expected Directory Structure Models must be placed in the following structure on your network volume: @@ -47,6 +51,32 @@ Models must be placed in the following structure on your network volume: > > Only create the subdirectories you actually need; empty or missing folders are fine. +## Uploading files via the S3-compatible API + +You can upload models to your network volume using RunPod’s [S3-compatible API](https://docs.runpod.io/storage/s3-api). You need an **S3 API key** (separate from your RunPod API key): create one under **Settings → S3 API Keys** in the RunPod console. The access key and secret are shown there; the docs explain [setup and authentication](https://docs.runpod.io/storage/s3-api#setup-and-authentication) in detail. + +With the [AWS CLI](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/s3/index.html) configured (or by passing credentials in the environment), upload a file like this: + +```bash +AWS_ACCESS_KEY_ID=your_access_key \ +AWS_SECRET_ACCESS_KEY=your_secret_key \ +aws s3 cp \ + --region DATACENTER \ + --endpoint-url https://s3api-DATACENTER.runpod.io \ + /path/to/local/model.safetensors \ + s3://NETWORK_VOLUME_ID/models/diffusion_models/ +``` + +Replace: + +- `your_access_key` / `your_secret_key` — your RunPod S3 API key (from [RunPod S3 API Keys](https://docs.runpod.io/storage/s3-api#setup-and-authentication)). +- `DATACENTER` — the datacenter where the volume lives (e.g. `eu-ro-1`). [Supported endpoints](https://docs.runpod.io/storage/s3-api#datacenter-availability) are listed in the RunPod S3 API docs. +- `NETWORK_VOLUME_ID` — your network volume ID (from the RunPod console). +- `/path/to/local/model.safetensors` — local path to the file to upload. +- `models/diffusion_models/` — path on the volume (relative to the volume root; do **not** use `/runpod-volume/` here). + +The file will then appear at `/runpod-volume/models/diffusion_models/model.safetensors` inside the worker. + ## Supported File Extensions ComfyUI only recognizes files with specific extensions when scanning model directories. @@ -76,6 +106,10 @@ Files with other extensions (for example `.txt`, `.zip`) are **ignored** by Comf If any of the above is true, ComfyUI will silently fail to discover models from the network volume. +### Tests and `runpod-volume` access + +RunPod’s test environment does **not** have access to the network volume mount: the `/runpod-volume` folder is not visible when tests run. When the endpoint starts for real, the volume is mounted and the folder appears. If tests fail or block deployment because of `Value not in list`, you can work around this by removing `.runpod/tests.json` so that the endpoint is deployed without running those tests. + ## Debugging with `NETWORK_VOLUME_DEBUG` The worker exposes an opt‑in debug mode controlled via the `NETWORK_VOLUME_DEBUG` environment variable. From cd720b92265a6982ec6412d9094cab47f8df4a75 Mon Sep 17 00:00:00 2001 From: Jay Date: Fri, 30 Jan 2026 12:18:24 -0500 Subject: [PATCH 2/3] docs: update network volumes documentation for S3 upload paths and model structure - Clarified the usage of relative paths for models on network storage to prevent detection issues. - Updated the section on uploading models via the S3-compatible API with new command examples and path formats. - Adjusted the expected directory structure to reflect changes in model naming conventions. --- docs/network-volumes.md | 24 +++++++----------------- 1 file changed, 7 insertions(+), 17 deletions(-) diff --git a/docs/network-volumes.md b/docs/network-volumes.md index 98afd3012..26b59f9b6 100644 --- a/docs/network-volumes.md +++ b/docs/network-volumes.md @@ -26,7 +26,7 @@ If you use the S3-compatible API, the same paths map as: > **Important: Path on the network storage** > -> When you upload or configure paths **on the network storage itself** (e.g. via S3), use paths **relative to the volume root**. For example, put models at `/models/[loras, vae, diffusion_models, etc]/file.ext`, **not** `/runpod-volume/models/...`. The worker mounts the volume at `/runpod-volume`, so if you use `/runpod-volume` in the path on the storage, the container will see `runpod-volume/runpod-volume/models/...` and models will not be found. +> When you upload or configure paths **on the network storage itself** (e.g. via S3), use paths **relative to the volume root**. For example, put models at `/models/[model]/file_name.ext`, **not** `/runpod-volume/models/...`. The worker mounts the volume at `/runpod-volume`, so if you use `/runpod-volume` in the path on the storage, the container will see `runpod-volume/runpod-volume/models/...` and models will not be found. ## Expected Directory Structure @@ -51,31 +51,21 @@ Models must be placed in the following structure on your network volume: > > Only create the subdirectories you actually need; empty or missing folders are fine. -## Uploading files via the S3-compatible API +## Uploading models via the S3-compatible API You can upload models to your network volume using RunPod’s [S3-compatible API](https://docs.runpod.io/storage/s3-api). You need an **S3 API key** (separate from your RunPod API key): create one under **Settings → S3 API Keys** in the RunPod console. The access key and secret are shown there; the docs explain [setup and authentication](https://docs.runpod.io/storage/s3-api#setup-and-authentication) in detail. -With the [AWS CLI](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/s3/index.html) configured (or by passing credentials in the environment), upload a file like this: - ```bash AWS_ACCESS_KEY_ID=your_access_key \ AWS_SECRET_ACCESS_KEY=your_secret_key \ aws s3 cp \ - --region DATACENTER \ - --endpoint-url https://s3api-DATACENTER.runpod.io \ - /path/to/local/model.safetensors \ - s3://NETWORK_VOLUME_ID/models/diffusion_models/ + --region [REGION] \ + --endpoint-url ENDPOINT_URL \ + /path/to/local/file.ext \ + s3://BUCKET_NAME/models/[model]/file_name.ext ``` -Replace: - -- `your_access_key` / `your_secret_key` — your RunPod S3 API key (from [RunPod S3 API Keys](https://docs.runpod.io/storage/s3-api#setup-and-authentication)). -- `DATACENTER` — the datacenter where the volume lives (e.g. `eu-ro-1`). [Supported endpoints](https://docs.runpod.io/storage/s3-api#datacenter-availability) are listed in the RunPod S3 API docs. -- `NETWORK_VOLUME_ID` — your network volume ID (from the RunPod console). -- `/path/to/local/model.safetensors` — local path to the file to upload. -- `models/diffusion_models/` — path on the volume (relative to the volume root; do **not** use `/runpod-volume/` here). - -The file will then appear at `/runpod-volume/models/diffusion_models/model.safetensors` inside the worker. +The file will then appear at `/runpod-volume/models/[model]/file_name.ext` inside the worker. ## Supported File Extensions From e5dc845ee87852a097ea37e63f74bd310167f9ed Mon Sep 17 00:00:00 2001 From: Jay Mendapara Date: Tue, 2 Jun 2026 10:46:54 -0400 Subject: [PATCH 3/3] docs: address review feedback on network-volume guidance - Update test workaround to reflect .runpod/tests_.json rename - Add changeset for the docs change Co-Authored-By: Claude Opus 4.8 (1M context) --- .changeset/docs-network-volume-guidance.md | 5 +++++ docs/network-volumes.md | 2 +- 2 files changed, 6 insertions(+), 1 deletion(-) create mode 100644 .changeset/docs-network-volume-guidance.md diff --git a/.changeset/docs-network-volume-guidance.md b/.changeset/docs-network-volume-guidance.md new file mode 100644 index 000000000..c06695c9c --- /dev/null +++ b/.changeset/docs-network-volume-guidance.md @@ -0,0 +1,5 @@ +--- +"worker-comfyui": patch +--- + +docs: clarify network-volume path semantics (paths relative to volume root), document S3-compatible upload flow, and note the `.runpod/tests_.json` rename for tests that fail without the volume. diff --git a/docs/network-volumes.md b/docs/network-volumes.md index 26b59f9b6..b6ef84a19 100644 --- a/docs/network-volumes.md +++ b/docs/network-volumes.md @@ -98,7 +98,7 @@ If any of the above is true, ComfyUI will silently fail to discover models from ### Tests and `runpod-volume` access -RunPod’s test environment does **not** have access to the network volume mount: the `/runpod-volume` folder is not visible when tests run. When the endpoint starts for real, the volume is mounted and the folder appears. If tests fail or block deployment because of `Value not in list`, you can work around this by removing `.runpod/tests.json` so that the endpoint is deployed without running those tests. +RunPod’s test environment does **not** have access to the network volume mount: the `/runpod-volume` folder is not visible when tests run. When the endpoint starts for real, the volume is mounted and the folder appears. The repository ships with the tests file renamed to `.runpod/tests_.json` so they don't run by default; if you've re-enabled them and they fail with `Value not in list`, remove or rename the file again so the endpoint is deployed without running those tests. ## Debugging with `NETWORK_VOLUME_DEBUG`