Skip to content

[BUG] DOWNLOAD_ONLY seems to ignore overrides #1088

Description

@msarahan

Context

Current issues

I was confused on some standalone builds. One job out of 4 failed: https://prod.blsm.nvidia.com/sw-rapids-cuvs-static/blue/organizations/jenkins/standalone-builds%2Frelease-26.10-bundle/detail/release-26.10-bundle/26/pipeline/364, but the other other 3 builds for that project passed. I asked Codex, and here's what it came up with:

The failure was not lane-specific, and the remap failed in all four cuDF jobs.
The difference was GitHub retry luck:

  • amd64 / CUDA 12.9: failed once, then cloned successfully.
  • amd64 / CUDA 13.4: failed twice, then cloned successfully.
  • arm64 / CUDA 12.9: cloned successfully immediately.
  • arm64 / CUDA 13.4: failed all three attempts and aborted.
    The underlying gap is DLPack’s DOWNLOAD_ONLY TRUE declaration. With CMake 4.2 and our vendored CPM 0.42.0, that path calls FetchContent_Populate(... ${ARGN}) with cuDF’s original GitHub arguments. It bypasses the source details pre-registered by the RAPIDS override. Other dependencies use the normal FetchContent path and are remapped correctly.
    So Robert’s description is generally correct, but our earlier verification missed this specific CPM + DOWNLOAD_ONLY + CMake >= 3.30.3 path. The successful jobs merely concealed it through intermittent GitHub access.
    We need to fix that deterministic gap, likely in the vendored CPM behavior or with an explicit DLPack CPM declaration. Simply rerunning could pass, but it would remain unreliable.

followup

Me: Would a different cmake or CPM version make a difference?

Codex answered

Yes, but only a CMake downgrade would change this behavior today.

  • CMake 3.30.2 or older: CPM uses FetchContent_Populate(name), which consumes the previously registered override. This should make the DLPack remap work.
  • CMake 3.30.3 through our current 4.2.3: CPM’s DOWNLOAD_ONLY path uses the full-arguments form, which ignores saved FetchContent details and therefore restores cuDF’s GitHub URL.
  • Newer CPM: CPM 0.42.1–0.42.3 and the current 0.43.1/master retain the same relevant implementation, so upgrading CPM alone will not fix it. CPM 0.43.1 implementation
    CMake documents that the full-arguments FetchContent_Populate() form ignores previously saved declaration details, which explains the behavior exactly. CMake FetchContent documentation
    I would not downgrade CMake for this. The better fix is narrowly addressing CPM/RAPIDS CMake’s DOWNLOAD_ONLY interaction so it propagates the override into CPM’s own declaration. Until that exists upstream, a targeted DLPack workaround is safer than changing the build’s CMake version.
    ve

Activity

  1. robertmaynard commented on Sep 15, 2026

    @robertmaynard
    Contributor

    Digging deeper into a root cause:

    Inside CPMAddPackage(), cpm_fetch_package() (CPM_0.42.0.cmake:1180) branches on DOWNLOAD_ONLY:

    • DOWNLOAD_ONLY=FALSE → calls FetchContent_MakeAvailable(dlpack). Because the override already declared dlpack first, CMake ignores CPM's redundant FetchContent_Declare(... v0.8 ...) and populates using the override's saved details (v1.3). Override works correctly.
    • DOWNLOAD_ONLY=TRUE → calls FetchContent_Populate(dlpack SOURCE_DIR ... GIT_REPOSITORY ... GIT_TAG v0.8 ...) directly, passing the v0.8 arguments explicitly as ${ARGN}, instead of going through the declared/saved details. FetchContent_Populate given explicit args ignores the previously saved (overridden) details entirely and fetches whatever was passed to it — v0.8, not the override's v1.3.

    So the divergence is in cpm_fetch_package() in CPM.cmake: the DOWNLOAD_ONLY branch calls FetchContent_Populate() with explicit arguments (bypassing the "first-declare-wins" override mechanism), while the non-DOWNLOAD_ONLY branch calls FetchContent_MakeAvailable() with no arguments (which correctly uses the saved/overridden declaration).

  2. msarahan commented on Sep 15, 2026

    @msarahan
    ContributorAuthor

    Is this a bug in CPM that we should fix, or a gotcha that we need to engineer around?

  3. robertmaynard commented on Sep 15, 2026

    @robertmaynard
    Contributor

    The FetchContent_Populate is never meant to be used by projects like CPM this way. It goes against the documented supported behavior of only being designed for CMake scripting mode.

    What CPM should have done is leverage SOURCE_SUBDIR which documents how todo this behavior:

     The path provided with ``SOURCE_SUBDIR`` must be relative,
    and it will be treated as relative to the top directory.  It can also
    point to a directory that does not contain a ``CMakeLists.txt`` file,
    or even to a directory that doesn't exist.  This can be used to avoid
    adding a project that contains a ``CMakeLists.txt`` file in its top
    directory.
    

    So what CPM should do when given DOWNLOAD_ONLY=ON is setup a SOURCE_SUBDIR with a directory name that is impossible to exist

  4. robertmaynard commented on Sep 15, 2026

    @robertmaynard
    Contributor

    I am working on a work around for rapids-cmake currently, and we can upstream a proper fix.

  5. robertmaynard commented on Sep 15, 2026

    @robertmaynard
    Contributor

    #1089 addresses this issue

  6. added a commit that references this issue on Sep 16, 2026
    8314a83
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    ? - Needs TriageNeed team to review and classifybugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions