Skip to content

fix(llvm): the macOS cfg asks xcrun for the SDK before a fixed CommandLineTools path - #858

Merged
Sunrisepeak merged 1 commit into
mainfrom
fix/macos-llvm-sysroot-xcrun
Sep 18, 2026
Merged

Sunrisepeak merged 1 commit into
mainfrom
fix/macos-llvm-sysroot-xcrun

Conversation

@Sunrisepeak

@Sunrisepeak Sunrisepeak commented Sep 18, 2026 •

Copy link
Copy Markdown
Member

The mechanism

__install_macosx_cfg() in pkgs/l/llvm.lua writes a --sysroot=<path> line
into the installed clang.cfg / clang++.cfg. Until now, whenever SDKROOT
was unset it picked the first directory that exists from a fixed list, with
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk checked before
/Applications/Xcode.app/.../MacOSX.sdk. The Command Line Tools directory
exists on a stock runner/laptop, so that is what got baked in — even on a
machine whose xcode-select is pointed at a working Xcode.app SDK instead.

On the GitHub xcode-27 runner image (2026-09-12+), that Command Line Tools
SDK's .tbd files list an arm64e.x1 architecture that ld64.lld 22.1.8
cannot parse (upstream support lands in llvm-project#222721, after 22.1.8),
so every link against a clang++ configured by this package failed with:

ld64.lld: error: could not load TAPI file at
  /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib/libSystem.tbd: malformed file

Why a command-line -isysroot did not fix it. mcpp's own CI (PR
mcpp-community/mcpp#665) worked around this for its own raw compiles by
passing -isysroot $(xcrun --show-sdk-path) on the compiler command line.
That did not help these two jobs, which had looked like proof the runner
image itself was simply broken. It is not: on Darwin, -isysroot only
redirects the header search path. The linker's syslibroot — which is what
decides which .tbd gets opened — follows --sysroot, and --sysroot is
exactly what this package's cfg file writes. A --sysroot baked into
clang++.cfg at install time wins over a command-line -isysroot for
linking, so mcpp's own fix could never reach a failure caused one layer
below it, in a package mcpp does not own.

The fix. Query xcrun --show-sdk-path — the same canonical probe mcpp's
own SDK selection tries first (modules/platform/src/macos/macos.cppm,
sdk_path()) — after the existing SDKROOT env check and before the fixed
candidate list, which now only runs when xcrun cannot answer (absent,
non-zero exit, empty output, or a path that does not actually exist —
__xcrun_sdk_path() checks os.isdir on what it gets back).

Runner evidence xcrun answers correctly here. The same failing CI job's
own "System info" and "SDK mcpp selects" steps run xcrun --show-sdk-path
on the exact xcode-27 image and print
/Applications/Xcode_27_Release_Candidate.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk
— a working SDK, not the Command Line Tools one (mcpp-community/mcpp run
35288886139). So xcrun is not itself blind on this runner; only this
package's own fixed-path preference was.

What I verified, and what I could not

I authored and tested this on Linux, where __install_macosx_cfg() cannot
run (it only executes when os.host() == "macosx"), so I personally never
watched it choose a working sysroot on macOS. What closes that gap is this
PR's own CI, on real macOS runners (see below) — I did not have to assert
it worked, it's checked.

Before pushing, on Linux, I verified:

  • This repo's static + isolation pytest suites are unaffected:
    pytest tests/ -m static (2612 passed, 14 skipped, unchanged) and
    pytest tests/l/test_llvm.py -m "static or isolation" (12 passed).

  • The file still parses under lua5.4 loadfile(...).

  • The sysroot-selection logic itself, in isolation: I loaded the real,
    unmodified pkgs/l/llvm.lua into a Lua 5.4 harness that stubs only the
    xmake-provided globals (path, pkginfo, log, xvm, import,
    os.isdir/os.host/os.tryrm/os.mv, io.writefile) and calls
    __install_macosx_cfg() directly, comparing its output against the
    pre-change version of the same function under matching fixtures:

    • xcrun on PATH and answering a real, existing directory → new code
      picks that directory (the fix doing its job).
    • xcrun absent, nothing exists anywhere (no Xcode at all) → byte-for-
      byte identical
      cfg output to the old code (no --sysroot line, same
      warning).
    • xcrun absent, only the CommandLineTools path exists → identical
      to the old code (CLT path chosen).
    • xcrun absent, only the Xcode.app path exists → identical to the
      old code (Xcode.app path chosen).
    • xcrun present but prints an empty line (exit 0) → falls through to
      the fixed candidate list, same as if xcrun were absent.
    • xcrun present but exits non-zero with no output → same fall-through.
    • xcrun present but prints a path that does not exist (stale/bogus
      answer) → rejected by the os.isdir check, same fall-through.
    • SDKROOT env var set → still wins over everything, unchanged.

    In short: a machine with no Xcode installed at all, or one where xcrun
    cannot answer for any reason, lands on exactly the cfg this package wrote
    before this change. The only behavior that changes is the case where
    xcrun can answer and its answer differs from the old fixed-path guess
    — which is precisely the runner where this bug shows up.

After pushing, this PR's own CI ran the real macOS install path and it
is green:

  • macos-install-test (.github/workflows/ci-test.yml, macos-latest)
    installs and uninstalls xim:llvm@22.1.8 for real via xlings install,
    exercising __install_macosx_cfg() on real macOS. It passed — no
    "SDK path not detected" warning appeared in the log, meaning the sysroot
    selection resolved to something, and every post-install shim check
    (clang, lld, etc.) passed.
  • consumer-smoke (macos-latest, macos) also passed.

I corrected myself here: I originally wrote in this PR that I could not
find a macOS job in this repo's CI at all. That was wrong — I had only
looked at ci-xpkg-test.yml, and the macOS legs live in a different
workflow, ci-test.yml. They exist and both ran clean on this branch.

What this does NOT prove: macos-latest is not xcode-27, so this
run does not reproduce the broken-CLT-SDK runner itself, and the log does
not print which sysroot path __install_macosx_cfg() actually chose (the
job installs then uninstalls in the same step, and nothing dumps
clang++.cfg's contents before removal). So this confirms the code path
runs cleanly end-to-end on real macOS and does not regress a normal
install — it does not, by itself, confirm the xcode-27 CLT-SDK bug is
gone. That confirmation is downstream, described next.

Where the real end-to-end confirmation comes from

mcpp resolves this package from main on every CI run
(.github/actions/setup-macos-llvm → xlings install llvm), so the actual
proof this works is downstream: mcpp-community/mcpp's "macOS ARM64 — xlings
LLVM end-to-end (xcode-27)" and "e2e suite (macOS ARM64, self-host,
xcode-27)" jobs turning green on a rerun once this merges. Not requesting
merge in this PR — flagging for review first.

…dLineTools path

__install_macosx_cfg wrote --sysroot into clang.cfg/clang++.cfg by checking a
fixed candidate list, /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
first, whenever SDKROOT was unset. On the GitHub xcode-27 runner image
(20260912+) that SDK's .tbd files list an arm64e.x1 architecture that
ld64.lld 22.1.8 cannot parse (llvm-project#222721 lands the support, after
22.1.8), so every link against a clang++ built from this cfg failed with:

    ld64.lld: error: could not load TAPI file at
    .../CommandLineTools/SDKs/MacOSX.sdk/usr/lib/libSystem.tbd: malformed file

This broke even mcpp's own raw-compile CI steps, which pass a working
`-isysroot $(xcrun --show-sdk-path)` on their own command line: on Darwin
-isysroot only redirects header search, not the linker's syslibroot, so it
cannot override a --sysroot baked into the driver's cfg file. mcpp worked
around this on its own compiles (mcpp#665) but had no way to reach into a
package it does not own.

Without this change, any consumer of the llvm package's clang++.cfg on a
host that has a working Xcode.app SDK *and* a broken CommandLineTools SDK
still gets the broken one, because the CommandLineTools path was checked
unconditionally before anything asked what the active developer directory
actually is.

Query `xcrun --show-sdk-path` (the same canonical probe mcpp's own SDK
selection tries first, modules/platform/src/macos/macos.cppm sdk_path())
before falling back to the fixed candidate list, which now only matters when
xcrun itself cannot answer.

Verified: this repo's static + isolation pytest suites (tests/l/test_llvm.py
and the full tests/ -m static run) pass unchanged, and the file still parses
under lua5.4. The macOS install/config path itself could not be exercised --
this machine is Linux and __install_macosx_cfg only runs on os.host() ==
"macosx".
@Sunrisepeak
Sunrisepeak merged commit a96ab1b into main Sep 18, 2026
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant