Add Windows build support with MSVC/clang-cl and vcpkg - #450
Merged
David Williams-Young (wavefunction91) merged 152 commits intoJun 26, 2026
Merged
David Williams-Young (wavefunction91) merged 152 commits into
David Williams-Young (wavefunction91) merged 152 commits into
Conversation
Two issues caused 27 test failures on Windows: 1. NATIVE_ULONG / size_t mismatch: On Windows LLP64, unsigned long is 4 bytes while size_t is 8 bytes. Use NATIVE_UINT64 instead, which is unambiguously 64-bit on all platforms. 2. NATIVE_HBOOL / bool mismatch: HDF5's hbool_t is unsigned int (4 bytes), but C++ bool is 1 byte. Reading/writing a bool* with NATIVE_HBOOL is undefined behavior. Use hbool_t intermediaries with static_cast at each serialization site.
On Windows, std::filesystem::remove() fails if the file still has an open handle — unlike Linux, which allows unlinking open files. Two tests opened an std::ifstream to verify FCIDUMP file contents but called remove() while the stream was still in scope, causing: "remove: The process cannot access the file because it is being used by another process" Fix by scoping each ifstream in a block so its destructor closes the file handle before the remove() call. Affected tests: - HamiltonianTest.SparseContainerFCIDUMP - HamiltonianTest.FCIDUMPActiveSpaceConsistency
Shell::from_json() used default initialization (`Shell sh`), leaving
the rpowers array indeterminate for non-ECP shells — the JSON round-
trip never writes rpowers for regular shells. Both BasisHasher and
BasisEqChecker read rpowers unconditionally, so comparing a database-
loaded shell (zero-initialized via `Shell sh{0}`) against a JSON-
deserialized shell was undefined behavior. Fix by value-initializing
(`Shell sh{}`), matching the pattern already used elsewhere.
Also fix BasisSetMap test which had three bugs:
- Reversed basis_json["shells"] but the actual key is "electron_shells"
(operator[] silently created an empty entry, making the reverse a no-op)
- Used Hydrogen (1 shell in STO-3G), so even reversing the correct key
would be a no-op; switch to Lithium which has 3 shells
- Test only passed on Linux by accident: uninitialized rpowers garbage
happened to make the hashes differ
Three issues prevented the Python package from building and importing on Windows when linking to a pre-installed C++ library: 1. pybind11 not found: vcpkg's find_package wrapper intercepts the search and misses pybind11 installed in pip's isolated build environment. Fix by querying `python -m pybind11 --cmakedir` and setting pybind11_DIR explicitly before find_package. 2. DLL load failure at import: Python 3.8+ no longer searches PATH for DLL dependencies of extension modules. Add a Windows-only block in __init__.py that reads the QDK_DLL_DIR environment variable and calls os.add_dll_directory() for each path before _core is imported. 3. Test script updates: use Ninja generator, clang-cl compilers, vcpkg toolchain, CMAKE_PREFIX_PATH for the pre-built C++ library, uv for venv management, and QDK_DLL_DIR for runtime DLL resolution.
In Windows the default encoding is cp1252, not UTF-8. This caused tests to fail.
NamedTemporaryFile with the default delete=True keeps an exclusive file handle on Windows, preventing C++ code from opening the same path. Use delete=False so the handle is released when the with-block exits, then clean up manually with Path.unlink() in a finally block. This pattern is needed wherever a temp file is created in Python and then passed to the C++ library for reading or writing. Affected tests: test_basis_set, test_orbitals, test_stability, test_noise_models (24 instances total). Note: when the minimum Python version is raised to 3.12+, this can be simplified using delete_on_close=False, which releases the file handle on close but auto-deletes on context manager exit — removing the need for manual unlink() calls.
The check_license_headers.py script prints a ✅ emoji on success, but Python defaults to cp1252 encoding for stdout on Windows, which cannot encode it. Reconfigure stdout to UTF-8 at the top of the script.
On Windows, spdlog::stdout_color_mt() creates a wincolor_stdout_sink that caches a Windows HANDLE via GetStdHandle(STD_OUTPUT_HANDLE) at construction time. All subsequent writes go through WriteFile(HANDLE) which bypasses the C runtime's file descriptor layer entirely. When pytest's capfd redirects fd 1 via dup2(), the cached HANDLE still points to the original stdout, so all Logger output bypasses capture — causing ~25 test failures across test_logger.py, test_constants_documentation.py, and test_scf.py. On Linux this is not a problem because stdout_color_sink_mt is aliased to ansicolor_stdout_sink, which writes via fwrite(stdout) through the C runtime's fd layer. Add a custom stdout_fd_sink for Windows that writes via fwrite(stdout), giving Windows the same fd-based write path that Linux gets by default. The sink is used only on Windows (#ifdef _WIN32); Linux/macOS continue to use stdout_color_mt() unchanged.
Windows uses cp1252 as the default locale encoding, which silently corrupts non-ASCII text in open(), read_text(), write_text(), and subprocess.run(text=True) calls. All text I/O in the package and its test suite now passes encoding="utf-8" explicitly, and subprocess invocations additionally propagate PYTHONIOENCODING=utf-8 to child processes so that both parent and child agree on the codec regardless of the system locale. A dedicated round-trip regression test verifies that Unicode content (φ, Å, ΔE, αβγ, 你好) survives file serialization and stdout capture end-to-end on Windows.
The uv-installed CPython 3.14 on Windows ships with an incomplete Tcl/Tk installation (missing tcl_findLibrary). Matplotlib defaults to the TkAgg backend, which crashes when trying to create a Tk window during circuit diagram plotting tests. Force the non-interactive Agg backend in conftest.py before any pyplot import. This is standard practice for CI/headless test environments and avoids the Tk dependency entirely.
This eliminates CRLF warnings on Windows and ensures consistent line endings across platforms.
Run each example script in a fresh TemporaryDirectory so output files (e.g. .h5, .json) don't pollute the source tree. On failure the temp directory is preserved for debugging; on success it is cleaned up.
Using more than 2 OpenMP threads causes some tests to fail (SCF not converging, ...) due to numerical instabilities. See: https://gist.github.com/lorisercole/c8081e5f4966c6bf9a2e64734c781fd3 The culprit appears to be GauXC's XC integrator, that uses element-by-element #pragma omp atomic accumulation on shared matrices (inc_by_submat_atomic in util.hpp). Under LLVM's libomp this causes non-deterministic floating-point summation order, leading to NaN/divergence in SCF with >2 threads. The issue does not manifest with GCC/libgomp due to its more conservative thread scheduling. An issue on GauXC repo will be opened. Temporary fix: disable GAUXC_ENABLE_OPENMP on MSVC while keeping OpenMP enabled for the rest of the project (MACIS, our own code).
Loris Ercole (lorisercole)
marked this pull request as ready for review
April 20, 2026 20:34
Guarantees cleanup of the temp run dir on both success and failure (no leftover dirs under docs/examples, no leaked handle on Windows). Drops the manual cleanup()/if-else in favor of a plain assert.
Add example CMake toolchain files under .pipelines/toolchains/ (linux.cmake, macos.cmake, windows.cmake) that set CMAKE_<LANG>_FLAGS_<CONFIG>_INIT before project(), instead of clobbering CMAKE_<LANG>_FLAGS_<CONFIG> after project() in cpp/CMakeLists.txt. The Windows scripts chainload the Windows toolchain via VCPKG_CHAINLOAD_TOOLCHAIN_FILE alongside the vcpkg toolchain. C and CXX flags are set independently; debug-print messages retained for now.
Pass -DCMAKE_TOOLCHAIN_FILE selecting .pipelines/toolchains/{linux,macos}.cmake by matrix.os-name, so the GHA build uses the same per-config flags as local/Windows builds.
CMake's compiler modules string(APPEND) their defaults after the toolchain's CMAKE_<LANG>_FLAGS_<CONFIG>_INIT, so e.g. the default -O2 overrode our -O3 for RelWithDebInfo. Set the per-config cache variables directly (FORCE) so the toolchain flags are authoritative.
Revert the /wd and -Wno- warning-suppression compile options added for the Windows build so the build stays loud: - third_party.cmake: drop the _qdk_suppress_if_built helper and the libint2_cxx /wd4018 /wd4068 /wd4389 INTERFACE flags (keep /Zc:__cplusplus /Zc:preprocessor correctness flags) - lobpcgxx/CMakeLists.txt: drop the /wd suppressions on blaspp/lapackpp (keep /FIcomplex) - windows.cmake toolchain: drop the MS 1CS /w1 warning set (CMake's default /W3 still applies)
Loris Ercole (lorisercole)
changed the title
…TRY_RUN_SLOW_TESTS
Shorten over-long explanatory comments (spdlog stdout sink, tar --force-local, HDF5 LLP64 note, MSVC libint2/ecpint notes, Windows DLL/UTF-8 notes) and remove duplicated NamedTemporaryFile file-lock notes, keeping one explained instance.
- qdk-uarch.cmake: for native MSVC cl (CMAKE_CXX_COMPILER_ID==MSVC), leave QDK_UARCH unset and use the compiler-default ISA, instead of building an invalid /arch:x86-64 that fails the compiler-flag check. clang-cl is unaffected (uses -march). - windows build scripts: clamp NCPUS to a minimum of 1.
…ch on Windows LLP64
Replace unsigned-long literals (0ul) used as size_t accumulators and loop counters with size_t to avoid 32-bit truncation/overflow on Windows (where unsigned long is 4 bytes). Covers std::accumulate/transform_exclusive_scan inits and auto-typed loop counters in asci, hamiltonian_generator, sparsexx and moller_plesset.
David Williams-Young (wavefunction91)
approved these changes
Jun 26, 2026
David Williams-Young (wavefunction91)
enabled auto-merge
June 26, 2026 21:37
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Introduces experimental native Windows support for QDK/Chemistry. The project now builds and passes all tests on Windows with both native MSVC
cl.exeandclang-cl(MSVC Build Tools), using vcpkg for native dependencies.Strategy
Compilers: Two Windows compilers are supported, both targeting the MSVC ABI:
cl.exe- supported via a set of source/header patches for the third-party dependencies (libint2, ecpint).clang-cl- the LLVM Clang frontend with MSVC-compatible ABI. Gives GCC/Clang C++20 feature support while linking against the MSVC runtime.Dependencies: vcpkg in manifest mode (
vcpkg.json), with a custom OpenBLAS overlay port that builds LAPACK from C sources (no Fortran toolchain needed). By default, dependencies are statically linked using thex64-windows-static-mdtriplet - all library code is linked directly into_core.pyd, eliminating DLL bundling and the need to callos.add_dll_directory()at import time. A-DynamicDepsswitch falls back to shared libraries if needed.Build scripts: Two PowerShell scripts under
.pipelines/windows/:windows-build-msvc-cmake.ps1- build with native MSVCcl.exewindows-build-clang-cl-cmake.ps1- build withclang-clBoth handle the full toolchain setup (VS Build Tools detection/installation, vcvarsall environment, vcpkg bootstrap, C++ build + Python pip install) and accept
-BuildType(Release(default),RelWithDebInfo,Debug) and-DynamicDeps.CMake flags: Per-config compile flags (
Debug,Release,RelWithDebInfo) live in per-platform CMake toolchain files under.pipelines/toolchains/(linux.cmake,macos.cmake,windows.cmake), loaded viaCMAKE_TOOLCHAIN_FILE(chainloaded after vcpkg on Windows throughVCPKG_CHAINLOAD_TOOLCHAIN_FILE). Setting them in the toolchain ensures the flags are applied across all build pipelines. No warning suppressions are applied. Coverage flags (--coverage,-fprofile-arcs) are guarded withAND NOT MSVC, sinceclang-cluses MSVC-style option parsing despite reportingCMAKE_CXX_COMPILER_ID=Clang.Testing: vcpkg also provides
catch2andgtest, so C++ tests build on Windows without system-installed test frameworks. AMACIS_ENABLE_TESTSoption controls building MACIS tests (ON by default, overridable via-D).Windows-specific issues discovered and fixed
Data model differences (LLP64 vs LP64)
size_tserialization: On Windows LLP64,unsigned longis 4 bytes (vs 8 on Linux LP64).NATIVE_ULONGwas silently truncating 64-bit values. Fixed by usingNATIVE_UINT64with astatic_assert(sizeof(size_t) == 8)(QDK/Chemistry is only supported on 64-bit platforms).boolserialization:hbool_tisunsigned int(4 bytes) but C++boolis 1 byte. Readingbool*withNATIVE_HBOOLwas UB. Fixed by usinghbool_tintermediaries withstatic_cast.DLL loading (Python 3.8+)
.pyddependencies are only found in the.pyd's own directory or viaos.add_dll_directory(). PATH is not searched. See this comment._core.pyd, like on our Linux/macOS wheels._core.pydvia CMakeinstall(). An explicit list of the required DLLs is installed; the OpenMP runtime (libomp) is excluded. TheQDK_BUNDLE_RUNTIME_DLLSoption (ON by default) auto-detects the vcpkg triplet'sVCPKG_LIBRARY_LINKAGEand can be disabled for CI pipelines usingdelvewheel repair.QDK_DLL_DIRenvironment variable is kept as an escape hatch for non-vcpkg setups.Encoding (cp1252 vs UTF-8)
encoding="utf-8". Q# circuit diagrams contain special characters that fail under cp1252.PYTHONIOENCODING=utf-8to child processes.File locking
NamedTemporaryFile(delete=False)+ manualunlink()) and 2 C++ test cases (scopingstd::ifstreamin blocks so the destructor closes the handle beforestd::filesystem::remove()).spdlog stdout capture
spdlog::stdout_color_mt()caches a Windows HANDLE at construction. When pytest'scapfdredirects fd 1 viadup2(), the cached HANDLE bypasses capture. Added a customstdout_fd_sink(in the logger module) that writes viafwrite(stdout)through the C runtime's fd layer, giving Windows the same behavior Linux gets by default.OpenMP
#pragma omp atomicaccumulation on shared matrices. Under LLVM'slibomp, this causes non-deterministic floating-point summation order, leading to NaN/divergence in SCF with >2 threads. Does not manifest with GCC/libgomp. See this GauXC issue./openmp), which lacks modern constructs the codebase relies on - e.g. atomicupdate/capturememory-order modifiers and array-section reductions - so it cannot compile the OpenMP paths without source-level workarounds.QDK_ENABLE_OPENMPdefaults to OFF onWIN32, the same way it is already disabled for AppleClang on macOS). The earlier approach of enabling OpenMP for the project while disabling it only for GauXC was reverted, along with the OpenMP-on-Windows source workarounds, to keep the C++ code identical to its original (non-Windows) form. Re-enabling OpenMP on Windows is deferred to the future.blaspp/lapackpp on Windows
BLASFinder.cmakenever cachesblaspp_defs_(which holds-DBLAS_FORTRAN_ADD_). On CMake reconfigure, detection is skipped soblaspp_definesbecomes empty, causingdefines.hto lose all Fortran mangling definitions. Fixed via a git patch that cachesblaspp_defs_at detection time and restores it on reconfigure.config.homits#include <complex>, causing compile failures in lobpcgxx. Fixed by force-including<complex>(/FIcomplex) for the lapackpp target on Windows.C++ code quality fixes (exposed by MSVC / clang-cl warnings)
-Wreorder-ctor: Fixed initializer list order to match declaration order in several source files.-Winconsistent-missing-override: Addedoverridespecifiers to virtual function redeclarations inDynamicalCorrelationCalculator,StabilityChecker,Macis, andOneBodyIntegralEnginesubclasses.-Wdefaulted-function-deleted: Explicitly=deletecopy/move assignment inStructure(const data members) and default ctor inERIMultiplexer(base class has no default ctor).Shell::rpowers:Shell::from_json()used default initialization leavingrpowersindeterminate for non-ECP shells. Fixed by value-initializing (Shell sh{}). Also fixed theBasisSetMaptest which had the wrong JSON key ("shells"instead of"electron_shells") and used a single-shell element making the test a no-op.CMake / compiler flags
_USE_MATH_DEFINESrequired forM_PIetc. under MSVC.-fPICskipped on Windows (position-independent code is the default for DLLs).Misc
.gitattributesenforcing LF line endings repo-wide.Agg) for headless test environments (avoids Tcl/Tk dependency crash on Windows).uint128_tpolyfill: added==,<<,|=operators for MSVC (which lacks__uint128_t)..patchfiles from trailing-whitespace and end-of-file fixers.python -m pybind11 --cmakedirand setpybind11_DIRexplicitly to avoid vcpkg'sfind_packagewrapper intercepting the search.__SSE__/__SSE2__on x64 (nativecl.exedoes not), and/Zc:__cplusplus /Zc:preprocessorso libint2's__cplusplusand Boost.Preprocessor checks work.cl.exe) withstd::vector.Considerations for future development
delvewheel repair(the Windows equivalent ofauditwheel) instead of CMake DLL bundling - it resolves the full PE dependency tree automatically.encoding="utf-8"explicitly. Relying on the default locale encoding will silently break on Windows.NamedTemporaryFile(delete=False)+ manualunlink()is the pattern. Python 3.12+ offersdelete_on_close=Falseas a cleaner alternative.unsigned longis 32-bit on Windows: Never useunsigned longfor 64-bit values in portable code. Useuint64_t/size_tinstead. This affects HDF5 type mappings in particular.