Python 3.11–3.14 is supported.
The recommended workflow uses uv, which handles virtual environments and dependency installation in one step. pip works too and the commands are shown side-by-side where they differ.
On macOS, install the Xcode Command Line Tools before running the installation commands below (unless they are already installed). They provide the compiler needed to build the native extension:
xcode-select --installWait for the installation to finish before continuing.
git clone https://github.com/Blosc/python-blosc2/
cd python-blosc2
# uv — creates a venv automatically and installs all deps
uv sync --group dev --group test
# pip — activate your own venv first, then:
pip install -e ".[parquet,tui]"
pip install pytest # or: pip install -e ".[parquet,tui]" with the test group belowThe project uses PEP 735 dependency groups
defined in pyproject.toml:
| Group | Contents |
|---|---|
dev |
dask, pandas, pyarrow, jupyterlab, ruff, pre-commit, … |
test |
pytest, psutil |
doc |
Sphinx and all documentation build dependencies |
To install a specific group with pip (pip ≥ 25.3 supports --group):
pip install --group test
pip install --group docRuff is enforced as formatter and linter via pre-commit. Activate the hooks once after cloning:
pre-commit installPre-commit will now run automatically on git commit.
As an alternative to a local setup, the repository includes a
dev container configuration in .devcontainer/. It
provides a Linux environment with Python 3.12 (to match the version used by the
main CI test job), uv, and pandoc, and uses uv to install the dev, test and
doc groups for you.
You need Docker (or a compatible engine) and a client that supports dev
containers, such as VS Code with the
Dev Containers
extension (or Cursor or a similar fork with a compatible extension), GitHub
Codespaces, or the
devcontainer CLI. In VS Code, open the
repository and run Dev Containers: Reopen in Container. The first build
compiles the C extension and takes a few minutes; later sessions reuse the
container and so skip the compilation and launch faster.
Things to know:
- The virtual environment lives at
/home/vscode/.venv, outside the checkout, and is onPATH, sopytest,python -m sphinx, etc. work as shown in this document. A.venvcreated on the host is ignored. - The pre-commit hook is installed for you. The
.gitdirectory is shared with the host (i.e., your computer outside the container), which means the same hook also runs for commits made on the host. Be sure to run commits from inside the container; committing from the host requirespre-committo be installed there as well. - After dependencies change in
pyproject.toml, runuv sync --group dev --group test --group docagain, or rebuild the container. - On Windows, clone the repository inside the WSL filesystem (or use Dev
Containers: Clone Repository in Container Volume). A checkout on a Windows
drive (e.g.,
C:\) is mounted through a slow file-sharing layer, so avoid opening the dev container straight from a Windows checkout, where you can.
clang-cl is required. Make sure LLVM is on PATH and build with Ninja:
CMAKE_GENERATOR=Ninja CC=clang-cl CXX=clang-cl pip install -e .When debugging issues in the C library it can be useful to build C-Blosc2
separately. Assuming it is installed in /usr/local:
CMAKE_PREFIX_PATH=/usr/local USE_SYSTEM_BLOSC2=1 pip install -e .Run the tests pointing at that library:
LD_LIBRARY_PATH=/usr/local/lib pytest(Replace LD_LIBRARY_PATH with DYLD_LIBRARY_PATH on macOS or PATH on Windows.)
When developing or debugging features in miniexpr simultaneously, you can point CMake to your local clone using CMake's standard FETCHCONTENT_SOURCE_DIR_<name> mechanism:
CMAKE_ARGS="-DFETCHCONTENT_SOURCE_DIR_MINIEXPR=$(pwd)/../miniexpr" pip install -e . --no-build-isolationAlternatively, uncomment # SOURCE_DIR ${CMAKE_CURRENT_SOURCE_DIR}/../miniexpr in CMakeLists.txt.
If you do frequent local rebuilds, sccache can significantly speed up C/C++ rebuilds.
macOS
brew install sccache ninjaLinux
# Via cargo (works everywhere):
cargo install sccache
# Or from your distro (Debian/Ubuntu):
apt install sccacheThen build with:
CMAKE_C_COMPILER_LAUNCHER=sccache \
SKBUILD_BUILD_DIR=build \
pip install -e . --no-build-isolationSKBUILD_BUILD_DIR keeps a stable build directory between runs, which improves
incremental rebuilds and sccache hit rates. --no-build-isolation lets
scikit-build-core reuse the existing build tree instead of rebuilding from
scratch in a fresh environment.
Check cache stats with:
sccache --show-statsWe use pytest. Run the full suite:
pytestRun only a subset (faster feedback during development):
pytest tests/ctable/ # CTable-specific tests
pytest tests/ndarray/ # NDArray-specific testsHeavyweight tests (larger data volumes):
pytest -m "heavy"Network tests:
pytest -m "network"The fast blosc2.matmul path uses platform-specific block kernels:
- macOS:
Accelerate - Linux/Windows: runtime-discovered
cblas - fallback: portable
naivekernel
For the runtime cblas backend, python-blosc2 probes the active Python/NumPy
environment rather than linking to one BLAS vendor at build time. Discovery
starts from NumPy's reported BLAS library directory when available, and then
searches common library names in the active environment's lib directories.
On Linux the current candidates include libcblas, libopenblas,
libflexiblas, libblis, libmkl_rt, and generic libblas. A candidate is
accepted only if it loads successfully and exports both cblas_sgemm and
cblas_dgemm. If no suitable provider is found, the fast path falls back to
the naive kernel.
Useful runtime helpers:
blosc2.get_matmul_library()reports the selected runtime library when availableBLOSC_TRACE=1logs candidate probing, rejection, selection, and backend fallback
Example:
BLOSC_TRACE=1 python -c "import blosc2; print(blosc2.get_matmul_library())"For the local wasm32 workflow (uv + pyodide-build + cibuildwheel + test loop),
use the repo skill at .skills/wasm32-pyodide-dev/SKILL.md.
Install it into Codex discovery with:
scripts/install-codex-skill-wasm32.sh --forceWe are using Sphinx for documentation. You can build the documentation by executing:
cd doc
rm -rf ../html _build
python -m sphinx . ../html[You may need to install the pandoc package first: https://pandoc.org/installing.html]
You will find the documentation in the ../html directory.
You can test array API compatibility with the array-api-tests module.
Use the tests/array-api-xfails.txt to skip the tests that are not supported
and run pytest from the array-api-tests source dir like this:
ARRAY_API_TESTS_MODULE=blosc2 pytest array_api_tests --xfails-file ${BLOSC2_DIR}/tests/array-api-xfails.txt -xsSince C-blosc2 is shipped as a compiled binary with python-blosc2, one can compile and run C code using C-blosc2 functions. As of python-blosc2 version 4.0, one can find the location of the include files and binaries as follows. Run the following command in the terminal, which will give as output the path to the __init__.py file within the blosc2 folder.
python -c "import blosc2; print(blosc2.__file__)"
path/to/blosc2/__init__.pyOne may then access the include files via path/to/blosc2/include and the binaries via path/to/blosc2/lib. Thus one may link a C-app via a CMakelists.txt file with the following snippet
# Add directory to search list for find_package
set(CMAKE_PREFIX_PATH "$(python - <<EOF
import blosc2, pathlib
print(pathlib.Path(blosc2.__file__).parent)
EOF)")
find_package(Blosc2 CONFIG REQUIRED)
target_link_libraries(myapp PRIVATE Blosc2::blosc2_shared)
If one prefers to avoid using CMake, one can also use the pkg-config that is shipped with the wheel by running the following sequence of commands
First
BLOSC2_PREFIX=$(python - <<'EOF'
import blosc2, pathlib
print(pathlib.Path(blosc2.__file__).parent)
EOF
)\
export PKG_CONFIG_PATH="$BLOSC2_PREFIX/lib/pkgconfig"
We can check that the .pc file has the required info and has been found via
pkg-config --modversion blosc2Then define a test program
cat > test.c <<'EOF'
#include <stdio.h>
#include <blosc2.h>
int main(void) {
printf(blosc2_get_version_string());
return 0;
}
EOFand compile it to an executable
gcc test.c \
$(pkg-config --cflags --libs blosc2) \
-Wl,--enable-new-dtags \
-Wl,-rpath,"\$ORIGIN" \
-o test_blosc2The executable has to have access to the C library, so we copy the shared library to the executable directory
cp "$BLOSC2_PREFIX/lib/"libblosc2.so .and run the executable
./test_blosc2