feat(csr): one read-only Model(sparse=True) key for the sparse path by FabianHofmann · Pull Request #985 · PyPSA/linopy

FabianHofmann · 2026-09-25T09:19:59Z

Closes #976.

Note

The following content was generated by AI.

Changes proposed in this Pull Request

The sparse (CSR) path now has one model-level switch instead of three independent opt-ins. With Model(sparse=True), the reproduction from #976 is sparse end to end: groupby().sum(), @ and the constraint.

New key

Model(sparse=True), read-only via Model.sparse (no setter).
groupby(...).sum() and @/dot against a constant return CSR-backed expressions. This works for expressions and variables.
add_constraints freezes constraints by default. freeze=False still gives a mutable Constraint, and so does add_indicator_constraints, which follows the same default.

Fail fast instead of silent dense fallbacks

Model(sparse=True) raises under legacy semantics.
groupby().sum(), @ and read_netcdf on a sparse model raise if the semantics were switched back to legacy.
Model(sparse=True, chunk=...) and setting chunk on a sparse model raise.
Passing or setting freeze_constraints on a sparse model raises.
With warn_on_densify, a groupby grouper without a sparse path now emits a densify notice in a sparse model.

Persistence

The key is written to netcdf and read back, and Model.copy keeps it.
Files that only store freeze_constraints, as written by 0.8.x, still load with their freeze default.

Deprecations (FutureWarning, removed with the legacy semantics)

Model(freeze_constraints=...) and the Model.freeze_constraints setter.
groupby(...).sum(sparse=...).
linopy.options["sparse_groupby"].

These keep their current behaviour, with one exception: @ no longer reads sparse_groupby, as proposed in the issue.

Decisions that differ from, or go beyond, the proposal

merge is not wired to the key. It stays sparse only when an operand is already CSR-backed, as today. Converting every dense a + b in a sparse model to CSR would cost time without saving memory. In a sparse model, merges of groupby or @ results stay sparse anyway.
Under legacy, the deprecated switches keep their old meaning instead of being mapped onto sparse. Mapping them would make Model(freeze_constraints=True) raise for legacy users, since the key requires v1.
Open questions from the issue are left at the status quo: warn_on_densify stays off by default, the objective and add_expressions do not follow the key, and Constraints.add(freeze=) stays.
Most of test_csr.py still uses sum(sparse=...) to compare sparse and dense results within one model. A pyproject.toml filter silences the deprecation warnings in the suite until the switches are removed. pytest.warns still sees them.

Docs updated: the release notes, api.rst, the CSR section of creating-constraints.ipynb and the nodal_balance benchmark.

Checklist

AI-generated content is marked (see AGENTS.md).
Code changes are sufficiently documented; i.e. new functions contain docstrings and further explanations may be given in doc.
Unit tests for new features were added (if applicable).
A note for the release notes doc/release_notes.rst of the upcoming release is included.
I consent to the release of this PR's code under the MIT license.

…976) Model(sparse=True) turns on sparse groupby-sum, sparse @ and frozen constraints for one model; v1 only, raises under legacy and with chunk; kept by copy and netcdf. Deprecate Model(freeze_constraints=...), its setter, sum(sparse=...) and options['sparse_groupby']; @ no longer reads the option.

github-actions · 2026-09-25T09:23:31Z

Build cost — v1 vs legacy

v1 build peak & time relative to legacy, on this commit — not a comparison against master (that is CodSpeed).

peak — v1 / legacy	time — v1 / legacy

Full table (time + peak, mean)

benchmarks/drivers/test_build.py::test_build[basic-n=10]
                  time (s)         peak (KiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.05894 (1.07)   │   15.03 (1.00) 
 (v1)        0.05494 (1.0)   │    15.00 (1.0) 

benchmarks/drivers/test_build.py::test_build[basic-n=250]
                  time (s)         peak (MiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.06477 (1.10)   │   12.04 (1.00) 
 (v1)        0.05889 (1.0)   │    12.04 (1.0) 

benchmarks/drivers/test_build.py::test_build[cumsum-severity=0]
                  time (s)        peak (KiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.02591 (1.08)   │   15.20 (1.0) 
 (v1)          0.024 (1.0)   │   15.20 (1.0) 

benchmarks/drivers/test_build.py::test_build[cumsum-severity=100]
                  time (s)        peak (MiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.03789 (1.03)   │   44.93 (1.0) 
 (v1)        0.03686 (1.0)   │   44.93 (1.0) 

benchmarks/drivers/test_build.py::test_build[cumsum-severity=50]
                  time (s)        peak (MiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.02859 (1.08)   │   11.51 (1.0) 
 (v1)        0.02638 (1.0)   │   11.51 (1.0) 

benchmarks/drivers/test_build.py::test_build[expression_arithmetic-n=10]
                  time (s)         peak (KiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.06696 (1.07)   │   24.34 (1.06) 
 (v1)        0.06263 (1.0)   │    23.04 (1.0) 

benchmarks/drivers/test_build.py::test_build[expression_arithmetic-n=250]
                  time (s)         peak (MiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.07502 (1.07)   │   16.12 (1.00) 
 (v1)        0.07012 (1.0)   │    16.12 (1.0) 

benchmarks/drivers/test_build.py::test_build[knapsack-n=10000]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.01585 (1.06)   │   752.18 (1.10) 
 (v1)        0.01499 (1.0)   │    685.15 (1.0) 

benchmarks/drivers/test_build.py::test_build[knapsack-n=100]
                  time (s)        peak (KiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.01577 (1.08)   │   3.12 (1.33) 
 (v1)        0.01459 (1.0)   │    2.34 (1.0) 

benchmarks/drivers/test_build.py::test_build[kvl_cycles-severity=0]
                  time (s)          peak (MiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.04575 (1.22)   │   126.16 (1.44) 
 (v1)        0.03762 (1.0)   │     87.71 (1.0) 

benchmarks/drivers/test_build.py::test_build[kvl_cycles-severity=100]
                  time (s)          peak (MiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.04645 (1.28)   │   126.16 (1.44) 
 (v1)        0.03642 (1.0)   │     87.71 (1.0) 

benchmarks/drivers/test_build.py::test_build[kvl_cycles-severity=50]
                  time (s)          peak (MiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.04562 (1.24)   │   126.16 (1.44) 
 (v1)         0.0368 (1.0)   │     87.71 (1.0) 

benchmarks/drivers/test_build.py::test_build[masked-n=100]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.03702 (1.03)   │    715.12 (1.0) 
 (v1)        0.03605 (1.0)   │   787.73 (1.10) 

benchmarks/drivers/test_build.py::test_build[masked-n=10]
                 time (s)        peak (KiB) 
 name                mean   │          mean 
────────────────────────────────────────────
 (legacy)   0.0354 (1.09)   │   4.54 (1.27) 
 (v1)       0.03235 (1.0)   │    3.57 (1.0) 

benchmarks/drivers/test_build.py::test_build[merge_balance-severity=0]
                 time (s)          peak (KiB) 
 name                mean   │            mean 
──────────────────────────────────────────────
 (legacy)   0.2504 (1.05)   │   704.12 (1.09) 
 (v1)        0.2382 (1.0)   │    643.85 (1.0) 

benchmarks/drivers/test_build.py::test_build[merge_balance-severity=100]
                 time (s)        peak (MiB) 
 name                mean   │          mean 
────────────────────────────────────────────
 (legacy)   0.2636 (1.05)   │   18.34 (1.0) 
 (v1)        0.2503 (1.0)   │   18.34 (1.0) 

benchmarks/drivers/test_build.py::test_build[merge_balance-severity=50]
                 time (s)       peak (MiB) 
 name                mean   │         mean 
───────────────────────────────────────────
 (legacy)   0.2608 (1.05)   │   9.54 (1.0) 
 (v1)        0.2475 (1.0)   │   9.54 (1.0) 

benchmarks/drivers/test_build.py::test_build[milp-n=10]
                  time (s)        peak (KiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.05186 (1.12)   │   3.77 (1.12) 
 (v1)        0.04625 (1.0)   │    3.37 (1.0) 

benchmarks/drivers/test_build.py::test_build[milp-n=50]
                 time (s)          peak (KiB) 
 name                mean   │            mean 
──────────────────────────────────────────────
 (legacy)   0.0514 (1.11)   │   216.59 (1.10) 
 (v1)       0.04648 (1.0)   │    196.23 (1.0) 

benchmarks/drivers/test_build.py::test_build[nodal_balance-severity=0]
                  time (s)         peak (KiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.02614 (1.11)   │   938.49 (1.0) 
 (v1)         0.0236 (1.0)   │   938.49 (1.0) 

benchmarks/drivers/test_build.py::test_build[nodal_balance-severity=100]
                  time (s)       peak (MiB) 
 name                 mean   │         mean 
────────────────────────────────────────────
 (legacy)   0.02678 (1.09)   │   9.66 (1.0) 
 (v1)        0.02464 (1.0)   │   9.66 (1.0) 

benchmarks/drivers/test_build.py::test_build[nodal_balance-severity=50]
                  time (s)       peak (MiB) 
 name                 mean   │         mean 
────────────────────────────────────────────
 (legacy)   0.02667 (1.10)   │   5.32 (1.0) 
 (v1)        0.02418 (1.0)   │   5.32 (1.0) 

benchmarks/drivers/test_build.py::test_build[nodal_balance_sparse-severity=0]
                  time (s)       peak (MiB) 
 name                 mean   │         mean 
────────────────────────────────────────────
 (legacy)   0.01467 (1.01)   │   1.28 (1.0) 
 (v1)        0.01452 (1.0)   │   1.28 (1.0) 

benchmarks/drivers/test_build.py::test_build[nodal_balance_sparse-severity=100]
                  time (s)       peak (MiB) 
 name                 mean   │         mean 
────────────────────────────────────────────
 (legacy)   0.01452 (1.01)   │   1.28 (1.0) 
 (v1)        0.01432 (1.0)   │   1.28 (1.0) 

benchmarks/drivers/test_build.py::test_build[nodal_balance_sparse-severity=50]
                  time (s)       peak (MiB) 
 name                 mean   │         mean 
────────────────────────────────────────────
 (legacy)    0.01466 (1.0)   │   1.28 (1.0) 
 (v1)       0.01483 (1.01)   │   1.28 (1.0) 

benchmarks/drivers/test_build.py::test_build[piecewise-n=1000]
                 time (s)          peak (KiB) 
 name                mean   │            mean 
──────────────────────────────────────────────
 (legacy)   0.1262 (1.03)   │   946.85 (1.06) 
 (v1)         0.123 (1.0)   │    891.54 (1.0) 

benchmarks/drivers/test_build.py::test_build[piecewise-n=10]
                time (s)         peak (KiB) 
 name               mean   │           mean 
────────────────────────────────────────────
 (legacy)   0.125 (1.06)   │   12.01 (1.00) 
 (v1)       0.1184 (1.0)   │    11.99 (1.0) 

benchmarks/drivers/test_build.py::test_build[qp-n=1000]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.03194 (1.06)   │   147.70 (1.06) 
 (v1)        0.03009 (1.0)   │    139.87 (1.0) 

benchmarks/drivers/test_build.py::test_build[qp-n=10]
                  time (s)        peak (KiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.03171 (1.08)   │   2.60 (1.09) 
 (v1)        0.02949 (1.0)   │    2.38 (1.0) 

benchmarks/drivers/test_build.py::test_build[rolling-severity=0]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.02657 (1.10)   │   696.75 (1.03) 
 (v1)        0.02414 (1.0)   │    673.70 (1.0) 

benchmarks/drivers/test_build.py::test_build[rolling-severity=100]
                  time (s)         peak (MiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.06353 (1.07)   │   137.97 (1.0) 
 (v1)        0.05964 (1.0)   │   137.97 (1.0) 

benchmarks/drivers/test_build.py::test_build[rolling-severity=50]
                  time (s)        peak (MiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)    0.04233 (1.0)   │   69.22 (1.0) 
 (v1)       0.04286 (1.01)   │   69.22 (1.0) 

benchmarks/drivers/test_build.py::test_build[sos-n=1000]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.03052 (1.10)   │   402.33 (1.00) 
 (v1)         0.0277 (1.0)   │    402.30 (1.0) 

benchmarks/drivers/test_build.py::test_build[sos-n=10]
                  time (s)        peak (KiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.03012 (1.10)   │   3.19 (1.19) 
 (v1)        0.02736 (1.0)   │    2.69 (1.0) 

benchmarks/drivers/test_build.py::test_build[sparse_network-n=10]
                  time (s)         peak (KiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.03349 (1.08)   │   29.00 (1.54) 
 (v1)        0.03103 (1.0)   │    18.84 (1.0) 

benchmarks/drivers/test_build.py::test_build[sparse_network-n=250]
                  time (s)         peak (MiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.04222 (1.11)   │   37.95 (1.43) 
 (v1)        0.03798 (1.0)   │    26.51 (1.0) 

benchmarks/drivers/test_build.py::test_build[storage-n=10]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)    0.06325 (1.0)   │    410.93 (1.0) 
 (v1)       0.06579 (1.04)   │   427.84 (1.04) 

benchmarks/drivers/test_build.py::test_build[storage-n=250]
                  time (s)         peak (MiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)    0.06751 (1.0)   │     9.94 (1.0) 
 (v1)       0.07079 (1.05)   │   10.22 (1.03)

📊 Interactive plots + CSV: download the semantics-report-v1-vs-legacy artifact from this run.

_{Report-only · not a gate · refreshed on every push · obsolete once legacy is dropped.}

codspeed · 2026-09-25T09:23:41Z

Merging this PR will not alter performance

✅ 181 untouched benchmarks
⏩ 181 skipped benchmarks¹

_{Comparing 976-model-sparse-switch (201aea8) with master (7f06ffa)}

181 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

… switches

FabianHofmann added v1 sparse Sparse / CSR-backed expressions and constraints labels Sep 25, 2026

FabianHofmann added 3 commits September 25, 2026 11:32

refactor(csr): route sparse-model checks through setters and constructor

691c730

refactor(csr): keep only the deprecated override in _freeze_constraints

d5a8cfa

test(csr): build through Model(sparse=True) instead of the deprecated…

201aea8

… switches

FabianHofmann merged commit f74c309 into master Sep 25, 2026
22 checks passed

FabianHofmann deleted the 976-model-sparse-switch branch September 25, 2026 10:55

FabianHofmann mentioned this pull request Sep 26, 2026

perf(sparse): trim the stages around the CSR solver hand-off #992

Merged

4 of 5 tasks

This was referenced Oct 6, 2026

Tracking: sparse/CSR path follow-ups after #961 #972

Open

Make the sparse path observable and controllable (.is_sparse, densify warning, per-call sparse=) #969

Open

Navigation Menu

Repository navigation

feat(csr): one read-only Model(sparse=True) key for the sparse path - #985

feat(csr): one read-only Model(sparse=True) key for the sparse path#985
FabianHofmann merged 4 commits into
masterfrom
976-model-sparse-switch

FabianHofmann commented Sep 25, 2026

Uh oh!

github-actions Bot commented Sep 25, 2026 •

edited

Loading

Uh oh!

codspeed Bot commented Sep 25, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

FabianHofmann commented Sep 25, 2026

Changes proposed in this Pull Request

Checklist

Uh oh!

github-actions Bot commented Sep 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Build cost — v1 vs legacy

Uh oh!

codspeed Bot commented Sep 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will not alter performance

Footnotes

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

github-actions Bot commented Sep 25, 2026 •

edited

Loading

codspeed Bot commented Sep 25, 2026 •

edited

Loading