You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Sparse sum(dim=...) and chained groupby on CSR-backed expressions #964
Make LinearExpression.sum(dim=...) keep a CSR backing instead of densifying. Child of #756.
This is the largest remaining hole in the sparse path. _sum (linopy/expressions.py:1694, 2214-2242) builds the dense rectangle and then stacks the summed dims into _term, so peak nterm becomes nterm x prod(summed sizes). After *, sum is the most common operation in model building, so a CSR-backed expression rarely survives to the constraint.
The same applies to a second groupby on an already CSR-backed expression (expressions.py:2033): it reads self.data and densifies, so chained group-sums always lose the backing.
Implementation ideas
The kernel already exists. CSRLinearExpression.contracted (added in #961) evaluates (I_kept (x) C^T) @ A in chunks. A sum over a dim is that contraction against a ones vector, so the work is wiring rather than new linear algebra:
sum(dim) maps to contracted with an all-ones operand over the summed dims and no new output dims.
sum() over everything maps to the existing full-contraction case, which already yields a zero-dim grid.
The result is compact canonical form, so nterm may differ from the dense path. That is the same contract @ established, v1 only; see Zero culling forces model rebuilding #925.
Chained groupby needs CSRLinearExpression.from_grouper to accept an existing CSR source rather than a dense Dataset.
Note
The following content was generated by AI.
Describe the feature you'd like to see
Make
LinearExpression.sum(dim=...)keep a CSR backing instead of densifying. Child of #756.This is the largest remaining hole in the sparse path.
_sum(linopy/expressions.py:1694,2214-2242) builds the dense rectangle and then stacks the summed dims into_term, so peakntermbecomesnterm x prod(summed sizes). After*,sumis the most common operation in model building, so a CSR-backed expression rarely survives to the constraint.The same applies to a second
groupbyon an already CSR-backed expression (expressions.py:2033): it readsself.dataand densifies, so chained group-sums always lose the backing.Implementation ideas
The kernel already exists.
CSRLinearExpression.contracted(added in #961) evaluates(I_kept (x) C^T) @ Ain chunks. A sum over a dim is that contraction against a ones vector, so the work is wiring rather than new linear algebra:sum(dim)maps tocontractedwith an all-ones operand over the summed dims and no new output dims.sum()over everything maps to the existing full-contraction case, which already yields a zero-dim grid.ntermmay differ from the dense path. That is the same contract@established, v1 only; see Zero culling forces model rebuilding #925.Chained
groupbyneedsCSRLinearExpression.from_grouperto accept an existing CSR source rather than a denseDataset.