Skip to content

Sparse sum(dim=...) and chained groupby on CSR-backed expressions #964

Description

@FabianHofmann

Note

The following content was generated by AI.

Describe the feature you'd like to see

Make LinearExpression.sum(dim=...) keep a CSR backing instead of densifying. Child of #756.

This is the largest remaining hole in the sparse path. _sum (linopy/expressions.py:1694, 2214-2242) builds the dense rectangle and then stacks the summed dims into _term, so peak nterm becomes nterm x prod(summed sizes). After *, sum is the most common operation in model building, so a CSR-backed expression rarely survives to the constraint.

The same applies to a second groupby on an already CSR-backed expression (expressions.py:2033): it reads self.data and densifies, so chained group-sums always lose the backing.

Implementation ideas

The kernel already exists. CSRLinearExpression.contracted (added in #961) evaluates (I_kept (x) C^T) @ A in chunks. A sum over a dim is that contraction against a ones vector, so the work is wiring rather than new linear algebra:

  • sum(dim) maps to contracted with an all-ones operand over the summed dims and no new output dims.
  • sum() over everything maps to the existing full-contraction case, which already yields a zero-dim grid.
  • The result is compact canonical form, so nterm may differ from the dense path. That is the same contract @ established, v1 only; see Zero culling forces model rebuilding #925.

Chained groupby needs CSRLinearExpression.from_grouper to accept an existing CSR source rather than a dense Dataset.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestperformanceThis improves performance while not (meaningfully) altering behaviour for userssparseSparse / CSR-backed expressions and constraints

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions