Skip to content

Faster freeze of dense constraints into CSRConstraint #1009

Description

@FabianHofmann

Note

The following content was generated by AI.

Describe the feature you'd like to see

Child of #972 (phase 2, PR 6). This also helps dense models with freeze=True.

In a sparse model, most constraints still start as dense expressions. Variable bounds and storage balances are examples. CSRConstraint.from_dense (linopy/constraints.py:1590-1660) converts them, and this conversion is the largest remaining cost on the build side.

import pandas as pd, linopy
linopy.options["semantics"] = "v1"
m = linopy.Model(sparse=True)
y = m.add_variables(0, 1, coords=[pd.RangeIndex(2000, name="i"), pd.RangeIndex(720, name="t")])
m.add_constraints(y <= 1)  # 1.44M one-term rows: about 90 ms

A direct conversion from the term rectangle to CSR (a mask plus cumsum, no sort) builds the same matrix in about 36 ms.

Where the time goes:

  • _matrix_export_data (linopy/constraints.py:2147-2173) makes several reshape and fancy-index copies. Its peak memory is about 4 times the size of the result.
  • sum_duplicates sorts each row, even when every row has only one term and is therefore already canonical.
  • np.unique over the per-row sign array checks whether all signs are equal.

Proposed direction

  • Ravel vars and coeffs once, keep vars != -1 and coeffs != 0, and build indptr with cumsum.
  • Canonicalise (sort and sum duplicates) only when nterm > 1. CSRConstraint must still promise label-ordered rows (v1 canonical form).
  • Check for a uniform sign with (sign == sign[0]).all().
  • Zero policy stays the same: zeros are dropped at freeze.
  • Tests: frozen against dense equality for duplicate variables, explicit zeros, masked rows and mixed signs, using the existing helpers in test/test_csr.py.
Measurements
case time peak memory
add_constraints(p <= pmax), 1.44M rows, sparse model 120-143 ms 153 MB
same, dense unfrozen model 89 ms 59 MB
direct rectangle-to-CSR prototype 36 ms about 75 MB
add_constraints(balance == demand), frozen dense model (ragged lhs) 602 ms 1739 MB

Metadata

Metadata

Assignees

No one assigned

    Labels

    performanceThis improves performance while not (meaningfully) altering behaviour for users

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions