Skip to content

Keep the CSR backing through shift, roll, diff and merge(dim=...) #1011

Description

@FabianHofmann

Note

The following content was generated by AI.

Describe the feature you'd like to see

Child of #972 (phase 2, PR 8). Follows up on the SnapshotWindow.merge fallback reported in a comment on #972.

These operations convert a CSR-backed expression to dense:

  • shift, roll and diff;
  • linopy.merge([...], dim=...) along an existing dimension.

The conversion is silent unless options["warn_on_densify"] is set. For a groupby result with uneven group sizes, the conversion costs about 250 ms and 1.4 GB in the #972 benchmark. These operations are common in energy models, for example in storage balances and in snapshot windows.

import numpy as np, pandas as pd, linopy
linopy.options["semantics"] = "v1"
i, t = pd.RangeIndex(2000, name="i"), pd.RangeIndex(720, name="t")
m = linopy.Model(sparse=True)
x = m.add_variables(0, 1, coords=[i, t], name="x")
g = pd.Series(np.r_[np.zeros(800, int), np.arange(1200) % 199 + 1], index=i, name="g")
s = x.groupby(g).sum()
assert s.is_sparse
print(s.shift(t=1).is_sparse, s.diff("t").is_sparse)  # False False
print(linopy.merge([s.sel(t=slice(0, 359)), s.sel(t=slice(360, None))], dim="t").is_sparse)  # False

Proposed direction

Still out of scope: multiplication by a DataArray with a new dimension, rolling, cumsum and quadratic products.

Metadata

Metadata

Assignees

No one assigned

    Labels

    performanceThis improves performance while not (meaningfully) altering behaviour for userssparseSparse / CSR-backed expressions and constraints

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions