Skip to content

fix: keep auxiliary coordinates of dense containers in netcdf round trip - #1024

Open
MaykThewessen wants to merge 1 commit into
PyPSA:masterfrom
MaykThewessen:fix/netcdf-keep-aux-coords
Open

MaykThewessen wants to merge 1 commit into
PyPSA:masterfrom
MaykThewessen:fix/netcdf-keep-aux-coords

Conversation

@MaykThewessen

@MaykThewessen MaykThewessen commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

Note

AI-assisted. Claude wrote the code and tests, and ran the checks listed below. I reviewed the diff.

Changes proposed in this Pull Request

read_netcdf dropped the auxiliary (non-index) coordinates of every dense variable, constraint, named expression and of Model.parameters, also for plain naive values. With v1 recommending a flat dimension with levels as auxiliary coordinates instead of a MultiIndex, a round trip through netcdf (and so solve(remote=...)) loses those levels. Frozen constraints already kept theirs.

The drop in get_prefix was there because selecting one container's data variables from the merged dataset attaches the scalar coordinates of all other containers (e.g. constraints-c0-snapshot after x.sel(snapshot=0) >= 1). Coordinates with dimensions can only attach to the container whose dimensions they share, so the read now drops only scalar coordinates that belong to another container, matched on the longest prefix because names may contain -. This also restores a container's own scalar coordinates, and makes the separate MultiIndex level bookkeeping in get_prefix unnecessary.

Overlap with #1021: both touch read_netcdf/get_prefix in linopy/io.py and merge without textual conflicts. The tests of both pass in either merge order. #1025 (dash-named parameters) also touches get_prefix: it adds an owner() lookup by longest container prefix, and whichever of #1024 and #1025 lands second should use it in place of the prefix split here, otherwise a dash-named parameter can take the scalar coords of another parameter (details in #1025).

Checklist

  • AI-generated content is marked (see AGENTS.md).
  • Code changes are sufficiently documented; i.e. new functions contain docstrings and further explanations may be given in doc.
  • Unit tests for new features were added (if applicable).
  • A note for the release notes doc/release_notes.rst of the upcoming release is included.
  • I consent to the release of this PR's code under the MIT license.
What was checked (AI-generated)
  • New test_model_to_netcdf_aux_coords in test/test_io.py: a flat snapshot dimension with time (datetime) and period aux coords on a variable, a dense constraint, a named expression and a parameter, plus a constraint c-first from x.sel(snapshot=0) whose scalar coords would leak into c under a plain prefix match. Asserts assert_model_equal, which compares coordinates. Fails on master and passes on the branch, under both legacy and v1 semantics.
  • Mutation checks on the new test: matching the shortest instead of the longest prefix fails it, and keeping all coordinates (no drop) fails it.
  • History: the drop came in with 4b16c1a (2023, "fix import of multiindexed coord names"), which kept only MultiIndex levels; probing a file shows the coords xarray attaches on selection are the scalar ones of other containers.
  • test/test_io.py, test/test_csr.py and the other netcdf users (test_sos_reformulation, test_indicator_constraints, test_dtypes, test_scaling, test_piecewise_constraints, test_fix_relax) pass: 1864 passed. The 6 failures are TestSignParameter SOS2 cases in test_piecewise_constraints.py (HiGHS here has no SOS support) and fail identically with master's io.py.
  • ruff check, ruff format --check clean; mypy reports no errors in the changed files (2 existing ones in solvers.py and remote/oetc.py).
  • Merge with fix: write timezone-aware coordinates in Model.to_netcdf #1021 (fix/netcdf-tz-aware-coords): no textual conflicts, and test/test_io.py plus test/test_csr.py pass on the merge (1048 passed). With both merged, dense tz-aware aux coords on a variable, constraint and expression come back with their zone (datetime64[ns, Europe/Amsterdam]).
  • Not in this PR: a parameter whose name contains - (e.g. m.parameters["my-param"]) is silently dropped by read_netcdf on master, because has_prefix splits on the last -. Separate fix.

🤖 Generated with Claude Code

read_netcdf dropped every non-index coordinate except MultiIndex levels
to keep out the scalar coordinates of other containers that xarray
attaches when selecting one container's data variables. That also
dropped the auxiliary coordinates of dense variables, constraints,
expressions and parameters, such as the levels on a flat dimension that
v1 recommends instead of a MultiIndex.

Only drop scalar coordinates owned by another container, matched on the
longest prefix since names may contain dashes. Coordinates with
dimensions only attach to the container whose dimensions they share.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaykThewessen added a commit to MaykThewessen/linopy that referenced this pull request Oct 8, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@codspeed

codspeed Bot commented Oct 8, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 181 untouched benchmarks
⏩ 181 skipped benchmarks1


Comparing MaykThewessen:fix/netcdf-keep-aux-coords (dce6104) with master (1d3c516)

Open in CodSpeed

Footnotes

  1. 181 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant