Skip to content

Selecting network subsets#625

Open
jpalm3r wants to merge 22 commits intomainfrom
japr/network-subsets
Open

Selecting network subsets#625
jpalm3r wants to merge 22 commits intomainfrom
japr/network-subsets

Conversation

@jpalm3r
Copy link
Copy Markdown
Collaborator

@jpalm3r jpalm3r commented Mar 26, 2026

Selective loading for Network.from_res1d()

Large Res1D files can contain hundreds of nodes and thousands of intermediate
gridpoints. Loading all of that data into memory is slow and can cause memory
issues when you only care about the handful of nodes where observations exist.

Network.from_res1d() now accepts two optional arguments — nodes and
reaches — to control what gets loaded:

Argument "all" (default) [] specific name(s)
nodes load all node timeseries skip all node data load only the listed nodes
reaches populate all gridpoints skip all gridpoints populate only the listed reaches

The full network topology is always constructed regardless of these
settings, so find(), recall(), and graph algorithms continue to work
across the entire network.

Usage

# Default — load everything (backwards compatible)
network = Network.from_res1d("model.res1d")

# Load only the nodes with observations, skip all gridpoints
network = Network.from_res1d(
    "model.res1d",
    nodes=["node_a", "node_b"],
    reaches=[],
)

# Load selected nodes + gridpoints for one reach only
network = Network.from_res1d(
    "model.res1d",
    nodes=["node_a", "node_b"],
    reaches=["reach_1"],
)

Notes

- Both parameters accept "all", a single string, or a list of strings,
making the API fully symmetric.
- A UserWarning is raised if the file contains a node or reach literally
named "all", since that conflicts with the sentinel value. The workaround
is to wrap it in a list: nodes=["all"].

Streamlining matching

NodeObservation now accepts three forms for node=, with lazy resolution handled by NetworkModelResult at match time.

node= options

Form Example Description
int node=42 Direct integer ID — unchanged behaviour
str node="78" Original node alias from the source format (e.g. Res1D node name)
tuple node=("94l1", 21.285) Breakpoint identified by (edge_id, distance)

String and tuple forms are resolved inside NetworkModelResult.extract() via an alias map derived from the Network object. A clear ValueError is raised if an alias cannot be resolved.

Comment thread src/modelskill/network.py Fixed
@jpalm3r jpalm3r marked this pull request as ready for review March 27, 2026 07:42
@jpalm3r jpalm3r requested a review from ecomodeller as a code owner March 27, 2026 07:42
@jpalm3r jpalm3r requested a review from ecomodeller April 15, 2026 11:17
@jpalm3r jpalm3r linked an issue Apr 15, 2026 that may be closed by this pull request
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Investigate performance issues loading big Res1D files

2 participants