Repository navigation
docs(benchmarks): the published benchmarks are re-measured at the new model sizes, five times each tenfold step - #2039
Merged
Merged
Conversation
…e refuses a workflow that does not An apostrophe inside a single-quoted description made the file unreadable, so GitHub saw no workflow_dispatch trigger and the trigger:published-bench label's dispatch failed with HTTP 422. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NTByC5UJJWvcFj3mvmv83v
Run https://github.com/fluxopt/specsolve/actions/runs/38520052042 measured fbc735a of claude/specsolve-2032-review-7hrcv0.
FBumann
changed the base branch from
claude/specsolve-2032-review-7hrcv0
to
main
October 14, 2026 12:04
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NTByC5UJJWvcFj3mvmv83v
Merging this PR will not alter performance
Comparing Footnotes
|
… numbers were taken on Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NTByC5UJJWvcFj3mvmv83v
FBumann
added a commit
that referenced
this pull request
Oct 14, 2026
…every model, and a box dragged on any chart zooms all of them (#2040) > **Prompt:** Can we add more brushes for seconds/MB ? And align all axes? > **Prompt:** I think such a box to zoom should replace all sliders!!! … Keep the box. We need to keep al plots in sync!! > [!NOTE] > The following content was generated by AI. Every panel of the benchmark chart has the same x and y axes. Dragging a box on any panel zooms all four to the box's size range and value range, and the table to its sizes. The size slider is gone, and a **Reset zoom** button sits above the charts. <details><summary>What changed</summary> `docs/javascripts/benchmarks.js`: - **Shared axes.** `draw()` collects every visible panel's points in view, and `sharedScales` builds one x and one y axis from all of them. Unzoomed, they look as on `main` but span every panel. - **Zoom box.** `state.zoom` holds the last box in data units: a size range `x` and a value range `y`, each null where the box left that axis alone. A drag on any panel draws the box in all four at once. The panels share axes and size, so the same pixels are the same range in each. Tooltips are hidden during the drag so they do not cover it. On release the box's corners are inverted through the panel's resolved scales, which a control that draws nothing (`reader`) keeps. A side under 8 px leaves that axis as it was, so a horizontal drag zooms only sizes and a vertical drag only values. A zoomed axis ends exactly at the box. Its log ticks are 1–2–5 inside the box, or d3's own log ticks where a narrow range leaves fewer than three. The table keeps only the sizes inside the box. - **Zooming out.** **Reset zoom** sits above the charts at all times and is disabled while nothing is zoomed. A double-click on a chart also zooms out. Switching measure or view zooms out too, because the units change. Switching the axis type keeps the zoom. The box also works under ÷ specsolve, where it replaces the fixed window, and the edge labels follow it. - **No text selection.** A press on a chart cancels `mousedown`, so neither a drag nor a double-click selects text on the page. The charts are also `user-select: none` with the `-webkit-` prefix, which Safari still needs. - **specsolve's name** sits right of its last point inside the y axis, and is left out where none is. It is drawn as a control, outside the marks' clip, so a zoomed axis had put it above the panel. - **The size slider is removed**, with its decade stops. Those stops counted a 5× size toward the decade above: dispatch `xs5` (50k) as 100k. - **x ticks on powers of ten.** One axis over four models would otherwise show every model's sizes as ticks. Where fewer than two powers fit, the ticks fall back to 1–2–5. - **y ticks evenly spaced.** `logTicks` always kept the top tick, so it could sit next to the tick before it: "50 s" printed over "20 s". That was already the case on `main` in the network and sector panels, and shared axes would put it in every panel. The domain now runs on to the next tick at the same step. `docs/about/benchmarks.md`: the slider is removed. A bar above the charts holds the hint "drag on a chart to zoom every chart" and the reset button. The reading guide says the panels share their axes and how the box works. `docs/stylesheets/benchmarks.css`: the slider rules are removed. The box, the reset bar, the crosshair cursor, and hiding tooltips during a drag are added. `touch-action: pan-y` keeps a vertical swipe scrolling the page on a phone, so there a drag zooms along x. **Why not the library's own gestures.** TanStack Charts 1.0 has `brushX` and `zoomX`, both along x only. Its docs say a different gesture should be built by the application, with `d3-brush` or directly, and that the y-domain policy belongs to the application too. This box is about 50 lines and adds no dependency, where `d3-brush` would add two. </details> <details><summary>Verified, and not done</summary> `pixi run check` on `208b84c`: lint, format and typecheck clean; 5601 passed, 566 skipped, 4 xfailed. `tests/test_docs_site.py`: 10 passed. `pixi run -e docs docs-build`: strict, no issues. The page was then loaded in Chromium with #2039's `benchmarks.json` dropped into the built site only. This session's egress policy blocks `cdn.jsdelivr.net`, so the scratch copies of the page loaded the same pinned packages (`@tanstack/charts@1.0.0`, `d3-scale@4.0.2`, `d3-shape@3.2.0`) from npm instead. The tree's import map is unchanged. Real mouse drags, each step checking that all four panels carry identical tick labels: | step | axes, all four panels | table sizes | | --- | --- | --- | | start | 1k–10M · 0.2 ms–200 s | all 21 | | box on network | 20k–2M · 1 ms–1 s; box shown in 4 panels mid-drag | 10k … 1.5M | | horizontal drag on nodal | 50k–200k · y kept | 50k, 100k, 150k | | double-click | back to start | all 21 | | vertical drag on dispatch | x kept · 50 ms–5 s | all 21 | | Reset zoom | back to start | all 21 | | box, then peak RSS | zoom cleared · 100 MB–5 GB | all 21 | | box under ÷ specsolve | 50k–1M · 0.5×–5× | narrowed | | then linear axes | zoom kept | narrowed | The browser selection stays empty during and after a drag inside a plot, past the panel's edge, and from the axis labels, and after a double-click. A text selection during zooming was reported in a browser this session does not have; Chromium did not reproduce it before or after the fix. Two console errors appear on both pages, `main`'s included: another blocked host, and a `TypeError` reading `output`. They are not from this change. Not done, by decision: the zoom has no keyboard path. The size slider had one, and the library's `zoomX` would have one for x only. On a phone a drag zooms along x only, because `pan-y` keeps the page scrollable. </details> 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01NTByC5UJJWvcFj3mvmv83v --------- Co-authored-by: Claude <noreply@anthropic.com>
FBumann
added a commit
that referenced
this pull request
Oct 14, 2026
…es as the columns and rows that reach the solver, and tables what each model declares (#2042) > **Prompt:** And should we make the model overview at the top a table with variables, constraints, dims, sparsity,... columns? > **Prompt:** Should we also add information about complex groupings? Thats what kills linopy memory... > **Prompt:** columns and rows across the page. But it needs to be clearly defined and understandable that its the count of variabels that reach the solver > **Prompt:** THis is quite some text to read before one sees results. Lets make it more compact. … It seems like this could be much more punchy too > [!NOTE] > The following content was generated by AI. The benchmarks page opens on two sentences and its charts, and its reading guide is half as long. Sizes are counted as **columns and rows**, the variables and constraints that reach the solver. Below the reading guide, a generated table says what each model is for, what it declares, and what one snapshot sends to the solver. Page order now: title, two sentences, charts, "Reading the charts", "The models" (definition, table, memory finding), "A script that solves one model", "Not measured yet", "Method". | | dispatch | network | nodal | sector | | --- | --- | --- | --- | --- | | for | bounds and one balance: the shape an array library suits best | a small PyPSA model: lines, stores, solar and wind, and pooled caps | dispatch at nodes, where a technology exists only where it is installed | each technology delivers one of five carriers, such as heat or hydrogen | | variables | p[generator] | p[bus, tech], f[line], charge[store], discharge[store], soc[store] | p[node, tech] | p[node, tech] | | constraints | power_balance | balance[bus], soc_balance[store], pool_cap[pool] | balance[node] | balance[node, carrier] | | grouped sums | — | line → bus, store → bus, bus × tech → pool | — | — | | dimensions | generator 100 | bus 50, tech 12, line 100, store 50 | node 50, tech 12 | node 50, tech 12, carrier 5 | | density | 100% | 25% | 25% | 25% | | snapshots | 2–100k | 2–25k | 2–20k | 2–20k | | columns per snapshot | 100 | 400 | 150 | 150 | | rows per snapshot | 1 | 150 | 50 | 125 | | nonzeros per snapshot | 100 | 800 | 150 | 150 | > From 100k columns up, linopy's peak memory stays within 1.3× of specsolve's on HiGHS, except on `network`, the one model that sums through a grouping. There it is 2.8× at 1M columns, 1.76 GB against 0.63 GB, and the memory budget stops linopy at that size (#2031). <details><summary>What changed</summary> `docs/about/benchmarks.md`: - The intro is two sentences, and the charts follow it. - "Reading the charts" says each rule in one or two short sentences, with a bold lead-in that carries the claim. It says the x axis counts columns, the variables that reach the solver. The round count and the projection's two edges move to "Method". The guide went from 424 to 229 words and from 25 to 17 sentences; the median sentence went from 14 to 11 words, and the sentences over 25 words from 5 to 0. - The chart's axis and tooltips say "columns", through the chart spec's `label`; the data field stays `variables`. - "The models" holds a one-sentence definition of columns and rows, the generated table, and the memory finding. The four description paragraphs are gone, replaced by the table's "for" row. - The `nodal` wording and the projection note say columns. `bench/cases.py`: `Case.about`, the one line on what each published model is for. It is set on `dispatch`, `nodal`, `network` and `sector`. `bench/report.py`: - `models()` writes the table into a new `models` fence, with one column per model and one row per property. The "for" row is `Case.about`. The declarations come from the case's `spec.yaml` (`declared()`), shown without `snapshot`, since every one runs over it. The grouped sums come from each distinct `sum(x, by=…, over=…, into=…)` (`grouped_sums()`). The dimensions and the density come from the case's ladder. The per-snapshot counts are `BASELINE`'s (specsolve's) largest measurement divided by its snapshots, which is the model as built. `_span` prints to three figures. - `fresh()` heads its columns "N columns" rather than "N variables". `bench/test_harness.py`: how a size prints, which declarations and grouped sums `dispatch` and `network` give, the table's columns and rows from the committed results (every model has a "for" line), and the fresh table's new heading. **What stays as it is.** The `bench.report` markdown tables that are not on the page (`results`, `sweeps`) and the long CSV (`bench.tidy`) still say `variables`; they are tooling output, not this page. </details> <details><summary>The memory sentence: numbers, and what they do not show</summary> linopy ÷ specsolve peak RSS, HiGHS, from #2031's committed results, at every size of 100k columns or more: | model | ratios | | --- | --- | | dispatch | 1.07× (100k), 1.00× (1M), 1.19× (10M) | | nodal | 1.14× (300k), 1.26× (3M) | | sector | 0.93× (300k), 1.17× (3M) | | network | 1.62× (100k), **2.78×** (1M: 1.76 vs 0.63 GB) | `network` is the only published model whose linopy port sums through `groupby`: lines and stores into buses, and plants into pools. `sector` sums along a dimension with a coefficient and does not group. On Gurobi the gap also grows on `dispatch` and `nodal` (2.3–2.6× at their largest size in the run behind #2039), so the sentence keeps to HiGHS. What the data does not separate is the line and store sums from the pool sums. #2024's results, before the pools (#2025), had linopy at 1.6× on `network` at 1M (1.01 vs 0.65 GB). But #2023 also landed in between, switching linopy's line and store sums from a dense incidence matrix to `groupby`. So the sentence says "a grouping", not "the pools". A `network` case without pool caps would separate the two. </details> <details><summary>Verified</summary> On `51aa16b`, with `main` (#2040) merged in: `pixi run check` (lint, format, typecheck; 5601 passed, 566 skipped, 4 xfailed), `pixi run -e docs docs-build` (strict, no issues), `tests/test_docs_site.py` and `tests/test_doc_examples.py` (123 passed, 4 skipped). `pixi run test-bench` on `ea38637` (475 passed, 15 skipped); the last commit changes only the page. `pixi run -e bench report` on the committed results writes the `models` fence and the fresh table's new heading, and nothing else. Not checked in a browser: the chart's axis now reads "columns". The label comes straight from the chart spec. </details> 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01NTByC5UJJWvcFj3mvmv83v --------- Co-authored-by: Claude <noreply@anthropic.com>
# Conflicts: # CHANGELOG.md
…ow shows #2039's results: network 2.70x at 1M columns (1.75 vs 0.65 GB); every other model at or under 1.26x from 100k columns up, on HiGHS. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NTByC5UJJWvcFj3mvmv83v
FBumann
enabled auto-merge (squash)
October 14, 2026 15:51
FBumann
pushed a commit
that referenced
this pull request
Oct 14, 2026
This was referenced Oct 14, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The published benchmark run on
fbc735a, which ismainat #2032 plus #2038's workflow fix. The comment on #2032 compares these numbers with the committed ones; the docs preview of this pull request shows the page.Note
The following content was generated by AI.
This publishes #2032's new sizes, measured on the box, to
main. It also carries #2038's two commits, because the run was dispatched from that branch: onmain,published-benchmark.ymldoes not parse. Merging this makes #2038 redundant.Merge this with a merge commit, not a squash.
bench/reproduce.pynow pinsfbc735a5b, the commit that was measured. A merge commit putsfbc735aintomain's history. After a squash it would be reachable only throughclaude/specsolve-2032-review-7hrcv0.What is in the diff, and what was checked
bench/results/*.json,docs/about/benchmarks.json,docs/about/benchmarks.md: from the run's artifact, as the workflow commits them. Withmainmerged in (docs(benchmarks): the benchmark charts share one pair of axes across every model, and a box dragged on any chart zooms all of them #2040, docs(benchmarks): the benchmarks page opens on its charts, counts sizes as the columns and rows that reach the solver, and tables what each model declares #2042),pixi run -e bench reporton these results writes the page'smodelsandfreshtables without a change.network2.7× at 1M columns, 1.75 GB against 0.65 GB. On HiGHS from 100k columns up, every other model stays at or under 1.26×, at every size including the new 5× ones.bench/reproduce.py,bench/reproduce.py.lock: specsolve moves fromaf1dac7d3tofbc735a5b, throughuv lock --script bench/reproduce.py, as docs(benchmarks): the published benchmarks are re-measured with the network model's pooled plants #2031 did. No other package moved. Without this,test_the_lock_installs_what_the_published_numbers_were_taken_onfails..github/workflows/published-benchmark.ymlandtests/test_architecture.py: test(ci): the published-benchmark workflow parses again, and the suite refuses a workflow that does not #2038's fix and its test, unchanged.CHANGELOG.md: this PR's line, as docs(benchmarks): the published benchmarks are re-measured with the network and sparse models, down to a few hundred variables #2005, docs(benchmarks): the published benchmarks are re-measured with the sector-coupled model #2024 and docs(benchmarks): the published benchmarks are re-measured with the network model's pooled plants #2031 added theirs.Run on a cloud container at
afff79b:pixi run check(lint, format, typecheck; 5613 passed, 566 skipped, 4 xfailed),pixi run test-bench(475 passed, 15 skipped),pixi run -e docs docs-build(strict, no issues),tests/test_docs_site.pyandtests/test_doc_examples.py(123 passed, 4 skipped).Read from the run's artifacts. All 8 cases finished. No cell was killed by the memory watchdog, and no library stops at a smaller size than in #2031. pyomo on network reaches
s5, one size further. linopy and gurobipy-loop on network still stop atm, by the 16 GB and 30 s budgets. The matrix floor's row leaves the "launch to exit" table, because #2034 madetest_freshskip both matrix arms.🤖 Generated with Claude Code
https://claude.ai/code/session_01NTByC5UJJWvcFj3mvmv83v