Skip to content

1.0.0: release review fixes (rollout leakage, convex CMNN, corrected paired t-test, merge of main) - #88

Merged
PatrickHenkel1 merged 5 commits into
refactoring_v1.0.0from
v1/release-fixes
Nov 7, 2026
Merged

PatrickHenkel1 merged 5 commits into
refactoring_v1.0.0from
v1/release-fixes

Conversation

@PatrickHenkel1

Copy link
Copy Markdown
Collaborator

Final review of 1.0.0 before the merge into main.

Merge of main (0.3.0, #86)

main is merged into this branch, so the PR refactoring_v1.0.0 → main has no conflicts. All conflicts are resolved in favour of 1.0: the old workflows, requirements.txt and .github/copilot-instructions.md stay removed, and pyproject.toml keeps the static version and the bounded dependencies. These are at or above the security fixes of 0.3.0 (constraints.txt checked against the pins of main). The tree after the merge is identical to refactoring_v1.0.0.

Fixes

  • Leakage in the rollout: lags (Q_lag1) and changes (Change(Q)) of derived columns that depend on the recursive feature were taken from the data, so the model saw the measured T over the horizon. This affected the horizon metrics, RecursiveTraining and the feature selection criterion. They are now written in the predicted history (Q_lag1 → expression of Q one step back); lagged raw columns are the data shifted by their lag.
  • shift=0: Change(Q) with Q derived from the output, and nested derived columns, are rejected (holds_current_value).
  • CMNN / PINN: without monotonicities, activation_split was [1, 0, 0]. Only convex neurons plus the non-negative weights of the following layers made the prediction convex in all inputs (affected PINNModel by default, e.g. example 08). The default is now [1, 1, 1].
  • Split settings (test_size, val_size, block_length, gap_steps) passed next to a BlockSplit raise a ValueError instead of being ignored.
  • The output rescaling keeps the scale 1 for a constant target (it was 0).

Methodology

  • ModelComparison.paired: the confidence interval and the p-value now come from the corrected resampled t-test (Nadeau & Bengio 2003): var(d) · (1/n + n_test/n_train), with the ratio taken from the split summary of the runs (new column resampling). There is no correction if all runs share one split (vary=('init',)). This replaces the plain t interval and the Wilcoxon test, which assume independent runs. A note on multiple testing is added to the docs.
  • The embargo of multi step data no longer adds the warmup, which lies inside the split of its sequence.
  • Feature selection: the docs and the docstring note that the score of the selected set is optimistic (validation data).

Documentation

  • CHANGELOG: 1.0.0 is marked unreleased (the checklist says to set the date before tagging), new entry for 0.3.0, wording of requirements.txt, entries for the fixes above.
  • README: the constrained regression is guaranteed on the box of the training data or on bounds, with a certificate per constraint (not "certified on the whole operating range").
  • No runtime claims ("about 1 s for 1e5 samples").
  • branch_review.md: branches already deleted are no longer listed.
  • Release checklist: merge main first.

Checks

  • pytest: 266 passed, 1 skipped; ruff: clean; smoke run of all 13 examples: OK; sphinx-build -W: clean.
  • New regression tests: rollout with Q_lag1 / Change(Q) against a recursion by hand, shift=0 cases, split settings next to a BlockSplit, CMNN neurons, output scale, multi step embargo, corrected t-test.

🤖 Generated with Claude Code

PatrickHenkel1 and others added 5 commits November 7, 2026 00:51
…ries

- requirements.txt: first patched releases for all 76 open Dependabot alerts
  (GitPython 3.2.1, keras 3.15.0, urllib3 2.8.0, protobuf 5.29.6, requests 2.33.0,
  certifi 2026.10.18, idna 3.15, Markdown 3.8.1, Pygments 2.20.0, Werkzeug 3.1.9,
  setuptools 84.1.0, wheel 0.46.2). keras 3.15 needs Python >= 3.11.
- pyproject.toml: GitPython >= 3.2.1; keras >= 3.15.0 on Python >= 3.11 (older
  keras versions are vulnerable when loading models), unchanged below.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Python 3.9 (EOL 2025-10, no longer available on the GitHub runners) and 3.10
  (EOL 2026-10) are dropped, 3.13 is added (CI matrices, classifiers).
- keras >= 3.15.0 for all supported versions (no Python markers needed anymore).
- requirements.txt regenerated from a fresh install (tensorflow 2.21, which
  supports 3.13); numpy and scipy are pinned per Python version, since numpy 2.5
  and scipy 1.18 need Python 3.12. All versions are at or above the security fixes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
physXAI 0.3.0: Python 3.11 - 3.13, dependency versions fixing the security advisories
Conflicts resolved in favour of 1.0.0: the old CI workflows, requirements.txt and the
copilot instructions were removed on purpose (CI template, constraints.txt), pyproject.toml
keeps the static version 1.0.0 and the bounded dependencies, which are at or above the
security fixes of 0.3.0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ed paired t-test

- Rollout: lags (Q_lag1) and changes (Change(Q)) of derived columns depending on the
  recursive feature are recomputed from the predicted trajectory instead of being taken
  from the data (leakage in the horizon metrics, the recursive training and the feature
  selection). Lagged raw columns of their history are the data shifted by the lag.
- shift=0: inputs holding the target through a change of a derived column or nested
  derived columns are rejected (holds_current_value).
- CMNNModel / PINNModel: activation_split defaults to [1, 1, 1]; without monotonicities
  only convex neurons made the prediction convex in all inputs.
- Settings of the split next to a BlockSplit raise a ValueError instead of being ignored.
- Output rescaling keeps the scale 1 for a constant target.
- Multi step embargo without the warmup (it lies inside the split of its sequence).
- ModelComparison.paired: confidence interval and p-value of the corrected resampled
  t-test (Nadeau & Bengio 2003) instead of the plain t interval and Wilcoxon test;
  no correction if all runs share one split; note on multiple testing.
- Feature selection: the score of the selected set is optimistic (validation data).
- Docs: CHANGELOG (unreleased, 0.3.0 entry, requirements.txt wording), README wording of
  the constrained regression, no runtime claims, deleted branches removed from the
  branch review, release checklist: merge main first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings November 7, 2026 10:52

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.

@PatrickHenkel1
PatrickHenkel1 merged commit 073d5f1 into refactoring_v1.0.0 Nov 7, 2026
26 checks passed
@PatrickHenkel1
PatrickHenkel1 deleted the v1/release-fixes branch November 7, 2026 11:00
@PatrickHenkel1 PatrickHenkel1 mentioned this pull request Nov 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants