Skip to content

Release review 1.0.0: embargo, seeds, shift, unknown arguments, docs - #84

Merged
PatrickHenkel1 merged 9 commits into
refactoring_v1.0.0from
v1/release-review
Nov 7, 2026
Merged

PatrickHenkel1 merged 9 commits into
refactoring_v1.0.0from
v1/release-review

Conversation

@PatrickHenkel1

@PatrickHenkel1 PatrickHenkel1 commented Nov 6, 2026 •

Copy link
Copy Markdown
Collaborator

Release review of refactoring_v1.0.0 before the merge into main and the tag v1.0.0. Architecture, pipeline, models, examples, tests and documentation were checked against each other. The smaller errors are fixed here; larger points are listed at the end as proposals.

Fixed errors

  • The embargo deleted test and validation rows. When a training block was shorter than gap_steps, apply_embargo purged rows of the neighbouring test or validation block (gap 2: [2,2,2,0,2,2,2] → [2,2,-1,-1,-1,2,2]). This happens with small data sets, where min_blocks shortens the blocks. Now only rows of the block at the boundary are purged.
  • physXAI.set_seed had no effect on the models. All keras models had random_seed=42 as default and set it again in prepare(). Now random_seed=None resolves to the global seed, otherwise a new seed is drawn and stored in the config (like random_state of the preprocessing). The same holds for px.RBF(random_state=...) and for the k-means of RBFModel/LinANNModel. Before, the k-means of these two models ignored the seed that repeat_training sets. online_pipeline now seeds as well.
  • Unknown model arguments were silently ignored (SingleStepModel/MultiStepModel.__init__(**kwargs): pass). They now raise a TypeError. from_config drops the meta keys of saved configs in one place. This also showed a test error: test_comparison.py trained the default network because of hidden_layer_size=[4].
  • Several outputs with keras single step networks. The single output neuron was broadcast against k target columns, so the network learned their mean and the metrics only showed output 0. These networks now require one output; LinearRegressionModel and RNNs keep supporting several.
  • shift is now 0 or 1 (decision in the review). With shift > 1, the horizon metrics, the recursive training and the RNN trajectories were wrong.
    • shift=1 (default): states and outputs over the interval [t, t+1), as AgentLib-MPC uses them.
    • shift=0: algebraic outputs of point-sampled data. A leakage guard rejects the output feature as input, also as Change(X) and inside columns expressions. shift=0 allows no Change output, no eval_horizon, no recursive training and no multi step data, and it cannot be exported to AgentLib-MPC.
    • TrainingData stores the shift.
  • LinANNModel evaluated its linear regression into td, including a recursive rollout. It is now only fitted.
  • RBF construction without normalization called .numpy() on a NumPy array.

Additions

  • check_agentlib_compatibility warns about inputs that the preprocessing computes (columns, Change(X), transform), because the MPC has to provide them as variables.
  • Saved configs record environment (versions of Python, NumPy, pandas, scikit-learn, Keras, TensorFlow and CasADi). All from_config methods ignore the key, and AgentLib-MPC reads only known keys.
  • Example 04, part D: px.Pretrained (the model of part B as a frozen block) plus a px.Polynomial correction scaled by a px.Param that starts at 0.

Tests (+40)

  • Config round trip through model_from_config for all 9 model classes, with the meta keys of saved files.
  • Unknown arguments.
  • Seeds: the global seed is used, an explicit seed wins, and training with the same seed gives bitwise identical weights.
  • Multi-output networks are rejected.
  • Monotonicity of CMNNModel and px.MonotoneMLP: gradient signs at points up to 2× outside the range of the training data. Before, monotonicity was not tested anywhere.
  • Save/load with a prediction comparison for CMNN, RBF and LinANN.
  • Embargo edge cases: short blocks, VAL/TEST boundaries, the edges of the data.
  • shift validation and the leakage guard, plus a shift=0 fit.
  • TrainingData.save/load including labels, valid, datetime, loss columns and derived columns.

Documentation

Docs and API were compared identifier by identifier. Corrected:

  • CHANGELOG: wrong example names, the path of model_report, and contradictions between Changed and Removed (recursive_feature_elimination_pipeline, save_result, ModularModel).
  • Guides:
    • The PCNN snippet had no td and no time_index_col.
    • The px.MonotoneMLP signature was wrong: its second positional argument is name.
    • Which blocks normalize their inputs.
    • The shift and loss_columns options, and the default of the embargo.
    • TrainingDataMultiStep has no save.
    • Feature selection keeps the recursive feature only with criterion='horizon'.
    • The output folder of the comparison CLI.
    • AgentLib rules: derived inputs, PCNN, branch 143.
    • Reproducibility.
    • Snippets made self-contained.
  • README:
    • python -m venv physXAI collided with git clone.
    • The CMNN description was wrong.
    • PCNN was missing, and the project structure is updated.
  • Docstrings: module and class docstrings of the keras models and configs (autodoc_inherit_docstrings = False), and 'class_name' → '__class_name__'.

Verification (Windows, Python 3.12)

  • PHYSXAI_REQUIRE_AGENTLIB=1 pytest tests: 199 passed, 1 skipped (test_agentlib_rnn: the installed AgentLib-MPC
    (main) has no RNN support; the saved step model is unchanged by this PR).
  • constraints.txt resolves on Windows and Linux (pip install --dry-run, Python 3.12).
  • python tests/smoke_examples.py: all 13 examples OK (74 s). Example 04 also ran with full epochs. RMSE of T after 12 h: A 0.353 K, B 0.080 K, D 0.110 K.
  • ruff check physXAI tests examples and sphinx-apidoc + sphinx-build -W: no findings.

Notes for the merge

Implemented from the review proposals

  1. Dependency bounds: keras>=3.15.0 (older versions are vulnerable when loading models, GitHub advisories; the pins of constraints.txt are all patched). Upper bounds on the next major versions (keras<4, tensorflow<3, numpy<3, pandas<4, plotly<8, ...). The CasADi converter of AgentLib-MPC recognises layers by their names and attributes, so a Keras major release could silently break the export. constraints.txt pins all 47 runtime dependencies to the tested versions (pip install . -c constraints.txt). The new job tested-versions in windows.yml installs them and runs the tests on Linux and Windows.

  2. GPU: physXAI itself is device-agnostic, but tensorflow from PyPI has no CUDA, and native Windows has no GPU support since TF 2.11. New extra physXAI[gpu] (tensorflow[and-cuda], Linux/WSL2) and a GPU section in Getting started. set_seed(seed, deterministic=True) enables tf.config.experimental.enable_op_determinism() for exactly reproducible runs on a GPU.

  3. RNN/PCNN export: the CHANGELOG and the release checklist state it as a known limitation (needs AgentLib-MPC branch 143-feature-multistep-model-support). check_agentlib_compatibility warns when the installed AgentLib-MPC cannot load recurrent models.

  4. Required contract tests: with PHYSXAI_REQUIRE_AGENTLIB=1 (set in windows.yml), a missing AgentLib-MPC or plugin fails the contract tests instead of skipping them. Only the RNN tests may still skip, until AgentLib-MPC releases RNN support. The model saved with physXAI 0.x is now versioned as a test fixture (tests/contract/data/legacy_model_0x.keras), so test_legacy_model_still_loads runs everywhere. ci.yml (shared template) does not set the variable; the Windows job covers it.

  5. Python 3.11–3.13: constraints.txt pins numpy 2.4.6 and scipy 1.17.1 for Python 3.11, because numpy 2.5 and scipy 1.18 require Python 3.12. Resolution was checked with uv for 3.11, 3.12 and 3.13 on Windows and Linux. tested-versions runs on 3.11 and 3.13. Python 3.14 needs TensorFlow 2.22, which is only a release candidate so far (noted in Getting started).

Still open, fine for 1.0: evaluation as a separate strategy class instead of late imports in SingleStepModel/MultiStepModel.

Notes on #83 (shape constraints)

The findings (undeclared scipy, certificate exact without a check of the corners) are fixed on #83 (verified in dc5b36b).

🤖 Generated with Claude Code

PatrickHenkel1 and others added 7 commits November 7, 2026 00:10
A training block shorter than gap_steps made apply_embargo purge rows of the
neighbouring test or validation block. Rows are now only taken from the run
of the purged side.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…or AgentLib

- shift > 1 gave wrong horizon metrics, recursive training and RNN trajectories.
  Allowed are 1 (next step, states and interval outputs, AgentLib-MPC) and 0
  (algebraic outputs of point sampled data). With shift=0 the output feature must
  not be an input (also not inside a derived column), no Change outputs, no
  eval_horizon, no recursive training and no multi step data.
- TrainingData stores the shift (saved with save/load).
- check_agentlib_compatibility warns about inputs computed by the preprocessing
  (columns, Change(X), transform), which the MPC has to provide.
- TrainingData save/load test covers labels, valid, datetime, loss columns, derived.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- random_seed (and px.RBF random_state) default to the global seed
  (physXAI.set_seed), else a new seed is drawn and stored in the config.
  Before, the default 42 overrode set_seed. online_pipeline seeds as well.
- Unknown model arguments raise a TypeError; from_config ignores the meta keys
  of saved configs centrally. test_comparison trained the default network
  because of a misspelled argument.
- Keras single step networks reject several outputs, which were broadcast
  against their single output neuron.
- LinANNModel only fits its linear regression (its evaluation overwrote td),
  RBF/LinANN take the k-means seed from the model at build time.
- RBF construction without normalization, docstrings.
- Tests: saved configs of all model classes, monotonicity of CMNN and
  MonotoneMLP far outside the data, save/load of CMNN, RBF, LinANN.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ng scripts

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- CHANGELOG: example names, model_report path, contradictions between Changed and
  Removed, entries of this review (seed, shift, unknown arguments, embargo, ...).
- Guides: PCNN snippet (td, time column), MonotoneMLP arguments, normalization of
  blocks, shift and loss_columns options, embargo default, multi step data has no
  save, feature selection keeps the recursive feature only with criterion='horizon',
  comparison CLI output, AgentLib-MPC rules (derived inputs, PCNN, branch 143),
  reproducibility, self-contained snippets.
- README: installation order (venv named like the repository), CMNN description,
  PCNN, project structure.
- Example 04: px.Pretrained (model of part B as frozen block) and px.Polynomial.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings November 6, 2026 23:21

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.

… export notice

- Upper bounds on the next major versions of all dependencies (keras<4,
  tensorflow<3, ...); constraints.txt pins the tested versions of all runtime
  dependencies, the job tested-versions installs them on Linux and Windows.
- Extra physXAI[gpu] (tensorflow[and-cuda], Linux / WSL2), documentation of
  GPU training; set_seed(seed, deterministic=True) enables the deterministic
  kernels of TensorFlow.
- check_agentlib_compatibility warns if the installed AgentLib-MPC cannot load
  recurrent models; known limitation in the CHANGELOG and the release checklist.
- PHYSXAI_REQUIRE_AGENTLIB=1 (set in windows.yml) fails the contract tests
  instead of skipping them; the model saved with physXAI 0.x is a test fixture.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PatrickHenkel1 added a commit that referenced this pull request Nov 6, 2026
Conflicts: random_seed default None (from #84) next to the shape parameter, CHANGELOG and examples/README.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PatrickHenkel1 added a commit that referenced this pull request Nov 6, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@PatrickHenkel1
PatrickHenkel1 merged commit 3cf93b3 into refactoring_v1.0.0 Nov 7, 2026
26 checks passed
@PatrickHenkel1
PatrickHenkel1 deleted the v1/release-review branch November 7, 2026 00:01
PatrickHenkel1 added a commit that referenced this pull request Nov 7, 2026
Conflict in the dependencies: the bounds of #84 plus scipy>=1.11,<2 (pinned 1.18.1 in constraints.txt).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@PatrickHenkel1 PatrickHenkel1 mentioned this pull request Nov 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants