[PatrickHenkel1]

Phase 1 of the physXAI 1.0.0 refactoring: leakage free data pipeline. Follows #70.

Summary

• BlockSplit (physXAI/preprocessing/splitting.py):
  • contiguous blocks are assigned to train/val/test, drawn stratified over the time axis so every split covers all seasons;
  • embargo gap_steps between splits (default: max lag + shift);
  • strategies blocks (default), holdout_end, random_rows (legacy).
• Windows are only cut out of one split (windows.py), so multi step training and test windows never overlap (fixes the leakage of timeseries_dataset_from_array + take/skip). Training windows are reshuffled every epoch.
• Regular time grid: missing time stamps become invalid rows. Targets are never paired across gaps; this fixes the ignore_nan=True bug, which is active in typical runs.
• Native X_lagN / Change(X) columns, plus lags={'X': n}, a transform(df) hook, datetime indices and eval_horizon / eval_stride (used by the recursive evaluation in phase 3). Constructed features still work during the transition.
• TrainingData holds the whole series as NumPy with split labels and a validity mask, and is saved/loaded without pickle. TrainingDataMultiStep is built from NumPy windows; tf.data is created on demand.
• physXAI.set_seed(seed): one global seed (Python, NumPy, Keras) and the default random_state. The used seed is always stored in the config.
• Configs: get_config / from_config are complete (csv options, split, ignore_nan …) and dispatch on __class_name__. The AgentLib-MPC keys are unchanged.
• .gitattributes normalizes line endings (Windows / Linux).

Behaviour changes

• Single step data is split in blocks by default. Test metrics get more realistic: autocorrelated random rows overestimated them. split='random_rows' restores the old behaviour.

Test plan

• pytest: 65 passed, including 22 new tests for splitting, windows, preprocessing, the gap regression and leakage.
• AgentLib-MPC contract tests pass.
• verify_installation.py and TAir_evaluateMultiStep.py run.
• CI on Linux and Windows.

🤖 Generated with Claude Code

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.