Skip to content

fix: stop forecast() from mutating the caller's input list - #468

Open
ihubanov wants to merge 1 commit into
google-research:masterfrom
ihubanov:fix/forecast-input-mutation
Open

ihubanov wants to merge 1 commit into
google-research:masterfrom
ihubanov:fix/forecast-input-mutation

Conversation

@ihubanov

Copy link
Copy Markdown

TimesFM_2p5.forecast mutates the list the caller passes in. Unrelated to #465/#466/#467, which only touch examples; this is in the main inference entry point.

The bug

When the batch needs padding, src/timesfm/timesfm_2p5/timesfm_2p5_base.py:168 does:

inputs += [np.array([0.0] * 3)] * (self.global_batch_size - w)

+= on a list is an in-place extend, so the padding is appended to the caller's own list:

>>> mine = [series, series, series]        # global_batch_size = 4
>>> len(mine)
3
>>> model.forecast(horizon=8, inputs=mine)
>>> len(mine)
4
>>> len(mine[-1])
3

The first call still returns the right answer, because the output is trimmed to num_inputs. The problem is the second one. num_inputs is recomputed from the now-longer list, so the caller gets back a fourth forecast for a length-3 padding array, presented as a real result, and the list grows again on every subsequent call. Silently wrong output rather than an exception.

I hit this for real while backtesting: 456 context windows became 464, with eight length-3 zero arrays appended, and the next np.array(ctxs) raised inhomogeneous shape. That failure was loud only because I happened to rebuild an array from the list afterwards — code that just calls forecast in a loop gets no signal at all.

The fix

Build the padded batch as a copy:

inputs = list(inputs) + [np.array([0.0] * 3)] * (self.global_batch_size - w)

One line, no behaviour change for a single call. As a side effect any sequence now works, where a tuple previously raised TypeError: can only concatenate tuple (not "list") to tuple on the +=.

Tests

tests/test_forecast_inputs.py covers the padded path, the evenly-divided path (no padding, no mutation), repeated calls on one list, and tuple input. They drive the base class through a stub decode function, so they run without model weights and add no network dependency to CI.

Verified they actually catch the bug: 4 of the 5 fail on master and all pass with the fix. The one that passes either way is the evenly-divided case, which never enters the padding branch.

$ pytest tests/ -q --ignore=tests/test_model_loading.py
64 passed

test_model_loading.py is excluded only because it needs the flax extra, which is unrelated to this change.

`forecast` pads the batch when the number of inputs is not a multiple of
global_batch_size:

    inputs += [np.array([0.0] * 3)] * (self.global_batch_size - w)

`+=` on a list is an in-place extend, so the padding lands in the list the
caller passed in:

    >>> mine = [series, series, series]      # global_batch_size = 4
    >>> model.forecast(horizon=8, inputs=mine)
    >>> len(mine)
    4
    >>> len(mine[-1])
    3

A caller that reuses the list then gets silently wrong results rather than an
error: the next call sees num_inputs = 4, so it returns a fourth forecast for a
padding array, and the list grows again on every call. This is easy to hit when
looping over origins in a backtest or reusing one batch across horizons.

Build the padded batch as a copy instead. As a side effect any sequence works,
where a tuple previously raised TypeError on the `+=`.

Add tests covering the padded and evenly-divided paths, repeated calls on one
list, and tuple input. They drive the base class through a stub decode function,
so they need no model weights. Four of the five fail before this change.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant