Skip to content

Stream density: define the measurement window in seconds, and delete the unused fail threshold #250

Description

@jcork-intel

What we are asking for

  1. Define the measurement window as a duration in seconds, make it configurable, and publish the
    value used in the result.
  2. Delete DEFAULT_HYSTERESIS_FPS and its four print sites.

Acceptance criteria

  1. The window length is a named setting with a stated default, expressed in seconds.
  2. The result states the window length and the number of windows that had to agree.
  3. DEFAULT_HYSTERESIS_FPS and its references are gone.
  4. A run before and after the deletion produces the same stream count on the same hardware.

Context

What we found

Four separate adjustments have been added to the pass logic over time, all of them because a
camera cannot sustain more than the rate its source publishes. Reading them against the code,
two are doing useful work, one duplicates another, and one has no effect on any outcome.

In the code What it is for Our read
Target lowered to 14.95 Stops a perfect 15.00 target failing on the first dip Drop it, the 0.95 ratio already does this
Pass ratio of 0.95, stream_density.py:34 Decides whether one camera is keeping up Keep, and rename it as the pass rule
Two windows must agree, stream_density.py:29 and stream_density.py:30 Stops the count changing on one unlucky measurement Keep, this is what steadies the ramp
Fail threshold of 0.10 fps, stream_density.py:33 Intended to stop the count flipping Delete, nothing depends on it

The first row is handled in #247, which covers what the result reports about the target.
#247 also asks for the pass rule and the window length to be stated in the result, so the
window length asked for here is the value that issue needs to print.

The window has a count but no length

DEFAULT_CONSECUTIVE_PASS_WINDOWS = 2 at stream_density.py:30, with its fail-side equivalent at stream_density.py:29, is
configurable and inspectable. The length of a
window is not defined anywhere. It is however many readings happened to be in the log when the
code read it. So "two windows agreed" is not a claim a reader can check, and two runs can apply
different amounts of evidence to the same decision.

The 0.10 fps threshold never reaches the decision

DEFAULT_HYSTERESIS_FPS at stream_density.py:33 creates a second threshold below the pass mark. With a pass
mark of 14.25 the fail threshold sits at 14.15, so a camera reading between the two is neither
passing nor failing.

fail_thresholds is built at stream_density.py:760-763 as the pass mark minus the hysteresis value, and
failing_streams is derived from it at stream_density.py:769-772. Neither reaches the decision. At stream_density.py:773,
all_streams_meet_target = len(passing_streams) == len(stream_fps_dict) counts only the streams
that cleared the pass mark, so a camera at 14.24 is treated exactly like a camera at 5.0.

failing_streams is read in three places, stream_density.py:822, stream_density.py:874 and stream_density.py:888, and each one is an
if failing_streams: that chooses which of two print() calls runs. Six print calls, three
pairs. Within each pair the messages differ in their opening clause and are identical after it:
same window counter, same held stream count, same next step.

Nothing reads those messages back. The run sends everything it prints into
results/stream_density.log at stream_density.py:946, and the result travels back to benchmark.py as a
returned value. No part of the run, and no report built afterwards, depends on which sentence
appeared.

Why it matters

Deleting the threshold cannot change a result, because it does not currently affect one. What it
removes is a piece of the pass logic that reads as though it decides something and does not,
which is exactly the sort of thing that costs a reviewer a day.

The alternative is to wire it into the decision. That would be a behaviour change that could
move published stream counts, and with results due shortly we would rather not take that on.

It would also make the benchmark harder to explain. Every extra threshold is one more thing a
customer has to hold in their head before the number means anything to them. Two windows in a
row agreeing takes a sentence to explain, where a pass mark, a fail threshold and the gap
between them takes a paragraph.

If the intent was to steady the ramp, the measurement window above already does that job.


Line references link to performance-tools @ 144d3b6, read 31 Aug 2026.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions