What we are asking for
- Define the measurement window as a duration in seconds, make it configurable, and publish the
value used in the result.
- Delete
DEFAULT_HYSTERESIS_FPS and its four print sites.
Acceptance criteria
- The window length is a named setting with a stated default, expressed in seconds.
- The result states the window length and the number of windows that had to agree.
DEFAULT_HYSTERESIS_FPS and its references are gone.
- A run before and after the deletion produces the same stream count on the same hardware.
Context
What we found
Four separate adjustments have been added to the pass logic over time, all of them because a
camera cannot sustain more than the rate its source publishes. Reading them against the code,
two are doing useful work, one duplicates another, and one has no effect on any outcome.
| In the code |
What it is for |
Our read |
| Target lowered to 14.95 |
Stops a perfect 15.00 target failing on the first dip |
Drop it, the 0.95 ratio already does this |
| Pass ratio of 0.95, stream_density.py:34 |
Decides whether one camera is keeping up |
Keep, and rename it as the pass rule |
| Two windows must agree, stream_density.py:29 and stream_density.py:30 |
Stops the count changing on one unlucky measurement |
Keep, this is what steadies the ramp |
| Fail threshold of 0.10 fps, stream_density.py:33 |
Intended to stop the count flipping |
Delete, nothing depends on it |
The first row is handled in #247, which covers what the result reports about the target.
#247 also asks for the pass rule and the window length to be stated in the result, so the
window length asked for here is the value that issue needs to print.
The window has a count but no length
DEFAULT_CONSECUTIVE_PASS_WINDOWS = 2 at stream_density.py:30, with its fail-side equivalent at stream_density.py:29, is
configurable and inspectable. The length of a
window is not defined anywhere. It is however many readings happened to be in the log when the
code read it. So "two windows agreed" is not a claim a reader can check, and two runs can apply
different amounts of evidence to the same decision.
The 0.10 fps threshold never reaches the decision
DEFAULT_HYSTERESIS_FPS at stream_density.py:33 creates a second threshold below the pass mark. With a pass
mark of 14.25 the fail threshold sits at 14.15, so a camera reading between the two is neither
passing nor failing.
fail_thresholds is built at stream_density.py:760-763 as the pass mark minus the hysteresis value, and
failing_streams is derived from it at stream_density.py:769-772. Neither reaches the decision. At stream_density.py:773,
all_streams_meet_target = len(passing_streams) == len(stream_fps_dict) counts only the streams
that cleared the pass mark, so a camera at 14.24 is treated exactly like a camera at 5.0.
failing_streams is read in three places, stream_density.py:822, stream_density.py:874 and stream_density.py:888, and each one is an
if failing_streams: that chooses which of two print() calls runs. Six print calls, three
pairs. Within each pair the messages differ in their opening clause and are identical after it:
same window counter, same held stream count, same next step.
Nothing reads those messages back. The run sends everything it prints into
results/stream_density.log at stream_density.py:946, and the result travels back to benchmark.py as a
returned value. No part of the run, and no report built afterwards, depends on which sentence
appeared.
Why it matters
Deleting the threshold cannot change a result, because it does not currently affect one. What it
removes is a piece of the pass logic that reads as though it decides something and does not,
which is exactly the sort of thing that costs a reviewer a day.
The alternative is to wire it into the decision. That would be a behaviour change that could
move published stream counts, and with results due shortly we would rather not take that on.
It would also make the benchmark harder to explain. Every extra threshold is one more thing a
customer has to hold in their head before the number means anything to them. Two windows in a
row agreeing takes a sentence to explain, where a pass mark, a fail threshold and the gap
between them takes a paragraph.
If the intent was to steady the ramp, the measurement window above already does that job.
Line references link to performance-tools @ 144d3b6, read 31 Aug 2026.
What we are asking for
value used in the result.
DEFAULT_HYSTERESIS_FPSand its four print sites.Acceptance criteria
DEFAULT_HYSTERESIS_FPSand its references are gone.Context
What we found
Four separate adjustments have been added to the pass logic over time, all of them because a
camera cannot sustain more than the rate its source publishes. Reading them against the code,
two are doing useful work, one duplicates another, and one has no effect on any outcome.
The first row is handled in #247, which covers what the result reports about the target.
#247 also asks for the pass rule and the window length to be stated in the result, so the
window length asked for here is the value that issue needs to print.
The window has a count but no length
DEFAULT_CONSECUTIVE_PASS_WINDOWS = 2at stream_density.py:30, with its fail-side equivalent at stream_density.py:29, isconfigurable and inspectable. The length of a
window is not defined anywhere. It is however many readings happened to be in the log when the
code read it. So "two windows agreed" is not a claim a reader can check, and two runs can apply
different amounts of evidence to the same decision.
The 0.10 fps threshold never reaches the decision
DEFAULT_HYSTERESIS_FPSat stream_density.py:33 creates a second threshold below the pass mark. With a passmark of 14.25 the fail threshold sits at 14.15, so a camera reading between the two is neither
passing nor failing.
fail_thresholdsis built at stream_density.py:760-763 as the pass mark minus the hysteresis value, andfailing_streamsis derived from it at stream_density.py:769-772. Neither reaches the decision. At stream_density.py:773,all_streams_meet_target = len(passing_streams) == len(stream_fps_dict)counts only the streamsthat cleared the pass mark, so a camera at 14.24 is treated exactly like a camera at 5.0.
failing_streamsis read in three places, stream_density.py:822, stream_density.py:874 and stream_density.py:888, and each one is anif failing_streams:that chooses which of twoprint()calls runs. Six print calls, threepairs. Within each pair the messages differ in their opening clause and are identical after it:
same window counter, same held stream count, same next step.
Nothing reads those messages back. The run sends everything it prints into
results/stream_density.logat stream_density.py:946, and the result travels back tobenchmark.pyas areturned value. No part of the run, and no report built afterwards, depends on which sentence
appeared.
Why it matters
Deleting the threshold cannot change a result, because it does not currently affect one. What it
removes is a piece of the pass logic that reads as though it decides something and does not,
which is exactly the sort of thing that costs a reviewer a day.
The alternative is to wire it into the decision. That would be a behaviour change that could
move published stream counts, and with results due shortly we would rather not take that on.
It would also make the benchmark harder to explain. Every extra threshold is one more thing a
customer has to hold in their head before the number means anything to them. Two windows in a
row agreeing takes a sentence to explain, where a pass mark, a fail threshold and the gap
between them takes a paragraph.
If the intent was to steady the ramp, the measurement window above already does that job.
Line references link to
performance-tools@144d3b6, read 31 Aug 2026.