Skip to content

Introduce a dedicated way to control batch size of batched measure loop #442

Description

@oleksandr-pavlyk

Problem

This is a subissue of #408.

NVBench currently uses the stdrel stopping-criterion parameter --min-time for two separate purposes:

  1. In cold and CPU-only measurements, --min-time controls when the stdrel stopping criterion is allowed to stop based on noise convergence.
  2. In hot/batched measurements, the same value is also used to estimate batch size and decide how much accumulated GPU time the batched measurement loop should collect.

These are different concerns and should not be coupled.

For cold measurements, --min-time is part of a convergence policy: collect enough samples before trusting relative standard deviation.

For hot/batched measurements, the target duration is a batching policy: choose enough repeated kernel launches to measure warm-cache / steady-state behavior with reasonable timing resolution.

Those two values may reasonably need to be different.

Why this matters

1. Cold convergence time and hot batch target time may not want the same value

A user may want cold measurements to collect longer for stable noise estimates, while keeping batched measurements short:

 ./bench --min-time 2.0

Today, that also increases the batched measurement target duration.

Conversely, a user may want a larger hot/batched target duration without forcing stdrel cold measurements to wait longer before convergence is allowed.

There is currently no separate control for that.

2. Batch-size control is lost when using a non-stdrel stopping criterion

Because --min-time belongs to the stdrel stopping criterion, it is not available when another stopping criterion is active:

./bench --stopping-criterion entropy --min-time 0.1
./bench --stopping-criterion sample-count --target-samples 20 --min-time 0.1

Both of the above fail because entropy and sample-count criteria do not accept --min-time parameter.

But the user may still need to control the hot/batched measurement loop target duration. That should not depend on the selected stopping criterion which applies to cold measurements loop.

Desired Behavior

NVBench should provide a dedicated way to control the hot/batched measurement loop target duration independently from stdrel. That control should:

  • apply to batched measurements directly
  • be valid regardless of the selected stopping criterion
  • not change cold or CPU-only stopping behavior
  • compose cleanly with --min-samples and --timeout
  • support global and per-benchmark option scoping

Proposed Direction

Introduce a dedicated benchmark option for the hot/batched measurement loop, for example:

--batch-target-time <seconds>

This option would control the target accumulated GPU time for batched measurements, while --min-time would remain a stdrel stopping-criterion parameter.

Also see: #408 (comment)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Status
    In Progress

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions