Add explicit plot output options to nvbench-compare

[oleksandr-pavlyk]

Add explicit plot output options to nvbench-compare

nvbench-compare currently does not let users explicitly choose where plots are written. The --plot-along path always opens an interactive
matplotlib window with plt.show(), while the summary --plot path has legacy environment-dependent behavior in headless sessions.

This is inconvenient for CI, remote sessions, and scripted workflows because the caller cannot explicitly choose whether plots should be shown interactively or written to files.

The behavior should be controlled by explicit CLI options instead of inferred from environment details such as DISPLAY.

Proposed behavior

Add output-path options for both plotting modes:

• --plot-output <path>
• --plot-along-output <path-or-template>

Default behavior:

• If the corresponding output option is omitted, use the current interactive behavior and call plt.show().
• If an output path is provided, save the plot and do not call plt.show().

This keeps the common local workflow unchanged while making non-interactive usage explicit and predictable.

--plot-output

Applies to the summary plot generated by --plot.

Example:

nvbench-compare --plot --plot-output compare.png reference.json compare.json

Behavior:

• saves the summary plot to "compare.png"
• does not open an interactive matplotlib window

--plot-along-output

Applies to plots generated by --plot-along. Because --plot-along may produce multiple plots, this option should support a filename template. For example:

nvbench-compare \
  --plot-along "Elements{io}" \
  --plot-along-output "plots/{benchmark}-device{device}-{axis}.png" \
  reference.json compare.json

Potential template fields are {benchmark}, {device}, {axis}. If a plain path without template fields is supplied and multiple plots would be generated, the command should fail with a clear error rather than overwrite files.

Motivation

Explicit output options are needed because:

• CI and headless environments should not rely on implicit GUI/backend detection.
• Remote terminal users may want files even when a display is technically present.
• Local users should retain the simple interactive plt.show() behavior by default.
• --plot and --plot-along should have consistent, documented behavior.
• Avoiding implicit file creation prevents surprising output files in the working directory.

Acceptance criteria

• nvbench-compare --plot ... keeps using plt.show() when --plot-output is omitted.
• nvbench-compare --plot --plot-output out.png ... saves out.png and does not call plt.show()
• nvbench-compare --plot-along AXIS ... keeps using plt.show() when --plot-along-output is omitted.
• nvbench-compare --plot-along AXIS --plot-along-output TEMPLATE ... saves plot-along output files and does not call plt.show().
• Multiple --plot-along outputs must not silently overwrite the same file.
• Documentation describes the default interactive behavior and file-output behavior.