Skip to content

Make the performance benchmarks comprehensive #108

Description

@ahmedwalid05

Context

The build measures memory on every change and stops when memory grows. That check is fast, but it does not show where the time goes, and it does not show what happens over a long run.

Other libraries are faster at reading. A contest for read speed alone ends in third place. The project needs one dimension where it wins.

Scope

Build a benchmark suite with two modes.

  1. A regression check that runs on every build. It must stay fast.
  2. A full performance and load run that a maintainer starts by hand. It measures more workbook shapes and sizes, and it runs long enough to show whether memory grows over time.

The full mode collects a flame graph. Measurement tools change the numbers they measure, so collect the flame graph in a separate run. The reported numbers come only from a run with no tools attached.

Find the dimension where this library can beat the others, and add a benchmark for it. One operation is a candidate: this library changes file content directly, and other libraries build the whole workbook again.

Where to start

The benchmark project holds a guide. It describes the suites that exist and the comparison against three other libraries.

Acceptance criteria

  • The regression check runs on every build and stops the build on a loss.
  • A maintainer starts the full run by hand.
  • The full run produces a flame graph.
  • The reported numbers come from a run with no measurement tools attached, and the output says so.
  • A long run reports whether memory grows over time.
  • A benchmark compares this library against two others on the candidate dimension.
  • This issue records which dimension the project will target.

Extra resources

  • dotnet-trace collects the data. speedscope shows the flame graph.
  • The result of the candidate benchmark decides the work in the next task and in M3. Run it early.

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions