`setup_class` time is not considered

[njzjz]

Considering a test case with setup_class:

import time

class TestA:
    @classmethod
    def setup_class(cls):
        time.sleep(6)

    def test1(self):
        time.sleep(2)

    def test2(self):
        time.sleep(3)

Run:

pytest test_a.py --store-durations

The result in .test_durations:

{
    "test_a.py::TestA::test1": 8.008305040886626,
    "test_a.py::TestA::test2": 3.003461800981313
}

Here, the setup_class time is counted in test1, but if test1 and test2 are split into different groups, setup_class in test2 matters. To minimize the time, pytest-split should group them into the same group.

[kavanase]

Bumping this! I agree this would be great to account for (makes a big difference in our tests with doped, where pytest-split is very useful).