tree d3768bf0046539b6071385017c9e1ea56b605105
parent 86d693ba70a371a7b5771fdff93314903095d8a0
author Robert Adam <dev@robert-adam.de> 1787876280 +0200
committer GitHub <noreply@github.com> 1787876280 +0300
gpgsig -----BEGIN PGP SIGNATURE-----
 
 wsFcBAABCAAQBQJqkNO4CRC1aQ7uu5UhlAAAK3oQAFg/VGAzNYRFdCcj/3RWtKaN
 ezREBBRm1Bd8AW7x/hQjYBCT4jl122vIkCXGlgTJ+TEE13uWJE2YYHO/ch0BpCR1
 AyIfGeC+7UdVAIGP9KfT0KxGiW9PjKhcddt9Gvrrh9sV+581sD9joXrYXNQfDjNU
 NcxFGIJVUpVMe6svj+Ncbrh4rgVsSKxgv5Zsijs8kdQC3/vUhDpJDbiKItH17xcM
 Yrnq0rwP0OoSnKDKQIvIow791MeQZ3b/bqORLHzR2NfqDqWRqjgl05mMCWHiDg4h
 1aSbAEaLI9pWWaI3XboEO/LSOpLCWnrV1jR6HP4vGhQRrmI/q+XBz0/141S2y4W9
 vnJt4zIHTDyR28XEQfYrXZcrKEz0HTc+IvWPm07q+V0x4MzfNLBvdR2MIQtZDS2N
 Y4AggRG8rY2TqKYSJpcvCGRBrg9qruGkdf6bOTh8t7+I5/c5fY2fajO4JlTB9wsh
 z7JL/AzDiRXIGAx/TYA6rPS7DZDnLmx7KfyCQ397ggSOeMe+8/fxTVUBMDU+KPF8
 aez1KJ1ITMzCUY5aSJ6efejN1o/9ma9TpEn11nNIPhfokb7lYTOnR2fm/8v38V3z
 zGkyaX0QKGm3YPhHzxXnOAhHIm/lzeMiFDcB3oiZiY5RQ+vr2AZVbtd8qPKvMLqZ
 071MUJsdtW5BaPfcWKpe
 =x3B1
 -----END PGP SIGNATURE-----
 

Py tooling: ignore skipped/error benchmarks (#2290)

* Ignore skipped benchmark when computing geomean

Skipped benchmarks have their real and CPU times set to zero. If they are part of the data for which the geomean is calculated, these zeros cause the computed geomean to be zero. Hence, a single skipped benchmark will effectively disable the computation of a geomean.

This seems unfortunate as in many cases there are lots of other benchmarks that still ran normally and for which one might still be interested in the overall geomean change.

* Use 'skipped' attribute instead of 'skip_message'

* Remove skipped benchmarks from partitions

* Add skipped benchmark to test data

* Also skip/ignore benchmarks with errors

* Pacify python pre-commit check

---------

Co-authored-by: Roman Lebedev <lebedev.ri@gmail.com>