ci: Filters out benchmark test

We are only testing to ensure that each target can compile and run
we don't need to benchmark third party software.