User:Ausutter

Audun Sutterud

I work in the Quality Engineering Team.

QTest Benchmarks Runner

I work on a runner for the QTest benchmarks in the QtBase repository.

Experiments

I perform experiments to figure out how we can obtain more stable results.

Setting the scheduler policy

The benchmarks run on Linux. The scheduler can choose to switch out the benchmark process in favor of another process. The other process will run on the same processor as the benchmark process, accessing the same caches and possibly causing cache entries used by the benchmark process to become evicted. The benchmark process may spend some time fetching those entries back into the cache, and this spent time will show up in results. This is a problem.

One way to reduce context switching is to run the benchmark process with a real-time scheduling policy. The scheduler will then prefer the process over processes with a non-real-time scheduling policy, keeping it in the processor longer before switching it out. I performed an experiment to measure the effect of running a benchmark process with a real-time scheduling policy. In short, I ran a benchmark 1000 times with the scheduling policy SCHED_OTHER (non-real-time) and 1000 times with the scheduling policy SCHED_FIFO. I used the -perf flag to measure the number of context switches. The results are as shown below. I use histograms to show how the number of context switches is distributed across the runs.