Chapter 5: Writing a Benchmark

In this final chapter we will demonstrate how to write benchmarks using Qt Test.

Writing a Benchmark

To create a benchmark we extend a test function with a QBENCHMARK macro. A benchmark test function will then typically consist of setup code and a QBENCHMARK macro that contains the code to be measured. This test function benchmarks QString::localeAwareCompare().

 void TestBenchmark::simple()
 {
     QString str1 = QLatin1String("This is a test string");
     QString str2 = QLatin1String("This is a test string");

     QCOMPARE(str1.localeAwareCompare(str2), 0);

     QBENCHMARK {
         str1.localeAwareCompare(str2);
     }
 }

Setup can be done at the beginning of the function, the clock is not running at this point. The code inside the QBENCHMARK macro will be measured, and possibly repeated several times in order to get an accurate measurement.

Several back-ends are available and can be selected on the command line.

Data Functions

Data functions are useful for creating benchmarks that compare multiple data inputs, for example locale aware compare against standard compare.

 void TestBenchmark::multiple_data()
 {
     QTest::addColumn<bool>("useLocaleCompare");
     QTest::newRow("locale aware compare") << true;
     QTest::newRow("standard compare") << false;
 }

The test function then uses the data to determine what to benchmark.

 void TestBenchmark::multiple()
 {
     QFETCH(bool, useLocaleCompare);
     QString str1 = QLatin1String("This is a test string");
     QString str2 = QLatin1String("This is a test string");

     int result;
     if (useLocaleCompare) {
         QBENCHMARK {
             result = str1.localeAwareCompare(str2);
         }
     } else {
         QBENCHMARK {
             result = (str1 == str2);
         }
     }
     Q_UNUSED(result);
 }

The "if (useLocaleCompare)" switch is placed outside the QBENCHMARK macro to avoid measuring its overhead. Each benchmark test function can have one active QBENCHMARK macro.

External Tools

Tools for handling and visualizing test data are available as part of the qtestlib-tools project. These include a tool for comparing performance data obtained from test runs and a utility to generate Web-based graphs of performance data.

See the qtestlib-tools announcement for more information on these tools and a simple graphing example.

Example project @ code.qt.io