.. _micro_benchmarks:

***************
Microbenchmarks
***************

Setup
-----
The benchmark tests are located in the ``src/jmh/java folder``. The configuration for the benchmarks is located under ``src/jmh/ressources``. If you want to create more benchmarks locate them here.
To start the benchmark just use ``./gradlew jmh``.

Configuration
-------------

The configuration of the bechmark is managed through the ``build.gradle`` file. There you will find the following segment with a currently minimal set
of configured properties. 

Example configuration::

    jmh {
        include = ['io.ox.benchmark.loadbalancer.test.LBLargeSetupTest']
        //include = ['io.ox.benchmark.loadbalancer.test.LBLargeSetupTest', 'io.ox.benchmark.loadbalancer.test.LBMediumSetupTest', 'io.ox.benchmark.loadbalancer.test.LBSmallSetupTest', 'io.ox.benchmark.proxyrule.test.PRLargeSetupTest', 'io.ox.benchmark.proxyrule.test.PRMediumSetupTest', 'io.ox.benchmark.proxyrule.test.PRSmallSetupTest']
        resultFormat = 'JSON'
        jmhVersion = '1.21'
        jvmArgs = ['-Djmh.separateClasspathJAR=true']
    }

Global configuration
--------------------

The properties set in build.gradle will overwrite all local properties which were used in your test class. So be careful if you want to run your local tests with a given
configuration. If you want different global properties for your tests just add your tests to the ``include`` property. Now only your own written tests will run with that setup.

If you want to explicit exclude tests use the ``exclude`` property.

Properties
----------

To configure the tests you can use the following properties:

:include: include pattern (regular expression) for benchmarks to be executed. Pattern: ['some regular expression']

:exclude: exclude pattern (regular expression) for benchmarks to be executed. Pattern: ['some regular expression']

:iterations: Number of measurement iterations to do.

:benchmarkMode: Benchmark mode. Available modes are: [Throughput/thrpt, AverageTime/avgt, SampleTime/sample, SingleShotTime/ss, All/all]

:batchSize: Batch size: number of benchmark method calls per operation. (some benchmark modes can ignore this setting)

:fork: How many times to fork a single benchmark. Use 0 to disable forking altogether

:failOnError: Should JMH fail immediately if any benchmark had experienced the unrecoverable error? Value: true/false

:forceGC: Should JMH force GC between iterations? Value: true/false

:jvm: Custom JVM to use when forking. Pattern: 'myjvm'

:jvmArgs: Custom JVM args to use when forking.

:jvmArgsAppend: Custom JVM args to use when forking. Pattern: ['append these']

:jvmArgsPrepend: Custom JVM args to use when forking. Pattern: ['prepend these']

:humanOutputFile: human-readable output file. Example: project.file("${project.buildDir}/reports/jmh/human.txt")

:resultsFile: results file. Example: project.file("${project.buildDir}/reports/jmh/results.txt")

:operationsPerInvocation: Operations per invocation.

:benchmarkParameters: Benchmark parameters. Pattern: [:]

:profilers: Use profilers to collect additional data. Supported profilers: [cl, comp, gc, stack, perf, perfnorm, perfasm, xperf, xperfasm, hs_cl, hs_comp, hs_gc, hs_rt, hs_thr]

:timeOnIteration: Time to spend at each measurement iteration.

:resultFormat: Result format type (one of CSV, JSON, NONE, SCSV, TEXT)

:synchronizeIterations: Synchronize iterations? Value: true/false

:threads: Number of worker threads to run with.

:threadGroups: Override thread group distribution for asymmetric benchmarks.

:timeout: Timeout for benchmark iteration.

:timeUnit: Output time unit. Available time units are: [m, s, ms, us, ns].

:verbosity: Verbosity mode. Available modes are: [SILENT, NORMAL, EXTRA]

:warmup: Time to spend at each warmup iteration.

:warmupBatchSize: Warmup batch size: number of benchmark method calls per operation.

:warmupForks: How many warmup forks to make for a single benchmark. 0 to disable warmup forks.

:warmupIterations: Number of warmup iterations to do.

:warmupMode: Warmup mode for warming up selected benchmarks. Warmup modes are: [INDI, BULK, BULK_INDI].

:warmupBenchmarks: Warmup benchmarks to include in the run in addition to already selected. JMH will not measure these benchmarks, but only use them for the warmup.

:zip64: Use ZIP64 format for bigger archives. Value: true/false

:jmhVersion: Specifies JMH version. Example: '1.21'

:includeTests: Allows to include test sources into generate JMH jar, i.e. use it when benchmarks depend on the test classes. Value: true/false

:duplicateClassesStrategy: Strategy to apply when encountring duplicate classes during creation of the fat jar (i.e. while executing jmhJar task). Example: 'fail'

Local configuration
-------------------

It is recommended to configure your tests with local config annotations. Just create an abstract class which holds the annotations and extend it in your test classes, e.g.:

.. code-block:: java

    @BenchmarkMode(Mode.AverageTime)
    @OutputTimeUnit(TimeUnit.NANOSECONDS)
    @State(Scope.Benchmark)
    @Fork(value = 1, jvmArgs = {"-Xms2G", "-Xmx2G"})
    @Warmup(iterations = 3, time = 500, timeUnit = TimeUnit.MILLISECONDS)
    @Measurement(iterations = 5, time = 500, timeUnit = TimeUnit.MILLISECONDS)
    public class AbstractBenchmarkConfig {
        
    }

If you want a specific test in one class to run with individual configurations just annotate them as well:

.. code-block:: java

    @Benchmark
    @Warmup(iterations = 1, time = 500, timeUnit = TimeUnit.MILLISECONDS)
    @Measurement(iterations = 1, time = 500, timeUnit = TimeUnit.MILLISECONDS)
    public void benchmarkStickySessionRouting(StickySessionParams data) {
         manager.chooseServer("APP1");
    }

.. note:: Log output can lead to some higher test outcome than expected. If you want to remove the effect of logging to your microbenchmarks configure a higher log level in the logback.xml.