bench - a simple benchmark tool
===============================

Benchmark tool for binary program. 

Features:

    * filtering of the results;
    * bench page faults, page reclaims, context switch;
    * input from a file or another program;
    * series test;
    * revision benchmark.

Series test
===========

The averages output for every N times. So you can see the drift of the test result.
Here is an example of the series output:

                                     SUMMARY
     #|             real time |             user time |           system time
    --+-----------------------+-----------------------+-----------------------
     1| 0.044 [ 0.047]  0.053 | 0.000 [ 0.002]  0.008 | 0.032 [ 0.040]  0.048 
     2| 0.045 [ 0.049]  0.059 | 0.000 [ 0.002]  0.004 | 0.036 [ 0.042]  0.048 
     3| 0.044 [ 0.048]  0.053 | 0.000 [ 0.001]  0.004 | 0.032 [ 0.041]  0.048 
     4| 0.045 [ 0.043]  0.058 | 0.000 [ 0.002]  0.008 | 0.036 [ 0.036]  0.044 
     5| 0.045 [ 0.047]  0.051 | 0.000 [ 0.002]  0.008 | 0.036 [ 0.042]  0.048 
     6| 0.045 [ 0.046]  0.051 | 0.000 [ 0.003]  0.008 | 0.036 [ 0.042]  0.048 
     7| 0.045 [ 0.043]  0.058 | 0.000 [ 0.002]  0.004 | 0.036 [ 0.038]  0.052 
     8| 0.045 [ 0.050]  0.058 | 0.000 [ 0.004]  0.008 | 0.032 [ 0.038]  0.044 
     9| 0.045 [ 0.047]  0.051 | 0.000 [ 0.003]  0.008 | 0.032 [ 0.040]  0.048 
    10| 0.044 [ 0.047]  0.054 | 0.000 [ 0.003]  0.008 | 0.036 [ 0.040]  0.044 
    --+-----------------------+-----------------------+-----------------------

Easy mode
=========

Output time for every program run. This mode turn off the calibration run.
Here is an example of the easy mode output:

                 SUMMARY
    -----+--------+--------+--------+----
         |  real  |  user  |  sys   | ec 
    -----+--------+--------+--------+----
       1 |  0.009 |  0.004 |  0.004 |  0
       2 |  0.009 |  0.000 |  0.012 |  0
       3 |  0.010 |  0.000 |  0.008 |  0
       4 |  0.009 |  0.004 |  0.004 |  0
       5 |  0.011 |  0.004 |  0.008 |  0
       6 |  0.010 |  0.004 |  0.004 |  0
       7 |  0.017 |  0.000 |  0.012 |  0
       8 |  0.009 |  0.000 |  0.008 |  0
       9 |  0.010 |  0.000 |  0.004 |  0
      10 |  0.009 |  0.000 |  0.012 |  0
    -----+--------+--------+--------+----
    real time:  0.009 [ 0.010 +-0.002]  0.017
    sys  time:  0.004 [ 0.008 +-0.004]  0.012
    user time:  0.000 [ 0.002 +-0.002]  0.004


Repository benchmark
====================

If you've got a repository and you wish to check the difference of time for 
different revisions.

There are bench-svn and bench-git tools. 

The sample usage:

    $ bench-git --repo /tmp/sleepy --rev "bef1 bdf3 6454 b440" ./sleepy
    ...........
    Time table
    ----------+-------------------+-------------------+------------------
    revision  |       real        |       user        |        sys
              |  mean       sd    |  mean       sd    |  mean       sd
    ----------+-------------------+-------------------+------------------
        bef1  |  0.082     0.000  |  0.000     0.000  |  0.001     0.002
        bdf3  |  0.062     0.000  |  0.000     0.000  |  0.001     0.002
        6454  |  0.072     0.000  |  0.000     0.000  |  0.002     0.002
        b440  |  0.041     0.000  |  0.000     0.001  |  0.001     0.002
    ----------+-------------------+-------------------+------------------

If options 'repo', 'dir' or 'config' are not defined, bench uses configure
file .bench-config from the current directory.

Configure file
--------------

Configure file is a normal shell script with the following options:
    
    REPO          --- repository path
    DIRECTORY     --- directory with working copy
    PROGRAM       --- program to run
    MAKE          --- program to use as make script
    INPUT         --- file to use as stdin for PROGRAM
    INPUT_PROGRAM --- cmd to use as stdin for PROGRAM
    CD            --- cd to directory INSIDE directory with working copy

The sample config:
REPO=git://..../... # comment
MAKE="python setup.py build"
PROGRAM="python time.py"   # the quotes are mandatory