cpython/Tools/pybench
Guido van Rossum cd16bf6404 Merged revisions 55817-55961 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/p3yk

................
  r55837 | guido.van.rossum | 2007-06-08 16:04:42 -0700 (Fri, 08 Jun 2007) | 2 lines

  PEP 3119 -- the abc module.
................
  r55838 | guido.van.rossum | 2007-06-08 17:38:55 -0700 (Fri, 08 Jun 2007) | 2 lines

  Implement part of PEP 3119 -- One Trick Ponies.
................
  r55847 | guido.van.rossum | 2007-06-09 08:28:06 -0700 (Sat, 09 Jun 2007) | 2 lines

  Different way to do one trick ponies, allowing registration (per PEP strawman).
................
  r55849 | guido.van.rossum | 2007-06-09 18:06:38 -0700 (Sat, 09 Jun 2007) | 3 lines

  Make sure that the magic looking for __hash__ (etc.) doesn't apply to
  real subclasses of Hashable.
................
  r55852 | guido.van.rossum | 2007-06-10 08:29:51 -0700 (Sun, 10 Jun 2007) | 2 lines

  Add some more examples, e.g. generators and dict views.
................
  r55853 | guido.van.rossum | 2007-06-10 08:31:59 -0700 (Sun, 10 Jun 2007) | 2 lines

  keys() and items() *are* containers -- just values() isn't.
................
  r55864 | georg.brandl | 2007-06-10 15:29:40 -0700 (Sun, 10 Jun 2007) | 2 lines

  PEP 3127: new octal literals, binary literals.
................
  r55865 | georg.brandl | 2007-06-10 15:31:37 -0700 (Sun, 10 Jun 2007) | 2 lines

  Some octal literal fixes in Tools.
................
  r55866 | georg.brandl | 2007-06-10 15:37:43 -0700 (Sun, 10 Jun 2007) | 2 lines

  Tokenizer changes for PEP 3127.
................
  r55867 | georg.brandl | 2007-06-10 15:37:55 -0700 (Sun, 10 Jun 2007) | 2 lines

  Some docs for PEP 3127.
................
  r55868 | georg.brandl | 2007-06-10 15:44:39 -0700 (Sun, 10 Jun 2007) | 2 lines

  Missed a place in intobject.c. Is that used anymore anyway?
................
  r55871 | neal.norwitz | 2007-06-10 18:31:49 -0700 (Sun, 10 Jun 2007) | 182 lines

  Merged revisions 55729-55868 via svnmerge from
  svn+ssh://pythondev@svn.python.org/python/trunk

  ........
    r55731 | neal.norwitz | 2007-06-01 00:29:12 -0700 (Fri, 01 Jun 2007) | 7 lines

    SF 1668596/1720897: distutils now copies data files
    even if package_dir is empty.

    This needs to be backported.  I'm too tired tonight.  It would be great
    if someone backports this if the buildbots are ok with it.  Otherwise,
    I will try to get to it tomorrow.
  ........
    r55732 | georg.brandl | 2007-06-01 04:33:33 -0700 (Fri, 01 Jun 2007) | 2 lines

    Bug #1722484: remove docstrings again when running with -OO.
  ........
    r55735 | georg.brandl | 2007-06-01 12:20:27 -0700 (Fri, 01 Jun 2007) | 2 lines

    Fix wrong issue number.
  ........
    r55739 | brett.cannon | 2007-06-01 20:02:29 -0700 (Fri, 01 Jun 2007) | 3 lines

    Have configure raise an error when building on AtheOS.  Code specific to AtheOS
    will be removed in Python 2.7.
  ........
    r55746 | neal.norwitz | 2007-06-02 11:33:53 -0700 (Sat, 02 Jun 2007) | 1 line

    Update expected birthday of 2.6
  ........
    r55751 | neal.norwitz | 2007-06-03 13:32:50 -0700 (Sun, 03 Jun 2007) | 10 lines

    Backout the original 'fix' to 1721309 which had no effect.
    Different versions of Berkeley DB handle this differently.
    The comments and bug report should have the details.  Memory is allocated
    in 4.4 (and presumably earlier), but not in 4.5.  Thus
    4.5 has the free error, but not earlier versions.

    Mostly update comments, plus make the free conditional.

    This fix was already applied to the 2.5 branch.
  ........
    r55752 | brett.cannon | 2007-06-03 16:13:41 -0700 (Sun, 03 Jun 2007) | 6 lines

    Make _strptime.TimeRE().pattern() use ``\s+`` for matching whitespace instead
    of ``\s*``.  This prevents patterns from "stealing" bits from other patterns in
    order to make a match work.

    Closes bug #1730389.  Will be backported.
  ........
    r55766 | hyeshik.chang | 2007-06-05 11:16:52 -0700 (Tue, 05 Jun 2007) | 4 lines

    Fix build on FreeBSD.  Bluetooth HCI API in FreeBSD is quite different
    from Linux's.  Just fix the build for now but the code doesn't
    support the complete capability of HCI on FreeBSD yet.
  ........
    r55770 | hyeshik.chang | 2007-06-05 11:58:51 -0700 (Tue, 05 Jun 2007) | 4 lines

    Bug #1728403: Fix a bug that CJKCodecs StreamReader hangs when it
    reads a file that ends with incomplete sequence and sizehint argument
    for .read() is specified.
  ........
    r55775 | hyeshik.chang | 2007-06-05 12:28:15 -0700 (Tue, 05 Jun 2007) | 2 lines

    Fix for Windows: close a temporary file before trying to delete it.
  ........
    r55783 | guido.van.rossum | 2007-06-05 14:24:47 -0700 (Tue, 05 Jun 2007) | 2 lines

    Patch by Tim Delany (missing DECREF). SF #1731330.
  ........
    r55785 | collin.winter | 2007-06-05 17:17:35 -0700 (Tue, 05 Jun 2007) | 3 lines

    Patch #1731049: make threading.py use a proper "raise" when checking internal state, rather than assert statements (which get stripped out by -O).
  ........
    r55786 | facundo.batista | 2007-06-06 08:13:37 -0700 (Wed, 06 Jun 2007) | 4 lines


    FTP.ntransfercmd method now uses create_connection when passive,
    using the timeout received in connection time.
  ........
    r55792 | facundo.batista | 2007-06-06 10:15:23 -0700 (Wed, 06 Jun 2007) | 7 lines


    Added an optional timeout parameter to function urllib2.urlopen,
    with tests in test_urllib2net.py (must have network resource
    enabled to execute them). Also modified test_urllib2.py because
    testing mock classes must take it into acount. Docs are also
    updated.
  ........
    r55793 | thomas.heller | 2007-06-06 13:19:19 -0700 (Wed, 06 Jun 2007) | 1 line

    Build _ctypes and _ctypes_test in the ReleaseAMD64 configuration.
  ........
    r55802 | georg.brandl | 2007-06-07 06:23:24 -0700 (Thu, 07 Jun 2007) | 3 lines

    Disallow function calls like foo(None=1).
    Backport from py3k rev. 55708 by Guido.
  ........
    r55804 | georg.brandl | 2007-06-07 06:30:24 -0700 (Thu, 07 Jun 2007) | 2 lines

    Make reindent.py executable.
  ........
    r55805 | georg.brandl | 2007-06-07 06:34:10 -0700 (Thu, 07 Jun 2007) | 2 lines

    Patch #1667860: Fix UnboundLocalError in urllib2.
  ........
    r55821 | kristjan.jonsson | 2007-06-07 16:53:49 -0700 (Thu, 07 Jun 2007) | 1 line

    Fixing changes to getbuildinfo.c that broke linux builds
  ........
    r55828 | thomas.heller | 2007-06-08 09:10:27 -0700 (Fri, 08 Jun 2007) | 1 line

    Make this test work with older Python releases where struct has no 't' format character.
  ........
    r55829 | martin.v.loewis | 2007-06-08 10:29:20 -0700 (Fri, 08 Jun 2007) | 3 lines

    Bug #1733488: Fix compilation of bufferobject.c on AIX.
    Will backport to 2.5.
  ........
    r55831 | thomas.heller | 2007-06-08 11:20:09 -0700 (Fri, 08 Jun 2007) | 2 lines

    [ 1715718 ] x64 clean compile patch for _ctypes, by Kristj?n Valur
    with small modifications.
  ........
    r55832 | thomas.heller | 2007-06-08 12:01:06 -0700 (Fri, 08 Jun 2007) | 1 line

    Fix gcc warnings intruduced by passing Py_ssize_t to PyErr_Format calls.
  ........
    r55833 | thomas.heller | 2007-06-08 12:08:31 -0700 (Fri, 08 Jun 2007) | 2 lines

    Fix wrong documentation, and correct the punktuation.
    Closes [1700455].
  ........
    r55834 | thomas.heller | 2007-06-08 12:14:23 -0700 (Fri, 08 Jun 2007) | 1 line

    Fix warnings by using proper function prototype.
  ........
    r55839 | neal.norwitz | 2007-06-08 20:36:34 -0700 (Fri, 08 Jun 2007) | 7 lines

    Prevent expandtabs() on string and unicode objects from causing a segfault when
    a large width is passed on 32-bit platforms.  Found by Google.

    It would be good for people to review this especially carefully and verify
    I don't have an off by one error and there is no other way to cause overflow.
  ........
    r55841 | neal.norwitz | 2007-06-08 21:48:22 -0700 (Fri, 08 Jun 2007) | 1 line

    Use macro version of GET_SIZE to avoid Coverity warning (#150) about a possible error.
  ........
    r55842 | martin.v.loewis | 2007-06-09 00:42:52 -0700 (Sat, 09 Jun 2007) | 3 lines

    Patch #1733960: Allow T_LONGLONG to accept ints.
    Will backport to 2.5.
  ........
    r55843 | martin.v.loewis | 2007-06-09 00:58:05 -0700 (Sat, 09 Jun 2007) | 2 lines

    Fix Windows build.
  ........
    r55845 | martin.v.loewis | 2007-06-09 03:10:26 -0700 (Sat, 09 Jun 2007) | 2 lines

    Provide LLONG_MAX for S390.
  ........
    r55854 | thomas.heller | 2007-06-10 08:59:17 -0700 (Sun, 10 Jun 2007) | 4 lines


    First version of build scripts for Windows/AMD64 (no external
    components are built yet, and 'kill_python' is disabled).
  ........
    r55855 | thomas.heller | 2007-06-10 10:55:51 -0700 (Sun, 10 Jun 2007) | 3 lines

    For now, disable the _bsddb, _sqlite3, _ssl, _testcapi, _tkinter
    modules in the ReleaseAMD64 configuration because they do not compile.
  ........
    r55856 | thomas.heller | 2007-06-10 11:27:54 -0700 (Sun, 10 Jun 2007) | 1 line

    Need to set the environment variables, otherwise devenv.com is not found.
  ........
    r55860 | thomas.heller | 2007-06-10 14:01:17 -0700 (Sun, 10 Jun 2007) | 1 line

    Revert commit 55855.
  ........
................
  r55880 | neal.norwitz | 2007-06-10 22:07:36 -0700 (Sun, 10 Jun 2007) | 5 lines

  Fix the refleak counter on test_collections.  The ABC metaclass creates
  a registry which must be cleared on each run.  Otherwise, there *seem*
  to be refleaks when there really aren't any.  (The class is held within
  the registry even though it's no longer needed.)
................
  r55884 | neal.norwitz | 2007-06-10 22:46:33 -0700 (Sun, 10 Jun 2007) | 1 line

  These tests have been removed, so they are no longer needed here
................
  r55886 | georg.brandl | 2007-06-11 00:26:37 -0700 (Mon, 11 Jun 2007) | 3 lines

  Optimize access to True and False in the compiler (if True)
  and the peepholer (LOAD_NAME True).
................
  r55905 | georg.brandl | 2007-06-11 10:02:26 -0700 (Mon, 11 Jun 2007) | 5 lines

  Remove __oct__ and __hex__ and use __index__ for converting
  non-ints before formatting in a base.

  Add a bin() builtin.
................
  r55906 | georg.brandl | 2007-06-11 10:04:44 -0700 (Mon, 11 Jun 2007) | 2 lines

  int(x, 0) does not "guess".
................
  r55907 | georg.brandl | 2007-06-11 10:05:47 -0700 (Mon, 11 Jun 2007) | 2 lines

  Add a comment to explain that nb_oct and nb_hex are nonfunctional.
................
  r55908 | guido.van.rossum | 2007-06-11 10:49:18 -0700 (Mon, 11 Jun 2007) | 2 lines

  Get rid of unused imports and comment.
................
  r55910 | guido.van.rossum | 2007-06-11 13:05:17 -0700 (Mon, 11 Jun 2007) | 2 lines

  _Abstract.__new__ now requires either no arguments or __init__ overridden.
................
  r55911 | guido.van.rossum | 2007-06-11 13:07:49 -0700 (Mon, 11 Jun 2007) | 7 lines

  Move the collections ABCs to a separate file, _abcoll.py, in order to avoid
  needing to import _collections.so during the bootstrap (this will become
  apparent in the next submit of os.py).

  Add (plain and mutable) ABCs for Set, Mapping, Sequence.
................
  r55912 | guido.van.rossum | 2007-06-11 13:09:31 -0700 (Mon, 11 Jun 2007) | 2 lines

  Rewrite the _Environ class to use the new collections ABCs.
................
  r55913 | guido.van.rossum | 2007-06-11 13:59:45 -0700 (Mon, 11 Jun 2007) | 72 lines

  Merged revisions 55869-55912 via svnmerge from
  svn+ssh://pythondev@svn.python.org/python/trunk

  ........
    r55869 | neal.norwitz | 2007-06-10 17:42:11 -0700 (Sun, 10 Jun 2007) | 1 line

    Add Atul Varma for patch # 1667860
  ........
    r55870 | neal.norwitz | 2007-06-10 18:22:03 -0700 (Sun, 10 Jun 2007) | 1 line

    Ignore valgrind problems on Ubuntu from ld
  ........
    r55872 | neal.norwitz | 2007-06-10 18:48:46 -0700 (Sun, 10 Jun 2007) | 2 lines

    Ignore config.status.lineno which seems new (new autoconf?)
  ........
    r55873 | neal.norwitz | 2007-06-10 19:14:39 -0700 (Sun, 10 Jun 2007) | 1 line

    Prevent these tests from running on Win64 since they don\'t apply there either
  ........
    r55874 | neal.norwitz | 2007-06-10 19:16:10 -0700 (Sun, 10 Jun 2007) | 5 lines

    Fix a bug when there was a newline in the string expandtabs was called on.
    This also catches another condition that can overflow.

    Will backport.
  ........
    r55879 | neal.norwitz | 2007-06-10 21:52:37 -0700 (Sun, 10 Jun 2007) | 1 line

    Prevent hang if the port cannot be opened.
  ........
    r55881 | neal.norwitz | 2007-06-10 22:28:45 -0700 (Sun, 10 Jun 2007) | 4 lines

    Add all of the distuils modules that don't seem to have explicit tests. :-(
    Move an import in mworkscompiler so that this module can be imported on
    any platform.  Hopefully this works on all platforms.
  ........
    r55882 | neal.norwitz | 2007-06-10 22:35:10 -0700 (Sun, 10 Jun 2007) | 4 lines

    SF #1734732, lower case the module names per PEP 8.

    Will backport.
  ........
    r55885 | neal.norwitz | 2007-06-10 23:16:48 -0700 (Sun, 10 Jun 2007) | 4 lines

    Not sure why this only fails sometimes on Unix machines. Better
    to disable it and only import msvccompiler on Windows since that's
    the only place it can work anyways.
  ........
    r55887 | neal.norwitz | 2007-06-11 00:29:43 -0700 (Mon, 11 Jun 2007) | 4 lines

    Bug #1734723: Fix repr.Repr() so it doesn't ignore the maxtuple attribute.

    Will backport
  ........
    r55889 | neal.norwitz | 2007-06-11 00:36:24 -0700 (Mon, 11 Jun 2007) | 1 line

    Reflow long line
  ........
    r55896 | thomas.heller | 2007-06-11 08:58:33 -0700 (Mon, 11 Jun 2007) | 3 lines

    Use "O&" in calls to PyArg_Parse when we need a 'void*' instead of "k"
    or "K" codes.
  ........
    r55901 | facundo.batista | 2007-06-11 09:27:08 -0700 (Mon, 11 Jun 2007) | 5 lines


    Added versionchanged flag to all the methods which received
    a new optional timeout parameter, and a versionadded flag to
    the socket.create_connection function.
  ........
................
  r55914 | guido.van.rossum | 2007-06-11 14:19:50 -0700 (Mon, 11 Jun 2007) | 3 lines

  New super() implementation, for PEP 3135 (though the PEP is not yet updated
  to this design, and small tweaks may still be made later).
................
  r55923 | guido.van.rossum | 2007-06-11 21:15:24 -0700 (Mon, 11 Jun 2007) | 4 lines

  I'm guessing this module broke when Neal ripped out the types module --
  it used 'list' both as a local variable and as the built-in list type.
  Renamed the local variable since that was easier.
................
  r55924 | guido.van.rossum | 2007-06-11 21:20:05 -0700 (Mon, 11 Jun 2007) | 5 lines

  Change all occurrences of super(<thisclass>, <firstarg>) to super().
  Seems to have worked, all the tests still pass.
  Exception: test_descr and test_descrtut, which have tons of these
  and are there to test the various usages.
................
  r55939 | collin.winter | 2007-06-12 13:57:33 -0700 (Tue, 12 Jun 2007) | 1 line

  Patch #1735485: remove StandardError from the exception hierarchy.
................
  r55954 | neal.norwitz | 2007-06-12 21:56:32 -0700 (Tue, 12 Jun 2007) | 51 lines

  Merged revisions 55913-55950 via svnmerge from
  svn+ssh://pythondev@svn.python.org/python/trunk

  ........
    r55926 | marc-andre.lemburg | 2007-06-12 02:09:58 -0700 (Tue, 12 Jun 2007) | 3 lines

    Apply patch #1734945 to support TurboLinux as distribution.
  ........
    r55927 | marc-andre.lemburg | 2007-06-12 02:26:49 -0700 (Tue, 12 Jun 2007) | 3 lines

    Add patch #1726668: Windows Vista support.
  ........
    r55929 | thomas.heller | 2007-06-12 08:36:22 -0700 (Tue, 12 Jun 2007) | 1 line

    Checkout, but do not yet try to build, exernal sources.
  ........
    r55930 | thomas.heller | 2007-06-12 09:08:27 -0700 (Tue, 12 Jun 2007) | 6 lines

    Add bufferoverflowU.lib to the libraries needed by _ssl (is this the
    right thing to do?).

    Set the /XP64 /RETAIL build enviroment in the makefile when building
    ReleaseAMD64.
  ........
    r55931 | thomas.heller | 2007-06-12 09:23:19 -0700 (Tue, 12 Jun 2007) | 5 lines

    Revert this change, since it breaks the win32 build:

    Add bufferoverflowU.lib to the libraries needed by _ssl (is this the
    right thing to do?).
  ........
    r55934 | thomas.heller | 2007-06-12 10:28:31 -0700 (Tue, 12 Jun 2007) | 3 lines

    Specify the bufferoverflowU.lib to the makefile on the command line
    (for ReleaseAMD64 builds).
  ........
    r55937 | thomas.heller | 2007-06-12 12:02:59 -0700 (Tue, 12 Jun 2007) | 3 lines

    Add bufferoverflowU.lib to PCBuild\_bsddb.vcproj.
    Build sqlite3.dll and bsddb.
  ........
    r55938 | thomas.heller | 2007-06-12 12:56:12 -0700 (Tue, 12 Jun 2007) | 2 lines

    Don't rebuild Berkeley DB if not needed (this was committed by accident).
  ........
    r55948 | martin.v.loewis | 2007-06-12 20:42:19 -0700 (Tue, 12 Jun 2007) | 3 lines

    Provide PY_LLONG_MAX on all systems having long long.
    Will backport to 2.5.
  ........
................
  r55959 | guido.van.rossum | 2007-06-13 09:22:41 -0700 (Wed, 13 Jun 2007) | 2 lines

  Fix a compilation warning.
................
2007-06-13 18:07:49 +00:00
..
package Merge p3yk branch with the trunk up to revision 45595. This breaks a fair 2006-04-21 10:40:58 +00:00
Arithmetic.py Merged revisions 55817-55961 via svnmerge from 2007-06-13 18:07:49 +00:00
Calls.py Merged revisions 55007-55179 via svnmerge from 2007-05-07 22:24:25 +00:00
CommandLine.py Merged revisions 55817-55961 via svnmerge from 2007-06-13 18:07:49 +00:00
Constructs.py Merged revisions 55007-55179 via svnmerge from 2007-05-07 22:24:25 +00:00
Dict.py Merged revisions 55007-55179 via svnmerge from 2007-05-07 22:24:25 +00:00
Exceptions.py Merged revisions 55007-55179 via svnmerge from 2007-05-07 22:24:25 +00:00
Imports.py Merged revisions 55007-55179 via svnmerge from 2007-05-07 22:24:25 +00:00
Instances.py Merged revisions 55007-55179 via svnmerge from 2007-05-07 22:24:25 +00:00
LICENSE Merge p3yk branch with the trunk up to revision 45595. This breaks a fair 2006-04-21 10:40:58 +00:00
Lists.py Merged revisions 55007-55179 via svnmerge from 2007-05-07 22:24:25 +00:00
Lookups.py Merged revisions 55007-55179 via svnmerge from 2007-05-07 22:24:25 +00:00
NewInstances.py Merged revisions 55007-55179 via svnmerge from 2007-05-07 22:24:25 +00:00
Numbers.py Merged revisions 55817-55961 via svnmerge from 2007-06-13 18:07:49 +00:00
README Merged revisions 46753-51188 via svnmerge from 2006-08-11 14:57:12 +00:00
Setup.py Merged revisions 46753-51188 via svnmerge from 2006-08-11 14:57:12 +00:00
Strings.py Merged revisions 55007-55179 via svnmerge from 2007-05-07 22:24:25 +00:00
Tuples.py Merged revisions 55007-55179 via svnmerge from 2007-05-07 22:24:25 +00:00
Unicode.py Merged revisions 55007-55179 via svnmerge from 2007-05-07 22:24:25 +00:00
clockres.py Merged revisions 46753-51188 via svnmerge from 2006-08-11 14:57:12 +00:00
pybench.py Merged revisions 53952-54987 via svnmerge from 2007-04-27 19:54:29 +00:00
systimes.py Merged revisions 55817-55961 via svnmerge from 2007-06-13 18:07:49 +00:00

README

________________________________________________________________________

PYBENCH - A Python Benchmark Suite
________________________________________________________________________

     Extendable suite of of low-level benchmarks for measuring
          the performance of the Python implementation 
                 (interpreter, compiler or VM).

pybench is a collection of tests that provides a standardized way to
measure the performance of Python implementations. It takes a very
close look at different aspects of Python programs and let's you
decide which factors are more important to you than others, rather
than wrapping everything up in one number, like the other performance
tests do (e.g. pystone which is included in the Python Standard
Library).

pybench has been used in the past by several Python developers to
track down performance bottlenecks or to demonstrate the impact of
optimizations and new features in Python.

The command line interface for pybench is the file pybench.py. Run
this script with option '--help' to get a listing of the possible
options. Without options, pybench will simply execute the benchmark
and then print out a report to stdout.


Micro-Manual
------------

Run 'pybench.py -h' to see the help screen.  Run 'pybench.py' to run
the benchmark suite using default settings and 'pybench.py -f <file>'
to have it store the results in a file too.

It is usually a good idea to run pybench.py multiple times to see
whether the environment, timers and benchmark run-times are suitable
for doing benchmark tests. 

You can use the comparison feature of pybench.py ('pybench.py -c
<file>') to check how well the system behaves in comparison to a
reference run. 

If the differences are well below 10% for each test, then you have a
system that is good for doing benchmark testings.  Of you get random
differences of more than 10% or significant differences between the
values for minimum and average time, then you likely have some
background processes running which cause the readings to become
inconsistent. Examples include: web-browsers, email clients, RSS
readers, music players, backup programs, etc.

If you are only interested in a few tests of the whole suite, you can
use the filtering option, e.g. 'pybench.py -t string' will only
run/show the tests that have 'string' in their name.

This is the current output of pybench.py --help:

"""
------------------------------------------------------------------------
PYBENCH - a benchmark test suite for Python interpreters/compilers.
------------------------------------------------------------------------

Synopsis:
 pybench.py [option] files...

Options and default settings:
  -n arg           number of rounds (10)
  -f arg           save benchmark to file arg ()
  -c arg           compare benchmark with the one in file arg ()
  -s arg           show benchmark in file arg, then exit ()
  -w arg           set warp factor to arg (10)
  -t arg           run only tests with names matching arg ()
  -C arg           set the number of calibration runs to arg (20)
  -d               hide noise in comparisons (0)
  -v               verbose output (not recommended) (0)
  --with-gc        enable garbage collection (0)
  --with-syscheck  use default sys check interval (0)
  --timer arg      use given timer (time.time)
  -h               show this help text
  --help           show this help text
  --debug          enable debugging
  --copyright      show copyright
  --examples       show examples of usage

Version:
 2.0

The normal operation is to run the suite and display the
results. Use -f to save them for later reuse or comparisons.

Available timers:

   time.time
   time.clock
   systimes.processtime

Examples:

python2.1 pybench.py -f p21.pybench
python2.5 pybench.py -f p25.pybench
python pybench.py -s p25.pybench -c p21.pybench
"""

License
-------

See LICENSE file.


Sample output
-------------

"""
-------------------------------------------------------------------------------
PYBENCH 2.0
-------------------------------------------------------------------------------
* using Python 2.4.2
* disabled garbage collection
* system check interval set to maximum: 2147483647
* using timer: time.time

Calibrating tests. Please wait...

Running 10 round(s) of the suite at warp factor 10:

* Round 1 done in 6.388 seconds.
* Round 2 done in 6.485 seconds.
* Round 3 done in 6.786 seconds.
...
* Round 10 done in 6.546 seconds.

-------------------------------------------------------------------------------
Benchmark: 2006-06-12 12:09:25
-------------------------------------------------------------------------------

    Rounds: 10
    Warp:   10
    Timer:  time.time

    Machine Details:
       Platform ID:  Linux-2.6.8-24.19-default-x86_64-with-SuSE-9.2-x86-64
       Processor:    x86_64

    Python:
       Executable:   /usr/local/bin/python
       Version:      2.4.2
       Compiler:     GCC 3.3.4 (pre 3.3.5 20040809)
       Bits:         64bit
       Build:        Oct  1 2005 15:24:35 (#1)
       Unicode:      UCS2


Test                             minimum  average  operation  overhead
-------------------------------------------------------------------------------
          BuiltinFunctionCalls:    126ms    145ms    0.28us    0.274ms
           BuiltinMethodLookup:    124ms    130ms    0.12us    0.316ms
                 CompareFloats:    109ms    110ms    0.09us    0.361ms
         CompareFloatsIntegers:    100ms    104ms    0.12us    0.271ms
               CompareIntegers:    137ms    138ms    0.08us    0.542ms
        CompareInternedStrings:    124ms    127ms    0.08us    1.367ms
                  CompareLongs:    100ms    104ms    0.10us    0.316ms
                CompareStrings:    111ms    115ms    0.12us    0.929ms
                CompareUnicode:    108ms    128ms    0.17us    0.693ms
                 ConcatStrings:    142ms    155ms    0.31us    0.562ms
                 ConcatUnicode:    119ms    127ms    0.42us    0.384ms
               CreateInstances:    123ms    128ms    1.14us    0.367ms
            CreateNewInstances:    121ms    126ms    1.49us    0.335ms
       CreateStringsWithConcat:    130ms    135ms    0.14us    0.916ms
       CreateUnicodeWithConcat:    130ms    135ms    0.34us    0.361ms
                  DictCreation:    108ms    109ms    0.27us    0.361ms
             DictWithFloatKeys:    149ms    153ms    0.17us    0.678ms
           DictWithIntegerKeys:    124ms    126ms    0.11us    0.915ms
            DictWithStringKeys:    114ms    117ms    0.10us    0.905ms
                      ForLoops:    110ms    111ms    4.46us    0.063ms
                    IfThenElse:    118ms    119ms    0.09us    0.685ms
                   ListSlicing:    116ms    120ms    8.59us    0.103ms
                NestedForLoops:    125ms    137ms    0.09us    0.019ms
          NormalClassAttribute:    124ms    136ms    0.11us    0.457ms
       NormalInstanceAttribute:    110ms    117ms    0.10us    0.454ms
           PythonFunctionCalls:    107ms    113ms    0.34us    0.271ms
             PythonMethodCalls:    140ms    149ms    0.66us    0.141ms
                     Recursion:    156ms    166ms    3.32us    0.452ms
                  SecondImport:    112ms    118ms    1.18us    0.180ms
           SecondPackageImport:    118ms    127ms    1.27us    0.180ms
         SecondSubmoduleImport:    140ms    151ms    1.51us    0.180ms
       SimpleComplexArithmetic:    128ms    139ms    0.16us    0.361ms
        SimpleDictManipulation:    134ms    136ms    0.11us    0.452ms
         SimpleFloatArithmetic:    110ms    113ms    0.09us    0.571ms
      SimpleIntFloatArithmetic:    106ms    111ms    0.08us    0.548ms
       SimpleIntegerArithmetic:    106ms    109ms    0.08us    0.544ms
        SimpleListManipulation:    103ms    113ms    0.10us    0.587ms
          SimpleLongArithmetic:    112ms    118ms    0.18us    0.271ms
                    SmallLists:    105ms    116ms    0.17us    0.366ms
                   SmallTuples:    108ms    128ms    0.24us    0.406ms
         SpecialClassAttribute:    119ms    136ms    0.11us    0.453ms
      SpecialInstanceAttribute:    143ms    155ms    0.13us    0.454ms
                StringMappings:    115ms    121ms    0.48us    0.405ms
              StringPredicates:    120ms    129ms    0.18us    2.064ms
                 StringSlicing:    111ms    127ms    0.23us    0.781ms
                     TryExcept:    125ms    126ms    0.06us    0.681ms
                TryRaiseExcept:    133ms    137ms    2.14us    0.361ms
                  TupleSlicing:    117ms    120ms    0.46us    0.066ms
               UnicodeMappings:    156ms    160ms    4.44us    0.429ms
             UnicodePredicates:    117ms    121ms    0.22us    2.487ms
             UnicodeProperties:    115ms    153ms    0.38us    2.070ms
                UnicodeSlicing:    126ms    129ms    0.26us    0.689ms
-------------------------------------------------------------------------------
Totals:                           6283ms   6673ms
"""
________________________________________________________________________

Writing New Tests
________________________________________________________________________

pybench tests are simple modules defining one or more pybench.Test
subclasses.

Writing a test essentially boils down to providing two methods:
.test() which runs .rounds number of .operations test operations each
and .calibrate() which does the same except that it doesn't actually
execute the operations.


Here's an example:
------------------

from pybench import Test

class IntegerCounting(Test):

    # Version number of the test as float (x.yy); this is important
    # for comparisons of benchmark runs - tests with unequal version
    # number will not get compared.
    version = 1.0
    
    # The number of abstract operations done in each round of the
    # test. An operation is the basic unit of what you want to
    # measure. The benchmark will output the amount of run-time per
    # operation. Note that in order to raise the measured timings
    # significantly above noise level, it is often required to repeat
    # sets of operations more than once per test round. The measured
    # overhead per test round should be less than 1 second.
    operations = 20

    # Number of rounds to execute per test run. This should be
    # adjusted to a figure that results in a test run-time of between
    # 1-2 seconds (at warp 1).
    rounds = 100000

    def test(self):

	""" Run the test.

	    The test needs to run self.rounds executing
	    self.operations number of operations each.

        """
        # Init the test
        a = 1

        # Run test rounds
	#
        # NOTE: Use xrange() for all test loops unless you want to face
	# a 20MB process !
	#
        for i in xrange(self.rounds):

            # Repeat the operations per round to raise the run-time
            # per operation significantly above the noise level of the
            # for-loop overhead. 

	    # Execute 20 operations (a += 1):
            a += 1
            a += 1
            a += 1
            a += 1
            a += 1
            a += 1
            a += 1
            a += 1
            a += 1
            a += 1
            a += 1
            a += 1
            a += 1
            a += 1
            a += 1
            a += 1
            a += 1
            a += 1
            a += 1
            a += 1

    def calibrate(self):

	""" Calibrate the test.

	    This method should execute everything that is needed to
	    setup and run the test - except for the actual operations
	    that you intend to measure. pybench uses this method to
            measure the test implementation overhead.

        """
        # Init the test
        a = 1

        # Run test rounds (without actually doing any operation)
        for i in xrange(self.rounds):

	    # Skip the actual execution of the operations, since we
	    # only want to measure the test's administration overhead.
            pass

Registering a new test module
-----------------------------

To register a test module with pybench, the classes need to be
imported into the pybench.Setup module. pybench will then scan all the
symbols defined in that module for subclasses of pybench.Test and
automatically add them to the benchmark suite.


Breaking Comparability
----------------------

If a change is made to any individual test that means it is no
longer strictly comparable with previous runs, the '.version' class
variable should be updated. Therefafter, comparisons with previous
versions of the test will list as "n/a" to reflect the change.


Version History
---------------

  2.0: rewrote parts of pybench which resulted in more repeatable
       timings:
        - made timer a parameter
        - changed the platform default timer to use high-resolution
          timers rather than process timers (which have a much lower
          resolution)
        - added option to select timer
        - added process time timer (using systimes.py)
        - changed to use min() as timing estimator (average
          is still taken as well to provide an idea of the difference)
        - garbage collection is turned off per default
        - sys check interval is set to the highest possible value
        - calibration is now a separate step and done using
          a different strategy that allows measuring the test
          overhead more accurately
        - modified the tests to each give a run-time of between
          100-200ms using warp 10
        - changed default warp factor to 10 (from 20)
        - compared results with timeit.py and confirmed measurements
        - bumped all test versions to 2.0
        - updated platform.py to the latest version
        - changed the output format a bit to make it look
          nicer
        - refactored the APIs somewhat
  1.3+: Steve Holden added the NewInstances test and the filtering 
       option during the NeedForSpeed sprint; this also triggered a long 
       discussion on how to improve benchmark timing and finally
       resulted in the release of 2.0
  1.3: initial checkin into the Python SVN repository


Have fun,
--
Marc-Andre Lemburg
mal@lemburg.com