Download Current Version: exp.py (2005-05-11)

Download Windows Executable: exp.exe.zip (2005-05-11)

Requirements: simpleCS.py

Important Notice: This system has been superceeded by CCMSuite, which is a much more comprehensive approach to the problem of running experiments.

Experiment Runner is a system for collecting the results of running the same chunk of code multiple times using different parameters using multiple computers. It is meant as a utility for doing computer simulation experiments where you have a particular computational model and you want to see the effects of varying various parameters.

To use it, make a file like this: 

   someParameter=1.0  # paramters can be of any type    
   otherValue='a'    
   def run():      
     # do something      
     return result # this is usually a dictionary, but can be anything

Now define another file which says what experiment you are running, and how the parameters should be varied 

   name='myexp' # to look in myexp.py, myexp.zip/myexp.py, myexp.zip/main.py    
   someParameter=[0.3,0.2,10] # use lists to give multiple settings    
   otherValue='b'

To run the experiment, you need to run the server on one machine and the clients on as many machines as you want.

Server

exp.py -s -econfig.txt

Client

exp.py -c -aserver.address.com

Both

exp.py -s -econfig.txt -c

All results end up pickled in a subdirectory with the same name as the experiment. Each run results in a .expParams file and a .expResult file. The name of the files is a timestamp based on when the result was received by the server (down to microseconds). The files are saved in the latest pickle format. For tools for viewing and processing these files, see my Experiment Viewer code.

Note that the clients will automatically download the experiment code from the server. Also, if more than one source file is required, create a zip file with everything needed called myexp.zip.

The default port used is 9000. To change this, specify something like -p9001 on the command line. You can also do -h for the command line argument list: 

 -h, --help        show this help message and exit    
 -s, --server      run a local server    
 -c, --client      run a work client    
 -p#, --port=#     set port    
 -aX, --address=X  set address for client to connect to    
 -eX, --exp=X      specify experiment parameter file

There is also an executable version. This is mostly useful for using windows machines as clients without needing to have Python installed on them. Use it in the same manner as the Python version. Unfortunately, the executable version currently does not have all of the basic Python modules, so depending on what your experiment uses, it may not work. Try it out, and if it complains about certain modules missing, let me know and I'll add that module to it.

History