Benchmarks
In order to determine how much of a performance gain is obtained by modifications to the tool it is necessarry to benchmark the tool at each major revision. The tool has two basic scenarios of operation, one being operation on local files and the other being operation on files obtained from a remote repository. In order to test both these scenarios and gain a fair estimate of the performance of the tool at any given revision a standard benchmarking procedure was developed.
The Procedure
The benchmark is performed on an SVN repository containing 359 files in several subfolders. The tool is run directly on the remote repository and also with an attempt to use local files if available. Each of these commands is executed 3 times and the average time is recorded. The statistic shown in the chart below is the average seconds/file the tool took to complete its run.
