An opinionated Apache Benchmark (ab) runner and result plotter.
Running ab once can give unreliable results. Maybe your server is doing other work or your machine running the test is busy. To circumvent that issue, this script can take multiple measurements with a wait time in between.
Requirements:
- A unix terminal (tested on macOS)
- Apache Benchmark (ab)
- Gnuplot
- NodeJS (>= 14.6)
To install:
- Clone this repository.
- Run
npm install
./abrunner.js measure --help
./abrunner.js compare --helpA typical run can be started like this:
./abrunner.js measure -u https://localhost.test/ -o results/fooThis used the default settings. It will run 10 measurements of 500 requests with 10 concurrency. Between each measurement it will wait for 5 minutes. The results will be stored in ./results/foo.
For more advanced options, read the advanced measure docs.
Running this command will create a bunch of outputs:
iteration*.datfiles contain theabraw measurementsiteration*.outfiles contain theaboutput (that is normally outputted in the terminal)combined.datcontains all combined measurementscombined.statscontain some statistics collected from the combined measurementsmeasure.pngcontains a plot with which you can visually inspect the response times of the individual runs and everything combinedmeasure.pis the Gnuplot script used to create above plot
Compare any number of measurements you took before. The result will be a combined boxplot.
For each measurement you want to incorporate into the comparison, provide the combined.dat datafile (or another ab gnuplot output file) and an appropriate label.
./abrunner.js compare -i results/foo/combined.dat results/bar/combined.dat -l "Foo" "Bar" -o results/comparisonRunning this command will create a bunch of outputs:
run*.datfiles are a copy of the input filesrun*.statsfiles contain some statistics collected from the input filecompare.pngcontains a plot comparing all input filescompare.pis the Gnuplot script used to create above plot
The comparison plot will look something like this:
When running a new measurement, the iteration*.out file captures the ab output. Sometimes this contains an error that helps you pinpoint the problem.
If you run ab against a domain, make sure it ends with a /. If the url includes a path this should not be a problem.
The output.log file always contains always contains a list of the command input and all commands that were run. If somehow the input arguments are not parsed correctly, you should be able to spot that here.