This directory is work in progress on a small performance benchmarking suite for JFlex.
-
use JMH as the benchmarking framework. There are good technical articles on the subtleties, and accessible short and slightly longer tutorials.
-
main goal is to gather performance numbers on the scanning engine. There are multiple options for this:
-
micro benchmark on just the generated JFlex code + skeleton, as tightly as possible, without action code
-
macro end-to-end benchmark for a full scanner on realistic input (somehow eliminating file reading overhead etc, although it might be interesting to see if anything we do makes any difference once IO is present)
-
anything in between these two
-
-
run on current development snapshot
-
add generated scanners from previous versions of JFlex to track development over time
-
as baseline (= can do no better than this), use a method that reads a Reader into a buffer (at least as long as the input) and touches each character once, sequentially. This should be the minimum a matcher with a Reader interface must do if it is supposed to consume the entire input.
-
use something like java.util.regex and maybe JLex as comparison
-
at some point automate and auto-publish results
We could also benchmark various aspects of the generator itself, but so far that is lower priority.
The plan is to start with a small micro benchmark and incrementally add from there. This should eventually include profiling to at least be informed about what we're actually measuring.
Open to ideas on any of this. Please comment on github issue #689 if you have opinions or would like to contribute.
mvn package
will build the benchmark and
java -jar target/benchmark-full-1.9.1-SNAPSHOT.jar
will run it.