This is a very simple example of how to generate a lexer with JFlex:
- for a very simple grammar of the Toy programming language described in the user manual.
- without integration to a parser. As a result, the program does nothing really useful, because there is no parser.
The generated lexer has the default name Yylex because the flex
specification doesn't define a name with the %class.
The project comes with a test class for the lexer: YylexTest.
The test:
- runs the lexer in debug mode on
test.txt - collects the output of JFlex by redirecting
System.out - and verifies that the verbose logs of JFlex corresponds to
the expected content of
output.good.
src/main/flex/simple.flex: the simple grammar specificationsrc/test/data/test.txt: sample inputsrc/test/data/output.good: golden file, i.e. expected output corresponding to the sample input fromtest.txtsrc/test/java/YylexTest.java: jUnit integration test that running the lexer on the sample input produces the same output as the golden file.
mvn generate-sources
The jflex-maven-plugin reads the grammar src/main/jflex/simple.jflex
and generates a Java scanner Yylex.
Expected output:
target/generated-sources/flex/Yylex.java.
This is defined by the following section
<build>
<plugins>
<plugin>
<groupId>de.jflex</groupId>
<artifactId>jflex-maven-plugin</artifactId>
<version>1.10.0-SNAPSHOT</version>
<executions>
<execution>
<goals>
<goal>generate</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>By default, the jflex-maven-plugin generates a lexer (scanner) for every
file in src/main/jflex/.
mvn compile
The compile phase will generate the sources, and build all Java classes, including those generated automatically.
Expected output:
- Java compiled class files in
target/classes.
Tip In fact, you don't have to invoke mvn generate-sources explicitly,
the compile phase will do it automatically.
mvn test
The test phase does everything above and executes the test in src/test/java.
There is only one test in src/test/java/YylexTest.java.
In this test, the scanner is run with the input file src/test/data/test.txt.
By default, the scanner outputs debugging information about each returned
token to System.out until the end of file is reached, or an error occurs.
But in the test, the output is redirected into an in-memory output stream.
Then, test opens the golden file src/test/data/output.good and compares
with the actual output.
mvn package
The package phase does everything above and packages the jar archive of the Java classes. You can then run
java -jar target/simple-1.0.jar src/test/data/test.txt
to test the lexer.
ant compile
is roughly equivalent to mvn compile above
ant run
will run the generated lexer on the provided sample input.
ant test
will run the same test as in mvn test.
Please see bazel_rules/examples.
make compile
is roughly equivalent to mvn compile, and
make test
will run generated lexer, comparing against expect output.