This repository contains several JavaScript-based fuzzers. jsfunfuzz tests JavaScript engines and can run in a JavaScript shell, compare_jit compares output from SpiderMonkey using different flags, while randorderfuzz throws in random tests from the mozilla-central directory into generated jsfunfuzz output.

Most of the code other than testcase generation is written in Python: restarting the program when it exits or crashes, noticing evidence of new bugs from the program's output, reducing testcases, and identifying when regressions were introduced.

Setup

Install the required pip packages using pip install -r requirements.txt (assuming you are in the funfuzz repository).

Some parts of the fuzzer will only activate if the Python scripts can find your mozilla-central tree:

mkdir -p ~/trees/
hg clone https://hg.mozilla.org/mozilla-central/ ~/trees/mozilla-central/

Some parts of the harness assume a clean Mercurial clone of the mozilla trees. There is insufficient testing with Git for now - please file an issue if you hit problems with Git repositories of mozilla trees.

If you want to use these scripts to compile SpiderMonkey, install the usual prerequisites for building SpiderMonkey. There are additional requirements for building with Address Sanitizer.

Windows (only 64-bit supported)

1. Install MozillaBuild (Using compile_shell for SpiderMonkey requires at least version 3.2).
2. Install Git to clone these funfuzz repositories.
3. Install Debugging Tools for Windows to get cdb.exe and thus stacks from crashes.
4. Make sure you install at least Microsoft Visual Studio 2017 (Community Edition is recommended) as per the build instructions above in the Setup section.
5. Run start-shell.bat to get a MSYS shell. You can use Git by calling its absolute path, e.g. /c/Program\ Files/Git/bin/git.exe.
   1. Run the batch file with administrator privileges to get gflags analysis working correctly.

Mac

1. If you encounter problems accessing the compiler, try re-running this command:

xcode-select --install

especially after updating major/minor OS versions. This sometimes manifests on Mac OS X Combo updates.

2. Install LLVM via Homebrew, to get llvm-symbolizer needed for symbolizing ASan crash stacks.

brew install llvm

Linux

1. To ensure your core dumps don't get mixed up when multiple instances crash at the same time, run:

echo -n 1 | sudo tee /proc/sys/kernel/core_uses_pid

2. Install 32-bit libraries to compile 32-bit binaries:

• Debian/Ubuntu: sudo apt-get install lib32z1 gcc-multilib g++-multilib
• Fedora: (Fedora is known to work, however the exact library names are unknown for now.)

3. Install gdb:

• Debian/Ubuntu: sudo apt-get install gdb
• Fedora: Please ensure that all development packages are installed (see rpm -qa "*devel"), and run yum install gdb

4. Install clang for clang/ASan builds:

• Debian/Ubuntu: sudo apt-get install clang
• Clang is used for 64-bit builds, while GCC is used for some older 32-bit builds

Running funfuzz

To run only the js fuzzers which compiles shells with random configurations every 8 hours and tests them:

<python executable> -u funfuzz.loop_bot -b "--random" --target-time 28800 | tee ~/log-loop_botPy.txt

To test a patch (assuming patch is in ~/patch.diff) against a specific branch (assuming Mercurial mozilla-inbound is in ~/trees/mozilla-inbound), using a debug 64-bit deterministic shell configuration, every 8 hours:

<python executable> -u funfuzz.loop_bot -b "--enable-debug -R ~/trees/mozilla-inbound -P ~/patch.diff" --target-time 28800 | tee ~/log-loop_botPy.txt

In js mode, loop_bot makes use of:

• compile_shell
• jsfunfuzz
• compare_jit (if testing deterministic builds)
• randorderfuzz (included in funfuzz, if tests are present in the mozilla repository)
• funbind (Linux-only, included in funfuzz, if binaryen can be downloaded)
• autobisectjs (if the mozilla repository is present).

The parameters in -b get passed into compile_shell and autobisectjs.

You will also need to need a ~/.fuzzmanagerconf file, similar to:

[Main]
serverhost = <your hostname>
serverport = <your port>
serverproto = https
serverauthtoken = <if any>
sigdir = /Users/<your username>/sigcache/
tool = jsfunfuzz

Replace anything between < and > with your desired parameters.

FAQ:

Q: What platforms does funfuzz run on?

A: compile_shell has been tested on:

• Windows 10 with MozillaBuild 3.2
• macOS 10.13 and 10.14
• Ubuntu 18.04 LTS (only LTS versions supported going forward)

Fedora Linux and openSUSE Leap (42.3 and later) have not been tested extensively and there may be a few bugs along the way.

The following operating systems are less common and while they may still work, be prepared to expect issues along the way:

• Windows 7, 8 / Windows 8.1
• Windows Server 2012 R2
• Ubuntu Linux 16.04 LTS (install Python 3.6 via a PPA)
• Ubuntu Linux 15.10 and prior

Support for the following operating systems have been removed:

• Windows Vista, Windows XP and earlier
• Mac OS X 10.12 and earlier
• Ubuntu Linux 13.10 and earlier
• Ubuntu (and variants) on ARM ODROID boards

Q: What version of Python does funfuzz require?

A: Python 3.6+