You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
131 lines
5.2 KiB
131 lines
5.2 KiB
JFuzz
|
|
=====
|
|
|
|
JFuzz is a tool for generating random programs with the objective
|
|
of fuzz testing the ART infrastructure. Each randomly generated program
|
|
can be run under various modes of execution, such as using the interpreter,
|
|
using the optimizing compiler, using an external reference implementation,
|
|
or using various target architectures. Any difference between the outputs
|
|
(**divergence**) may indicate a bug in one of the execution modes.
|
|
|
|
JFuzz can be combined with DexFuzz to get multi-layered fuzz testing.
|
|
|
|
How to run JFuzz
|
|
================
|
|
|
|
jfuzz [-s seed] [-d expr-depth] [-l stmt-length]
|
|
[-i if-nest] [-n loop-nest] [-v] [-h]
|
|
|
|
where
|
|
|
|
-s : defines a deterministic random seed
|
|
(randomized using time by default)
|
|
-d : defines a fuzzing depth for expressions
|
|
(higher values yield deeper expressions)
|
|
-l : defines a fuzzing length for statement lists
|
|
(higher values yield longer statement sequences)
|
|
-i : defines a fuzzing nest for if/switch statements
|
|
(higher values yield deeper nested conditionals)
|
|
-n : defines a fuzzing nest for for/while/do-while loops
|
|
(higher values yield deeper nested loops)
|
|
-t : defines a fuzzing nest for try-catch-finally blocks
|
|
(higher values yield deeper nested try-catch-finally blocks)
|
|
-v : prints version number and exits
|
|
-h : prints help and exits
|
|
|
|
The current version of JFuzz sends all output to stdout, and uses
|
|
a fixed testing class named Test. So a typical test run looks as follows.
|
|
|
|
jfuzz > Test.java
|
|
mkdir classes
|
|
javac -d classes Test.java
|
|
dx --dex --output=classes.dex classes
|
|
art -cp classes.dex Test
|
|
|
|
How to start JFuzz testing
|
|
==========================
|
|
|
|
run_jfuzz_test.py
|
|
[--num_tests=NUM_TESTS]
|
|
[--device=DEVICE]
|
|
[--mode1=MODE] [--mode2=MODE]
|
|
[--report_script=SCRIPT]
|
|
[--jfuzz_arg=ARG]
|
|
[--true_divergence]
|
|
[--dexer=DEXER]
|
|
[--debug_info]
|
|
|
|
where
|
|
|
|
--num_tests : number of tests to run (10000 by default)
|
|
--device : target device serial number (passed to adb -s)
|
|
--mode1 : m1
|
|
--mode2 : m2, with m1 != m2, and values one of
|
|
ri = reference implementation on host (default for m1)
|
|
hint = Art interpreter on host
|
|
hopt = Art optimizing on host (default for m2)
|
|
tint = Art interpreter on target
|
|
topt = Art optimizing on target
|
|
--report_script : path to script called for each divergence
|
|
--jfuzz_arg : argument for jfuzz
|
|
--true_divergence : don't bisect timeout divergences
|
|
--dexer=DEXER : use either dx or d8 to obtain dex files
|
|
--debug_info : include debugging info
|
|
|
|
How to start JFuzz nightly testing
|
|
==================================
|
|
|
|
run_jfuzz_test_nightly.py
|
|
[--num_proc NUM_PROC]
|
|
|
|
where
|
|
|
|
--num_proc : number of run_jfuzz_test.py instances to run (8 by default)
|
|
|
|
Remaining arguments are passed to run\_jfuzz_test.py.
|
|
|
|
How to start J/DexFuzz testing (multi-layered)
|
|
==============================================
|
|
|
|
run_dex_fuzz_test.py
|
|
[--num_tests=NUM_TESTS]
|
|
[--num_inputs=NUM_INPUTS]
|
|
[--device=DEVICE]
|
|
[--dexer=DEXER]
|
|
[--debug_info]
|
|
|
|
where
|
|
|
|
--num_tests : number of tests to run (10000 by default)
|
|
--num_inputs : number of JFuzz programs to generate
|
|
--device : target device serial number (passed to adb -s)
|
|
--dexer=DEXER : use either dx or d8 to obtain dex files
|
|
--debug_info : include debugging info
|
|
|
|
Background
|
|
==========
|
|
|
|
Although test suites are extremely useful to validate the correctness of a
|
|
system and to ensure that no regressions occur, any test suite is necessarily
|
|
finite in size and scope. Tests typically focus on validating particular
|
|
features by means of code sequences most programmers would expect. Regression
|
|
tests often use slightly less idiomatic code sequences, since they reflect
|
|
problems that were not anticipated originally, but occurred “in the field”.
|
|
Still, any test suite leaves the developer wondering whether undetected bugs
|
|
and flaws still linger in the system.
|
|
|
|
Over the years, fuzz testing has gained popularity as a testing technique for
|
|
discovering such lingering bugs, including bugs that can bring down a system
|
|
in an unexpected way. Fuzzing refers to feeding a large amount of random data
|
|
as input to a system in an attempt to find bugs or make it crash. Generation-
|
|
based fuzz testing constructs random, but properly formatted input data.
|
|
Mutation-based fuzz testing applies small random changes to existing inputs
|
|
in order to detect shortcomings in a system. Profile-guided or coverage-guided
|
|
fuzzing adds a direction to the way these random changes are applied. Multi-
|
|
layered approaches generate random inputs that are subsequently mutated at
|
|
various stages of execution.
|
|
|
|
The randomness of fuzz testing implies that the size and scope of testing is no
|
|
longer bounded. Every new run can potentially discover bugs and crashes that were
|
|
hereto undetected.
|