i#1734 Dr. Fuzz: add option -fuzz_libfuzzer

Commit log for first patchset:
---------------
i#1734 Dr. Fuzz: add option -fuzz_libfuzzer

- add option -fuzz_libfuzzer for fuzzing tests written for LibFuzzer
- add shadow_init_region to initialize shadow memory for app
- remove input_buffer_copy, not used anywhere
---------------

[bruening, 2015-12-16 23:52:13]

Please add sthg to drfuzz.dox too.

I'm confused about how this interacts w/ your other CL wrt lock usage, and about
why libfuzzer has different shadow behavior.

https://codereview.appspot.com/277410043/diff/1/drmemory/fuzzer.c
File drmemory/fuzzer.c (right):

https://codereview.appspot.com/277410043/diff/1/drmemory/fuzzer.c#newcode99
drmemory/fuzzer.c:99: * is safe to access the input_size fields of each state
while
fields??

https://codereview.appspot.com/277410043/diff/1/drmemory/fuzzer.c#newcode99
drmemory/fuzzer.c:99: * is safe to access the input_size fields of each state
while
How does this jive w/ your other CL which seemed to include many fields in this
category?

https://codereview.appspot.com/277410043/diff/1/drmemory/fuzzer.c#newcode146
drmemory/fuzzer.c:146: /* Protects the fuzz_state_t field input_size for access
from
Shouldn't this be merged w/ your other CL first?  This lock was used for
multiple fields, wasn't it?

https://codereview.appspot.com/277410043/diff/1/drmemory/fuzzer.c#newcode894
drmemory/fuzzer.c:894: /* For fuzzing libfuzzer program, we mark the shadow
memory as initiailized.
!!

https://codereview.appspot.com/277410043/diff/1/drmemory/fuzzer.c#newcode894
drmemory/fuzzer.c:894: /* For fuzzing libfuzzer program, we mark the shadow
memory as initiailized.
grammar

https://codereview.appspot.com/277410043/diff/1/drmemory/fuzzer.c#newcode894
drmemory/fuzzer.c:894: /* For fuzzing libfuzzer program, we mark the shadow
memory as initiailized.
Wait, I don't get it: why is the shadow different for libfuzzer?  Please explain
in the comment (and maybe in commit log too).

https://codereview.appspot.com/277410043/diff/1/drmemory/fuzzer.c#newcode895
drmemory/fuzzer.c:895: * We do not need to initilaize the input buffer.
!!

https://codereview.appspot.com/277410043/diff/1/drmemory/options.c
File drmemory/options.c (right):

https://codereview.appspot.com/277410043/diff/1/drmemory/options.c#newcode562
drmemory/options.c:562:
options.fuzz_function[strlen(FUZZ_FUNC_LIBFUZZER_NAME)+1] = '\0';
Seem buggy: if the strncpy did truncate, this would then overflow.  I would
suggest just using NULL_TERMINATE_BUFFER.

https://codereview.appspot.com/277410043/diff/1/drmemory/optionsx.h
File drmemory/optionsx.h (right):

https://codereview.appspot.com/277410043/diff/1/drmemory/optionsx.h#newcode664
drmemory/optionsx.h:664: "Fuzz tests written for LibFuzzer, where the target
function is "FUZZ_FUNC_LIBFUZZER_NAME".")
Explain that -fuzz_function doesn't need to be specified

https://codereview.appspot.com/277410043/diff/1/drmemory/shadow.h
File drmemory/shadow.h (right):

https://codereview.appspot.com/277410043/diff/1/drmemory/shadow.h#newcode207
drmemory/shadow.h:207: shadow_init_region(app_pc start, size_t size, uint
shadow_value);
The function name is confusing.  Better to put save in the name somewhere so
this isn't confused w/ writing real shadow values.  Maybe
"shadow_save_region_as()"

[zhaoqin, 2015-12-17 04:33:52]

https://codereview.appspot.com/277410043/diff/1/drmemory/fuzzer.c
File drmemory/fuzzer.c (right):

https://codereview.appspot.com/277410043/diff/1/drmemory/fuzzer.c#newcode99
drmemory/fuzzer.c:99: * is safe to access the input_size fields of each state
while
On 2015/12/16 23:52:13, bruening wrote:
> How does this jive w/ your other CL which seemed to include many fields in
this
> category?

It was trying to remove the input_buffer_copy field, which is actually not used
anywhere.
Probably better a separate CL after another CL in.
will revert this part of changes.

https://codereview.appspot.com/277410043/diff/1/drmemory/fuzzer.c#newcode894
drmemory/fuzzer.c:894: /* For fuzzing libfuzzer program, we mark the shadow
memory as initiailized.
On 2015/12/16 23:52:13, bruening wrote:
> Wait, I don't get it: why is the shadow different for libfuzzer?  Please
explain
> in the comment (and maybe in commit log too).

In libfuzzer, LLVMFuzzerTestOneInput takes a string that initialized by
libfuzzer. I replace it with an allocated memory without initializing it.
so we want to mark the shadow memory as SHADOW_DEFINED. It would also be used
for the case we replace the memory.

https://codereview.appspot.com/277410043/diff/1/drmemory/optionsx.h
File drmemory/optionsx.h (right):

https://codereview.appspot.com/277410043/diff/1/drmemory/optionsx.h#newcode664
drmemory/optionsx.h:664: "Fuzz tests written for LibFuzzer, where the target
function is "FUZZ_FUNC_LIBFUZZER_NAME".")
On 2015/12/16 23:52:13, bruening wrote:
> Explain that -fuzz_function doesn't need to be specified

I have a different idea about adapt to libfuzzer to avoid this option.
Basically, we just need to check the main module if there is a
LLVMFuzzerTestOneInput function, if yes and fuzz_function is not specified, we
will fuzz LLVMFuzzerTestOneInput.

What do you think?

[bruening, 2015-12-17 04:40:42]

https://codereview.appspot.com/277410043/diff/1/drmemory/optionsx.h
File drmemory/optionsx.h (right):

https://codereview.appspot.com/277410043/diff/1/drmemory/optionsx.h#newcode664
drmemory/optionsx.h:664: "Fuzz tests written for LibFuzzer, where the target
function is "FUZZ_FUNC_LIBFUZZER_NAME".")
On 2015/12/17 04:33:52, zhaoqin wrote:
> On 2015/12/16 23:52:13, bruening wrote:
> > Explain that -fuzz_function doesn't need to be specified
> 
> I have a different idea about adapt to libfuzzer to avoid this option.
> Basically, we just need to check the main module if there is a
> LLVMFuzzerTestOneInput function, if yes and fuzz_function is not specified, we
> will fuzz LLVMFuzzerTestOneInput.
> 
> What do you think?

Are there cases where LLVMFuzzerTestOneInput would be in a library?  That is my
only concern, o/w seems good.

[zhaoqin, 2015-12-17 04:53:58]

On 2015/12/17 04:40:42, bruening wrote:
> https://codereview.appspot.com/277410043/diff/1/drmemory/optionsx.h
> File drmemory/optionsx.h (right):
> 
> https://codereview.appspot.com/277410043/diff/1/drmemory/optionsx.h#newcode664
> drmemory/optionsx.h:664: "Fuzz tests written for LibFuzzer, where the target
> function is "FUZZ_FUNC_LIBFUZZER_NAME".")
> On 2015/12/17 04:33:52, zhaoqin wrote:
> > On 2015/12/16 23:52:13, bruening wrote:
> > > Explain that -fuzz_function doesn't need to be specified
> > 
> > I have a different idea about adapt to libfuzzer to avoid this option.
> > Basically, we just need to check the main module if there is a
> > LLVMFuzzerTestOneInput function, if yes and fuzz_function is not specified,
we
> > will fuzz LLVMFuzzerTestOneInput.
> > 
> > What do you think?
> 
> Are there cases where LLVMFuzzerTestOneInput would be in a library?  That is
my
> only concern, o/w seems good.

I would say it is possible, but not very likely.
In that case, I think it makes sense to let user to specify the fuzz_function.
We can check it on every module.
Basically, we set LLVMFuzzerTestOneInput as another default fuzz target and may
need adjust some other default options value.