Re: Fuzz testing COPY FROM parsing

Greetings,

* Heikki Linnakangas (hlinnaka(at)iki(dot)fi) wrote:
> I've been mucking around with COPY FROM lately, and to test it, I wrote some
> tools to generate input files and load them with COPY FROM:
>
> https://github.com/hlinnaka/pgcopyfuzz

Neat!

> I used a fuzz testing tool called honggfuzz [1] to generate test inputs for
> COPY FROM. At first I tried to use afl and libfuzzer, but honggfuzz was much
> easier to use with PostgreSQL. It has a "persistent fuzzing mode", which
> allows starting the server normally (well, in single-user mode), and calling
> a function to get the next input. With the other fuzzers I tried, you have
> to provide a callback function that the fuzzer calls for each test
> iteration, and that was hard to integrate into the PostgreSQL main
> processing loop.

Yeah, that's been one of the challenges with fuzzers I've played with
too.

> I ran it for about 2 h on my laptop with the patch I was working on [2]. It
> didn't find any crashes, but it generated about 1300 input files that it
> considered "interesting" based on code coverage analysis. When I took those
> generated inputs, and ran them against unpatched and patched server, some
> inputs produced different results. So that revealed a couple of bugs in the
> patch. (I'll post a fixed patched version on that thread soon.)
>
> I hope others find this useful, too.

Nice! I wonder if there's a way to have a buildfarm member or other
system doing this automatically on new commits and perhaps adding
coverage for other things like the JSON code..

Thanks!

Stephen