Re: Fuzz testing COPY FROM parsing

On 2/5/21 2:50 PM, Heikki Linnakangas wrote:
> On 05/02/2021 21:16, Andrew Dunstan wrote:
>>
>> On 2/5/21 10:54 AM, Stephen Frost wrote:
>>> * Heikki Linnakangas (hlinnaka(at)iki(dot)fi) wrote:
>>>> I ran it for about 2 h on my laptop with the patch I was working on
>>>> [2]. It
>>>> didn't find any crashes, but it generated about 1300 input files
>>>> that it
>>>> considered "interesting" based on code coverage analysis. When I
>>>> took those
>>>> generated inputs, and ran them against unpatched and patched
>>>> server, some
>>>> inputs produced different results. So that revealed a couple of
>>>> bugs in the
>>>> patch. (I'll post a fixed patched version on that thread soon.)
>>>>
>>>> I hope others find this useful, too.
>>> Nice!  I wonder if there's a way to have a buildfarm member or other
>>> system doing this automatically on new commits and perhaps adding
>>> coverage for other things like the JSON code..
>>
>> Not easily in the buildfarm as it is today. We can easily create modules
>> for extensions and other things that don't require modification of core
>> code, but things that require patching core code are a whole different
>> story.
>
> It might be possible to call the fuzzer's HF_ITER() function from a C
> extension instead. So you would run a query like "SELECT
> next_fuzz_iter()" in a loop, and next_fuzz_iter() would be a C
> function that calls HF_ITER(), and executes the actual query with SPI.
>
> That said, I don't think it's important to run the fuzzer in the
> buildfarm. It should be enough to do that every once in a while, when
> you modify the COPY FROM code (or something else that you want to fuzz
> test). But we could easily include the test inputs generated by the
> fuzzer in the regular tests. We've usually been very frugal in adding
> tests, though, to keep the time it takes to run all the tests short.
>
>

This strikes me as a better design in any case.

cheers

andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com