Re: atomic pin/unpin causing errors

On Tue, May 3, 2016 at 4:05 PM, Andres Freund <andres(at)anarazel(dot)de> wrote:
> Hi Jeff,
>
> On 2016-04-29 10:38:55 -0700, Jeff Janes wrote:
>> I've bisected the errors I was seeing, discussed in
>> http://www.postgresql.org/message-id/CAMkU=1xQEhC0Ok4d+tkjFQ1nvUhO37PYRKhJP6Q8oxifMx7OwA@mail.gmail.com
>>
>> It look like they first appear in:
>>
>> commit 48354581a49c30f5757c203415aa8412d85b0f70
>> Author: Andres Freund <andres(at)anarazel(dot)de>
>> Date: Sun Apr 10 20:12:32 2016 -0700
>>
>> Allow Pin/UnpinBuffer to operate in a lockfree manner.
>>
>>
>> I get the errors:
>>
>> ERROR: attempted to delete invisible tuple
>> STATEMENT: update foo set count=count+1,text_array=$1 where text_array @> $2
>>
>> And also:
>>
>> ERROR: unexpected chunk number 1 (expected 2) for toast value
>> 85223889 in pg_toast_16424
>> STATEMENT: update foo set count=count+1 where text_array @> $1
>
> Hm. I appear to have trouble reproducing this issue (continuing to try)
> on master as of 8826d8507. Is there any chance you could package up a
> data directory after the issue hit?

I'll look into. I haven't been saving them, as they are very large
(tens of GB) by the time the errors happen. In case I can't find a
way to transfer that much data, is there something I could do in situ
to debug it?

>
>
>> (This was all run using Teodor's test-enabling patch
>> gin_alone_cleanup-4.patch, so as not to change horses in midstream.
>> Now that a version of that patch has been committed, I will try to
>> repeat this in HEAD)
>
> Any news on that front?

I couldn't reproduce it in 82881b2b432c9433b45a (which is what HEAD
was at the time).

The last commit I saw the problem in was 8f1911d5e6d5, and in that
commit it took longer than usual to see the error, and I never saw at
all in one run (which lead me down the wrong path in git bisect) but
then saw errors upon another try. Up until that commit, it seemed to
give the errors like clockwork, always after 8 to 10 hours of running.

I also have never seen the errors with the crashing turned off. I
even tried it with autovac off and
autovacuum_freeze_max_age=1500000000 (to emulate the way vacuum never
gets a chance to run to completion in the crashing mode) and then I
don't get any errors up to the point I run out of disk space.

Cheers,

Jeff