Re: BUG #7914: pg_dump aborts occasionally

From: Shin-ichi MORITA <s-morita(at)beingcorp(dot)co(dot)jp>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Andrus <kobruleht2(at)hot(dot)ee>, pgsql-bugs(at)postgresql(dot)org
Subject: Re: BUG #7914: pg_dump aborts occasionally
Date: 2014-05-09 11:18:13
Message-ID: 536CB975.9050104@beingcorp.co.jp
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-bugs

Hi Tom,
Thank you for the information.
And I'm sorry for not replying sooner. It took time to recall this issue.

(2014/05/08 8:58), Tom Lane wrote:
> I started thinking about this issue again after seeing a similar report at
> http://www.postgresql.org/message-id/flat/850BF81CE7324F81AF2A44C8660CF2EE(at)dell2
> and this time I think I see the problem. Look at the loop in pqReadData()
> created by this test:
>
> I've been able to reproduce buffer bloat by inserting some delay into
> this loop (I put "usleep(10000);" in front of the goto above) and then
> sending a steady stream of circa-50KB data messages.

Oh, I'm glad to hear that. Thank you for the explanation.

> exactly the opposite of what generally happens on Unix machines. Perhaps
> Windows schedules differently? Or maybe the reason we're seeing this now,

It might be so. Although I'm not sure why it occurred, I think our
conditions were rare:
Hypervisor: VMware vSphere Hypervisor 5.1
Allocated vCPUs: 1
OS: Windows Server 2012 or 2008R2 (64bit)
Run pg_dump under high load or run pb_dump with the Low priority using
Windows Task Manager

> Anyway, I still don't like your original suggestion of simply skipping
> the pqCheckInBufferSpace call; that's just losing the knowledge we have of
> how big the next message will be. Rather I think what we are missing here
> is that we should deduct the excess space to the left of inStart, and/or
> forcibly left-justify the buffer, before deciding that we need to enlarge
> the buffer. I think it'd be safe for pqCheckInBufferSpace to do that
> (ie move the data within the buffer), though I need to look at the callers
> more closely to be sure.

I agree with you.
The new pqCheckInBufferSpace seems to update inStart, inEnd, and inCursor.
I think it would be OK unless the callers assume they won't be updated.

Thank you.
Shin-ichi

In response to

Browse pgsql-bugs by date

  From Date Subject
Next Message vijay777pawar 2014-05-09 12:19:31 BUG #10273: Not Able To See Full Data
Previous Message Leif Jensen 2014-05-09 06:55:20 Re: [GENERAL] Server process crash - Segmentation fault