Re: 8.4 open item: copy performance regression?

On Sun, 2009-06-21 at 20:37 +0300, Heikki Linnakangas wrote:
> Robert Haas wrote:
> > On Sun, Jun 21, 2009 at 11:52 AM, Tom Lane<tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> >> So to my mind, the only question left to answer (at least for the 8.4
> >> cycle) is "is 16MB enough, or do we want to make the ring even bigger?".
> >> Right at the moment I'd be satisfied with 16, but I wonder whether there
> >> are scenarios where 32MB would show a significant advantage.
> >
> > Even 32MB is not that much. It seems to me that in any realistic
> > production scenario you're going to have at least half a gig of shared
> > buffers, so we're really talking about at most one-sixteenth of the
> > shared buffer arena, and possibly quite a bit less. I think that's
> > pretty conservative.
>
> I was going to say that since we flush the WAL every 16MB anyway (at
> every XLOG file switch), you shouldn't see any benefit with larger ring
> buffers, since to fill 16MB of data you're not going to write more than
> 16MB WAL. But then I realized that that's not true if you have an
> unusually low fillfactor. If you only fill each page say 50% full,
> you're going to use 32MB worth of data pages but only write 16MB of WAL.
> And maybe you could have a situation like that with very wide rows as
> well, with wasted space on each page that's not enough to store one more
> row.

If walwriter is working correctly then it should be writing and fsyncing
WAL, while the COPY process just inserts WAL. I don't see that as an
argument to limit us to 16MB. But I take your point as being an argument
in favour of that as a consensus value for us to choose.

--
Simon Riggs www.2ndQuadrant.com
PostgreSQL Training, Services and Support