Re: 8.4 open item: copy performance regression?

Heikki Linnakangas <heikki(dot)linnakangas(at)enterprisedb(dot)com> writes:
> Tom Lane wrote:
>> I'm not convinced that WAL segment boundaries are particularly relevant
>> to this. The unit of flushing is an 8K page, not a segment.

> We fsync() the old WAL segment every time we switch to a new WAL
> segment. That's what I meant by "flush".

> If the walwriter is keeping up, it will fsync() the WAL more often, but
> 16MB is the maximum distance between fsync()s.

I'm still not convinced --- to my mind the issue is not whether fsyncs
happen but whether the COPY process has to wait for 'em, and I don't
think that segment boundaries directly affect that. I'd still be
interested to see similar measurements done with different wal_buffer
settings.

However, in the interests of getting this resolved in time for 8.4.0,
I propose that we just settle on 16MB as the bulkwrite ring buffer size.
There doesn't seem to be any evidence that a larger size will make for
a significant improvement, and we shouldn't allow COPY to trash a bigger
fraction of the arena than it really has to.

regards, tom lane