Re: XLOG_BLCKSZ vs. wal_buffers table

On Fri, 2006-05-05 at 16:00 -0700, Mark Wong wrote:
> On Tue, 02 May 2006 10:52:38 +0100
> Simon Riggs <simon(at)2ndquadrant(dot)com> wrote:
>
> > On Sun, 2006-04-30 at 22:14 -0700, Mark Wong wrote:
> > > I would have gotten this out sooner but I'm having trouble with our
> > > infrastructure. Here's a link to a table of data I've started putting
> > > together regarding XLOG_BLCKSZ and wal_buffers on a 4-way Opteron
> > > system:
> > > http://developer.osdl.org/markw/pgsql/xlog_blcksz.html
> > >
> > > There are a couple of holes in the table but I think it shows enough
> > > evidence to say that with dbt2 having a larger XLOG_BLCKSZ improves the
> > > overall throughput of the test.
> > >
> > > I'm planning on continuing to increase XLOG_BLCKSZ and wal_buffers to
> > > determine when the throughput starts to level out or drop off, and then
> > > start experimenting with varying BLCKSZ. Let me know if there are other
> > > things that would be more interesting to experiment with first.
> >
> > IMHO you should be testing with higher wal_buffers settings. ISTM likely
> > that the improved performance is due to there being more buffer space,
> > rather than actually improving I/O. Setting wal_buffers to something
> > fairly high say 4096 would completely remove any such effect so we are
> > left with a view on the I/O.
>
> I ran another few tests at the 600 scale factor just in case I was
> getting close to peaking at 500 warehouses. (Link above has updated
> data.) With wal_buffers set to 4096 the difference between 2048, 8192,
> and 32768 seem negligible. Some of the disks are at 90% utilization so
> perhaps I need to take a close look to make sure none of the other
> system resources are pegged.

The profiles are fairly different though.

Could you turn full_page_writes = off and do a few more tests? I think
the full page writes is swamping the xlog and masking the performance we
might see for normal small xlog writes.
I'd try XLOG_BLCKSZ = 4096 and 8192 to start with. Thanks.

(Is VACUUM running at the start of these tests?)

--
Simon Riggs
EnterpriseDB http://www.enterprisedb.com