Re: XLOG_BLCKSZ vs. wal_buffers table

On Mon, 08 May 2006 19:08:59 +0100
Simon Riggs <simon(at)2ndquadrant(dot)com> wrote:

> On Fri, 2006-05-05 at 16:00 -0700, Mark Wong wrote:
> > On Tue, 02 May 2006 10:52:38 +0100
> > Simon Riggs <simon(at)2ndquadrant(dot)com> wrote:
> >
> > > On Sun, 2006-04-30 at 22:14 -0700, Mark Wong wrote:
> > > > I would have gotten this out sooner but I'm having trouble with our
> > > > infrastructure. Here's a link to a table of data I've started putting
> > > > together regarding XLOG_BLCKSZ and wal_buffers on a 4-way Opteron
> > > > system:
> > > > http://developer.osdl.org/markw/pgsql/xlog_blcksz.html
> > > >
> > > > There are a couple of holes in the table but I think it shows enough
> > > > evidence to say that with dbt2 having a larger XLOG_BLCKSZ improves the
> > > > overall throughput of the test.
> > > >
> > > > I'm planning on continuing to increase XLOG_BLCKSZ and wal_buffers to
> > > > determine when the throughput starts to level out or drop off, and then
> > > > start experimenting with varying BLCKSZ. Let me know if there are other
> > > > things that would be more interesting to experiment with first.
> > >
> > > IMHO you should be testing with higher wal_buffers settings. ISTM likely
> > > that the improved performance is due to there being more buffer space,
> > > rather than actually improving I/O. Setting wal_buffers to something
> > > fairly high say 4096 would completely remove any such effect so we are
> > > left with a view on the I/O.
> >
> > I ran another few tests at the 600 scale factor just in case I was
> > getting close to peaking at 500 warehouses. (Link above has updated
> > data.) With wal_buffers set to 4096 the difference between 2048, 8192,
> > and 32768 seem negligible. Some of the disks are at 90% utilization so
> > perhaps I need to take a close look to make sure none of the other
> > system resources are pegged.
>
> The profiles are fairly different though.
>
> Could you turn full_page_writes = off and do a few more tests? I think
> the full page writes is swamping the xlog and masking the performance we
> might see for normal small xlog writes.
> I'd try XLOG_BLCKSZ = 4096 and 8192 to start with. Thanks.

Ok, will get on it.

> (Is VACUUM running at the start of these tests?)

VACUUM is run immediately after the database tables are loaded. I've
been reloading the database prior to each test.

Mark