Re: [HACKERS] [PATCHES] wal_checksum = on (default) | off

From: "Simon Riggs" <simon(at)2ndquadrant(dot)com>
To: "Bruce Momjian" <bruce(at)momjian(dot)us>
Cc: "Tom Lane" <tgl(at)sss(dot)pgh(dot)pa(dot)us>, "Jim Nasby" <decibel(at)decibel(dot)org>, "Zeugswetter Andreas ADI SD" <ZeugswetterA(at)spardat(dot)at>, <pgsql-hackers(at)postgresql(dot)org>, <pgsql-patches(at)postgresql(dot)org>
Subject: Re: [HACKERS] [PATCHES] wal_checksum = on (default) | off
Date: 2007-01-11 11:32:12
Message-ID: 1168515133.3951.503.camel@silverbirch.site
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers pgsql-patches

On Wed, 2007-01-10 at 23:32 -0500, Bruce Momjian wrote:
> Simon Riggs wrote:
> > On Fri, 2007-01-05 at 22:57 -0500, Tom Lane wrote:
> > > Jim Nasby <decibel(at)decibel(dot)org> writes:
> > > > On Jan 5, 2007, at 6:30 AM, Zeugswetter Andreas ADI SD wrote:
> > > >> Ok, so when you need CRC's on a replicate (but not on the master) you
> > >
> > > > Which sounds to me like a good reason to allow the option in
> > > > recovery.conf as well...
> > >
> > > Actually, I'm not seeing the use-case for a slave having a different
> > > setting from the master at all?
> > >
> > > "My backup server is less reliable than the primary."
> > >
> > > "My backup server is more reliable than the primary."
> > >
> > > Somehow, neither of these statements seem likely to be uttered by
> > > a sane DBA ...
> >
> > If I take a backup of a server and bring it up on a new system, the
> > blocks in the backup will not have been CRC checked before they go to
> > disk.
> >
> > If I take the same server and now stream log records across to it, why
> > *must* that data be CRC checked, when the original data has not been?
> >
> > I'm proposing choice, with a safe default. That's all.
>
> I am assuming this item is dead because no performance results have been
> reported.

It's been on my hold queue, as a result of its lack of clear acceptance.

Results from earlier tests show the routines which are dominated by CRC
checking overhead are prominent in a number of important workloads.
Those workloads all have a substantial disk component, so test results
will vary between no saving at all on a disk-bound system to some
savings on a CPU bound system.

Restore RecordIsValid() #1 on oprofile results at 50-70% CPU

COPY XLogInsert() #1 on oprofile results at 17% CPU
(full_page_writes = on)

OLTP no test with full_page_writes = on (less relevant)

OLTP XLogInsert() #5 on oprofile results at 1.2% CPU
(full_page_writes = off)
Removing the CRC checks on WAL would likely be the easiest to remove 1%
CPU on the system as it stands. Other changes require algorithmic or
architectural changes to improve matters, though gains can be much
larger. 1% doesn't sound much, but PostgreSQL is a very sleek beast
these days. As we improve things in other areas the importance of this
patch as a tuning switch will grow.

Clearly the current patch is not accepted, but can we imagine a patch
that saved substantial CPU time in these areas that would be acceptable?
*Always* as a non-default option, IMHO, with careful documentation as to
its possible use.

--
Simon Riggs
EnterpriseDB http://www.enterprisedb.com

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Simon Riggs 2007-01-11 12:15:50 Re: table partioning performance
Previous Message Gurjeet Singh 2007-01-11 11:17:16 Re: SPAR Simple PostgreSQL AddOn Replication System

Browse pgsql-patches by date

  From Date Subject
Next Message Tom Lane 2007-01-11 14:01:09 Re: [HACKERS] [PATCHES] wal_checksum = on (default) | off
Previous Message Heikki Linnakangas 2007-01-11 10:56:50 Re: [HACKERS] [PATCHES] Last infomask bit