Re: rmgr hooks (v2)

On Wed, 2009-01-21 at 14:05 +0200, Heikki Linnakangas wrote:
> Simon Riggs wrote:
> > Latest version of rmgr hooks patch for later review in current
> > commitfest.
>
> I'd like to reject this patch.
...
> I've read through all the related threads again, and I just still don't
> see a convincing use case for it. I think that tools that let you
> introspect and modify WAL files should be written as an external
> toolkit, like pglesslog.

The only reasonable way to examine the contents of WAL files is with
reference to a copy of the catalog that wrote them, timed *exactly* in
synchronisation with the WAL stream.

If somebody issued
CREATE TABLE x
INSERT INTO x
DROP TABLE
then the only time you can reasonably look at the data from the insert
is while replaying that record. At no other time does the data have
certain meaning.

So you *must* replay catalog entries and recreate the original catalog
in exact synchronisation with reading WAL files. Recreating the catalog
can only be done by Postgres itself. It simply isn't practical to do
this all with an external tool, or even link in to replay somehow to
keep replay and the reading of the external file synchronised. If it
*was*, somebody would have done it already - some have already tried and
failed.

(I haven't suggested modifying WAL files, BTW, not sure where that came
from).

> Regarding the example plugin included, for debugging purposes you could
> just compile with WAL_DEBUG, and the plugin to suppress actions for all
> but one database is clearly not ready for any real work. It only
> suppresses heapam records, replaying index updates and full-page-images
> as usual, and it requires that you know the Oid of the database,

They're minor examples, so don't reject the plugin patch because the
example of usage isn't as useful as it could be. I'm hardly likely to
invest lots of time in a plugin while the approach has not been agreed,
am I?

It is viable for us to filter WAL records in this way, and not very
viable any other way. It doesn't require you to know the Oid of the
database, cos you can look that up in the catalog (with hot standby).
The example plugin doesn't do that, but it could.

So two use cases: inspecting WAL and filtering records before applying
them are covered here. Pluggable indexes is another, and there are
others also, as discussed on the original patch.

--
Simon Riggs www.2ndQuadrant.com
PostgreSQL Training, Services and Support