Re: Using streaming replication as log archiving

From: "Kevin Grittner" <Kevin(dot)Grittner(at)wicourts(dot)gov>
To: "Magnus Hagander" <magnus(at)hagander(dot)net>, "PostgreSQL-development" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Using streaming replication as log archiving
Date: 2010-09-29 21:45:11
Message-ID: 4CA36D17020000250003613B@gw.wicourts.gov
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Magnus Hagander <magnus(at)hagander(dot)net> wrote:

> Comments and contributions are most welcome.

This is probably too esoteric to be worked on yet, but for this to
be useful for us we would need to pass the resulting files through
pg_clearxlogtail and gzip in an automated fashion. And we would
need to do regular log file archiving in parallel with it.

As background, our databases around the state archive to a directory
which is then pushed via rsync to a "dumb" backup location in the
same room as the database server (we're lucky to have rsync on the
target of this copy; any other executable is out of the question),
and the same directory is pulled via rsync to a central location.
We would be interested in using streaming replication to a tool such
as you describe for the copy to the central location, but since we
would still be forcing a wal-file switch once per hour we would need
the current capability to shrink an "empty" file from 16MB to 16kB
using the above-mentioned tools.

Also, a the ability to limit bandwidth would be a nice feature for
us, preferably in a way which could be changed on the fly.

If you could keep the development "friendly" to such features, I may
get around to adding them to support our needs....

-Kevin

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Thom Brown 2010-09-29 22:29:24 Review: Patch for Synchronous Replication
Previous Message Andrew Dunstan 2010-09-29 19:46:10 Re: WIP: extensible enums