Re: Implementing incremental backup

>> I'm thinking of implementing an incremental backup tool for
>> PostgreSQL. The use case for the tool would be taking a backup of huge
>> database. For that size of database, pg_dump is too slow, even WAL
>> archive is too slow/ineffective as well. However even in a TB
>> database, sometimes actual modified blocks are not that big, may be
>> even several GB. So if we can backup those modified blocks only,
>> that would be an effective incremental backup method.
>
> I'm trying to figure out how that's actually different from WAL..? It
> sounds like you'd get what you're suggesting with simply increasing the
> checkpoint timeout until the WAL stream is something which you can keep
> up with. Of course, the downside there is that you'd have to replay
> more WAL when recovering.

Yeah, at first I thought using WAL was a good idea. However I realized
that the problem using WAL is we cannot backup unlogged tables because
they are not written to WAL.
--
Tatsuo Ishii
SRA OSS, Inc. Japan
English: http://www.sraoss.co.jp/index_en.php
Japanese: http://www.sraoss.co.jp