pg_combinebackup --incremental

From: Robert Haas <robertmhaas(at)gmail(dot)com>
To: "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: pg_combinebackup --incremental
Date: 2024-11-04 17:53:05
Message-ID: CA+TgmobBx9Qu9HKgE2+ne0xLg8R08GjVXDspe=hO1J7W0Lxh9Q@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

When I gave my talk on pg_basebackup at pgconf.eu, people took the
opportunity to mention various improvements which they would find
useful. One of those improvements was the ability to combine several
incremental backups into one. This might be useful if you take very
frequent incremental backups but want to roll up older ones to reduce
storage requirements. For example, you could take an incremental
backup every hour, but then once more than a day has gone by, you
might roll up those up six at a time into bigger incremental backups
so that you store 4 backups per day instead of 24 backups per day.

The attached patch set lets you do this. I have only tested it a
little bit, and before it gets committed, it needs substantially more
testing, as well as code review. I plan to write some TAP tests to
include in the patch, too, but I haven't done that yet. The idea is
that instead of doing this:

$ pg_combinebackup full incr1 incr2 incr3 incr4 incr5 -o result

You could instead do this:

$ pg_combinebackup -i incr1 incr2 incr3 -o incr1-3
$ rm -rf incr{1,2,3}
<time passes>
$ pg_combinebackup full incr1-3 incr5 -o result

If you're interested in this feature, please give this a try and let
me know what you find out!

Thanks,

--
Robert Haas
EDB: http://www.enterprisedb.com

Attachment Content-Type Size
v1-0001-pg_combinebackup-Factor-some-code-out-of-write_re.patch application/octet-stream 9.8 KB
v1-0002-pg_combinebackup-Add-a-incremental-option.patch application/octet-stream 28.1 KB

Browse pgsql-hackers by date

  From Date Subject
Next Message Jelte Fennema-Nio 2024-11-04 18:20:59 Re: Allow specifying a dbname in pg_basebackup connection string
Previous Message Ranier Vilela 2024-11-04 17:38:54 Re: define pg_structiszero(addr, s, r)