From: | Bruce Momjian <bruce(at)momjian(dot)us> |
---|---|
To: | Tomas Vondra <tomas(dot)vondra(at)2ndquadrant(dot)com> |
Cc: | Robert Haas <robertmhaas(at)gmail(dot)com>, Stephen Frost <sfrost(at)snowman(dot)net>, Michael Paquier <michael(at)paquier(dot)xyz>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org> |
Subject: | Re: finding changed blocks using WAL scanning |
Date: | 2019-04-23 01:18:52 |
Message-ID: | 20190423011852.thmribz2y4bihg5e@momjian.us |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
On Mon, Apr 22, 2019 at 08:52:11PM -0400, Bruce Momjian wrote:
> Well, the interesting question is whether the server will generate a
> single modblock file for all WAL in pg_wal only right before we are
> ready to expire some WAL, or whether modblock files will be generated
> offline, perhaps independent of the server, and perhaps by aggregating
> smaller modblock files.
>
> To throw out an idea, what if we had an executable that could generate a
> modblock file by scanning a set of WAL files? How far would that take
> us to meeing incremental backup needs? I can imagine db/relfilenode oid
> volatility could be a problem, but might be fixable.
Well, this actually brings up a bunch of questions:
* How often do we create blockmod files? Per segment, per checkpoint,
at WAL deletion time (1GB?)
* What is the blockmod file format? dboid, relfilenode, blocknum?
Use compression? Sorted?
* How do we create incremental backups?
* What is the incremental backup file format?
* How do we apply incremental backups to base backups?
And there are some secondary questions:
* Can blockmod files be merged?
* Can incremental backups be merged?
* Can blockmod files be used for restore prefetching?
--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ As you are, so once was I. As I am, so you will be. +
+ Ancient Roman grave inscription +
From | Date | Subject | |
---|---|---|---|
Next Message | Andres Freund | 2019-04-23 01:30:05 | Re: memory leak checking |
Previous Message | Amit Langote | 2019-04-23 01:03:15 | Re: bug in update tuple routing with foreign partitions |