Re: Proposal: Incremental Backup

On Wed, Aug 6, 2014 at 01:15:32PM -0300, Claudio Freire wrote:
> On Wed, Aug 6, 2014 at 12:20 PM, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
> >
> > Well, for file-level backups we have:
> >
> > 1) use file modtime (possibly inaccurate)
> > 2) use file modtime and checksums (heavy read load)
> >
> > For block-level backups we have:
> >
> > 3) accumulate block numbers as WAL is written
> > 4) read previous WAL at incremental backup time
> > 5) read data page LSNs (high read load)
> >
> > The question is which of these do we want to implement? #1 is very easy
> > to implement, but incremental _file_ backups are larger than block-level
> > backups. If we have #5, would we ever want #2? If we have #3, would we
> > ever want #4 or #5?
>
> You may want to implement both #3 and #2. #3 would need a config
> switch to enable updating the bitmap. That would make it optional to
> incur the I/O cost of updating the bitmap. When the bitmap isn't
> there, the backup would use #2. Slow, but effective. If slowness is a
> problem for you, you enable the bitmap and do #3.
>
> Sounds reasonable IMO, and it means you can start by implementing #2.

Well, Robert Haas had the idea of a separate process that accumulates
the changed WAL block numbers, making it low overhead. I question
whether we need #2 just to handle cases where they didn't enable #3
accounting earlier. If that is the case, just do a full backup and
enable #3.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ Everyone has their own god. +