Re: Freeze avoidance of very large table.

On Thu, Apr 23, 2015 at 4:19 AM, Simon Riggs <simon(at)2ndquadrant(dot)com> wrote:
> We were talking about having an incremental backup map also. Which sounds a
> lot like the freeze map.

Yeah, possibly. I think we should try to set things up so that the
backup map can be updated asynchronously by a background worker, so
that we're not adding more work to the foreground path just for the
benefit of maintenance operations. That might make the logic for
autovacuum to use it a little bit more complex, but it seems
manageable.

> We only need a freeze/backup map for larger relations. So if we map 1000
> blocks per map page, we skip having a map at all when size < 1000.

Agreed. We might also want to map multiple blocks per map slot - e.g.
one slot per 32 blocks. That would keep the map quite small even for
very large relations, and would not compromise efficiency that much
since reading 256kB sequentially probably takes only a little longer
than reading 8kB.

I think the idea of integrating the freeze map into the VM fork is
also worth considering. Then, the incremental backup map could be
optional; if you don't want incremental backup, you can shut it off
and have less overhead.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company