Re: Freeze avoidance of very large table.

On 9 December 2015 at 18:31, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:

> On Mon, Nov 30, 2015 at 12:58 PM, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
> > On Mon, Nov 30, 2015 at 10:48:04PM +0530, Masahiko Sawada wrote:
> >> On Sun, Nov 29, 2015 at 2:21 AM, Jeff Janes <jeff(dot)janes(at)gmail(dot)com>
> wrote:
> >> > On Tue, Nov 24, 2015 at 3:13 PM, Masahiko Sawada <
> sawada(dot)mshk(at)gmail(dot)com> wrote:
> >> >>
> >> >> Yeah, we need to consider to compute checksum if enabled.
> >> >> I've changed the patch, and attached.
> >> >> Please review it.
> >> >
> >> > Thanks for the update. This now conflicts with the updates doesn to
> >> > fix pg_upgrade out-of-space issue on Windows. I've fixed (I think) the
> >> > conflict in order to do some testing, but I'd like to get an updated
> >> > patch from the author in case I did it wrong. I don't want to find
> >> > bugs that I just introduced myself.
> >> >
> >>
> >> Thank you for having a look.
> >
> > I would not bother mentioning this detail in the pg_upgrade manual page:
> >
> > + Since the format of visibility map has been changed in version 9.6,
> > + <application>pg_upgrade</> creates and rewrite new
> <literal>'_vm'</literal>
> > + file even if upgrading from 9.5 or before to 9.6 or later with link
> mode (-k).
>
> Really? I know we don't always document things like this, but it
> seems like a good idea to me that we do so.
>

Agreed.

For me, rewriting the visibility map is a new data loss bug waiting to
happen. I am worried that the group is not taking seriously the potential
for catastrophe here. I think we can do it, but I think it needs these
things

* Clear notice that it is happening unconditionally and unavoidably
* Log files showing it has happened, action by action
* Very clear mechanism for resolving an incomplete or interrupted upgrade
process. Which VMs got upgraded? Which didn't?
* Ability to undo an upgrade attempt, somehow, ideally automatically by
default
* Ability to restart a failed upgrade attempt without doing a "double
upgrade", i.e. ensure transformation is immutable

--
Simon Riggs http://www.2ndQuadrant.com/
<http://www.2ndquadrant.com/>
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services