Re: Freeze avoidance of very large table.

Hello,

At Tue, 2 Feb 2016 20:25:23 +0900, Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> wrote in <CAD21AoA5iaKQ6K7gUZyzN2KJnPNMeHc6PPPxj6cJgmssjj=fqw(at)mail(dot)gmail(dot)com>
> On Tue, Feb 2, 2016 at 7:22 PM, Alvaro Herrera <alvherre(at)2ndquadrant(dot)com> wrote:
> > Masahiko Sawada wrote:
> >
> >> I misunderstood. Sorry for noise.
> >> I agree with adding conversion method as a pageConverter routine.
> >
> > \o/
> >
> >> This patch doesn't change page layout actually, but pageConverter
> >> routine checks only the page layout.
> >> And we have to plugin named convertLayout_X_to_Y.
> >>
> >> I think we have two options.
> >>
> >> 1. Change page layout(PG_PAGE_LAYOUT_VERSION) to 5. pg_upgrade detects
> >> it and then converts only VM files.
> >> 2. Change pg_upgrade plugin mechanism so that it can handle other name
> >> conversion plugins (e.g., convertLayout_vm_to_vfm)
> >>
> >> I think #2 is better. Thought?
> >
> > My vote is for #2 as well. Maybe we just didn't have forks when this
> > functionality was invented; maybe the author just didn't think hard
> > enough about what would be the right interface to do it.
>
> Thanks.
>
> I'm planning to change as follows.
> - pageCnvCtx struct has new function pointer convertVMFile().
> If the layout of other relation such as FSM, CLOG in the future, we
> could add convertFSMFile() and convertCLOGFile().
> - Create new library convertLayoutVM_add_frozenbit.c that has
> convertVMFile() function which converts only visibilitymap.
> When rewriting of VM is required, convertLayoutVM_add_frozenbit.so
> is dynamically loaded.
> convertLayout_X_to_Y converts other relation files.
> That is, converting VM and converting other relations are independently done.
> - Current plugin mechanism puts conversion library (*.so) into
> ${bin}/plugins (i.g., new plugin directory is required), but I'm
> thinking to puts it into ${libdir}.
>
> Please give me feedbacks.

I agree that the plugin mechanism would be usable and needs to be
redesigned, but..

Since the destination version is fixed, the advantage of the
plugin mechanism for pg_upgade would be capability to choose a
plugin to load according to some characteristics of the source
database. What do you think the trigger characteristics for
convertLayoutVM_add_frozenbit.so or similars? If it is hard-coded
like what transfer_single_new_db is doing for fsm and vm, I
suppose the module is not necessary to be a plugin.

regards,

--
Kyotaro Horiguchi
NTT Open Source Software Center