Re: WIP: Access method extendability

On Tue, Oct 28, 2014 at 7:57 PM, Simon Riggs <simon(at)2ndquadrant(dot)com> wrote:

> On 28 October 2014 14:53, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
> > On Tue, Oct 28, 2014 at 10:22 AM, Simon Riggs <simon(at)2ndquadrant(dot)com>
> wrote:
> >> Or put it another way, it will be easier to write new index AMs
> >> because we'll be able to skip the WAL part until we know we want it.
> >
> > I like the feature you are proposing, but I don't think that we should
> > block Alexander from moving forward with a more-extensible WAL format.
> > I believe that's a general need even if we get the features you're
> > proposing, which would reduce the need for it. After all, if somebody
> > builds an out-of-core index AM, ignoring WAL-logging, and then decides
> > that it works well enough that they want to add WAL-logging, I think
> > we should make that possible without requiring them to move the whole
> > thing in-core.
>
> I'm not proposing an alternate or additional feature.
>
> I'm saying that the first essential step in adding WAL support to new
> AMs is to realise that they *will* have bugs (since with the greatest
> respect, the last two AMs from our friends did have multiple bugs) and
> so we must have a mechanism that prevents such bugs from screwing
> everything else up. Which is the mark-corrupt-index and rebuild
> requirement.
>
> We skip straight to the add-buggy-AMs part at our extreme peril. We've
> got about 10x as many users now since the 8.x bugs and all the new
> users like the reputation Postgres has for resilience. I think we
> should put the safety net in place first before we start to climb.
>
>
agree and we thought about this

> The patch as submitted doesn't have any safety checks for whether the
> WAL records refer to persistent objects defined by the AM. At the very
> least we need to be able to isolate an AM to only screw up their own
> objects. Such checks look like they'd require some careful thought and
> refactoring first.
>

the patch Alexander submitted is the PoC, we wanted to hear developers
opinion and
I see no principal objection to work in this direction and we'll continue
to work on all
possible issues.

>
> --
> Simon Riggs http://www.2ndQuadrant.com/
> PostgreSQL Development, 24x7 Support, Training & Services
>
>
> --
> Sent via pgsql-hackers mailing list (pgsql-hackers(at)postgresql(dot)org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-hackers
>