Re: GSoC on WAL-logging hash indexes

On Thu, Mar 6, 2014 at 11:34 AM, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:

>
> Putting the split-in-progress flag in the new bucket's primary page
> makes a lot of sense. I don't have any problem with dumping the rest
> of it for a first cut if we have a different long-term plan for how to
> improve concurrency, but I don't see much point in going to a lot of
> work to implement a system for WAL logging if we're going to end up
> having to afterwards throw it out and start from scratch to get rid of
> the heavyweight locks - and it's not obvious to me how what you have
> here could be extended to do that.
>

+1 I don't think we have to improve concurrency at the same time as WAL
logging, but we at least have to implement WAL logging in a way that
doesn't foreclose future improvements to concurrency.

I've been tempted to implement a new type of hash index that allows both
WAL and high concurrency, simply by disallowing bucket splits. At the
index creation time you use a storage parameter to specify the number of
buckets, and that is that. If you mis-planned, build a new index with more
buckets, possibly concurrently, and drop the too-small one.

I think it would be easier to add bucket splitting to something like this
than it would be to add WAL logging and concurrency to what we already
have--mostly because I think that route would be more amenable to
incremental programming efforts, and also because if people had an actually
useful hash index type they would use it and see that it worked well
(assuming of course that it does work well), and then be motivated to
improve it.

If this could be done as an extension I would just go (attempt to) do it.
But since WAL isn't pluggable, it can't go in as an extension.

Cheers,

Jeff