Re: GSoC on WAL-logging hash indexes

On Thu, Mar 6, 2014 at 8:11 AM, Heikki Linnakangas
<hlinnakangas(at)vmware(dot)com> wrote:
> I don't think it's necessary to improve concurrency just to get WAL-logging.
> Better concurrency is a worthy goal of its own, of course, but it's a
> separate concern.

To some extent, I agree, but only to some extent. To make hash
indexes generally usable, we really need to solve both problems. When
I got rid of just some of the heavyweight locking in commit
76837c1507cb5a5a0048046233568447729e66dd, the results were pretty
dramatic at higher levels of concurrency:

http://www.postgresql.org/message-id/CA+Tgmoaf=nOJxLyzGcbrrY+pe-0VLL0vfHi6tjdM3fFtVwsOmw@mail.gmail.com

But there was still an awful lot of contention inside the heavyweight
lock manager, and I don't think hash indexes are going to be ready for
prime time until we solve that problem.

> This is similar to your description, as you scan both buckets if you see an
> interrupted split, but doesn't require the per-tuple moved-by-split flag, or
> waiting out scans. Also, I put the split-in-progress flag in the new
> bucket's primary page instead of the metapage. That allows multiple splits
> to be in progress at the same time.

Putting the split-in-progress flag in the new bucket's primary page
makes a lot of sense. I don't have any problem with dumping the rest
of it for a first cut if we have a different long-term plan for how to
improve concurrency, but I don't see much point in going to a lot of
work to implement a system for WAL logging if we're going to end up
having to afterwards throw it out and start from scratch to get rid of
the heavyweight locks - and it's not obvious to me how what you have
here could be extended to do that.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company