Re: Re: Request for Patch Feedback: Lag & Lead Window Functions Can Ignore Nulls

From: Nicholas White <n(dot)j(dot)white(at)gmail(dot)com>
To: Heikki Linnakangas <hlinnakangas(at)vmware(dot)com>
Cc: Robert Haas <robertmhaas(at)gmail(dot)com>, Alvaro Herrera <alvherre(at)2ndquadrant(dot)com>, Peter Eisentraut <peter_e(at)gmx(dot)net>, Jeff Davis <pgsql(at)j-davis(dot)com>, Dean Rasheed <dean(dot)a(dot)rasheed(at)gmail(dot)com>, Troels Nielsen <bn(dot)troels(at)gmail(dot)com>, Hitoshi Harada <umi(dot)tanuki(at)gmail(dot)com>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Re: Request for Patch Feedback: Lag & Lead Window Functions Can Ignore Nulls
Date: 2013-09-30 15:22:42
Message-ID: CA+=vxNbpWEyCsjrEdh2VgFCJcTv6aafbR_STggNPtAnoWm8bhw@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

I've attached another iteration of the lead-lag patch.

> I suggest you run that over your local copy before your next submission

I ran pgindent before generating my patch (without -w this time), and
I've got a few more whitespace differences in the files that I
touched. I hope that hasn't added too much noise.

> I suggest enclosing that in /*---- to avoid the problem.

Done

> create a new windowapi.h function which returns the MemoryContext for the partition
...
> But even if we did decide to switch memory contexts on every call, it would still be much cleaner than this.

I've removed all the bms_initalize code from the patch and am using
this solution. As the partition memory is zero-initialised I just
store a Bitmapset pointer in the WinGetPartitionLocalMemory. The
bms_add_member and bms_is_member functions behave sensibly for
null-pointer inputs (they return a bms_make_singleton in the current
memory context and false respectively). I've surrounded the calls to
bms_make_singleton with a memory context switch (to the partition's
context) so the Bitmapset stays in the partition's context.

> Maybe we should have a new entry point in bitmapset.h, say "bms_grow" that ensures you have enough space for that many bits

This would be useful, as currently n additions require O(n) repallocs,
especially as I'm iterating through the indices in ascending order.
However, I'd rather "cheat" as I know the number of bits I'll need up
front; I can just set the (n+1)-th bit to force a single repalloc to
the final size. It's worth noting that other Bitmap implementations
(e.g. Java's java.util.BitSet) try to minimise re-allocations by
increasing the size to (e.g.) Max(2 * current size, n) if a re-size is
needed.

> but don't you need to free the clone in the branch that finds a duplicate window spec?

Good catch - I've fixed that

> Is this parent/child relationship thingy a preexisting concept

Yes, although it's not very well documented. I've added a lot of
documentation to the WindowDef struct in
src/include/nodes/parsenodes.h to explain which of the struct's
members use this mechanism. The WindowDef is very much like an object
in a higher-level language, where some of the members are 'virtual',
so use the parent's version if they don't have a value, and some
members are 'final', so values in this member in child WindowDefs are
ignore (i.e. the parent WindowDef's value is always used). I don't
think this degree of complexity is necessary for the lead-lag patch
alone, but since it was there I decided to take advantage of it.

Thanks -

Nick

Attachment Content-Type Size
lead-lag-ignore-nulls.patch application/octet-stream 44.1 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Robert Haas 2013-09-30 15:32:34 Re: INSERT...ON DUPLICATE KEY LOCK FOR UPDATE
Previous Message Andres Freund 2013-09-30 14:50:28 Cmpact commits and changeset extraction