Re: wip: functions median and percentile

From: Hitoshi Harada <umi(dot)tanuki(at)gmail(dot)com>
To: Dean Rasheed <dean(dot)a(dot)rasheed(at)gmail(dot)com>
Cc: Robert Haas <robertmhaas(at)gmail(dot)com>, Greg Stark <gsstark(at)mit(dot)edu>, Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, David Fetter <david(at)fetter(dot)org>, PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: wip: functions median and percentile
Date: 2010-10-05 06:04:31
Message-ID: AANLkTinKzfkMkM4DA2uoUFdqR-rOLBKcR-KfhFETovsP@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers pgsql-rrreviewers

2010/10/5 Dean Rasheed <dean(dot)a(dot)rasheed(at)gmail(dot)com>:
> On 4 October 2010 18:22, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
>> On Mon, Oct 4, 2010 at 2:58 AM, Dean Rasheed <dean(dot)a(dot)rasheed(at)gmail(dot)com> wrote:
>>> That requires a new sort for each row. I generated this with a minor
>>> tweak to Pavel's patch to just restart the tuplesort each time (the
>>> "quick-fix" solution). The problem is that performance really sucks,
>>> because it is an O(n^2 log(n)) algorithm.
>>
>> Maybe that's OK.  If you're doing repeated median operations on large
>> data sets, perhaps you should expect that to be slow.  I bet that
>> people who want to use this as a window function will want one median
>> per group, not n medians per group; and it doesn't seem like a good
>> idea to say - we're not going to let you use this as a window function
>> AT ALL because you might decide to do something that will be really
>> slow.  You can always hit ^C if you get tired of waiting.  This seems
>> like it's very far from being the most important thing for us to
>> optimize, though of course it's great if we can.
>>
>
> Well FWIW, here's the quick-fix solution to make this median function
> support window queries.

Is this safe? It seems to me that the tuplesort_end isn't called in
window aggregates at the last of a partition, which results in
forgetting to close temp file if tuplesort is in state of tape.

Regards,

--
Hitoshi Harada

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Markus Wanner 2010-10-05 06:57:09 Re: standby registration (was: is sync rep stalled?)
Previous Message Pavel Stehule 2010-10-05 05:12:38 Re: Basic JSON support

Browse pgsql-rrreviewers by date

  From Date Subject
Next Message Dean Rasheed 2010-10-05 08:47:52 Re: wip: functions median and percentile
Previous Message Robert Haas 2010-10-05 01:49:06 Re: [HACKERS] top-level DML under CTEs