Re: Memory management with C aggregate

Tom Lane wrote:
> Ian Burrell <imb(at)rentrak(dot)com> writes:
>
>>We have user-defined aggregates written in C running inside the server.
>> We are running into some memory management issues and wondering what
>>is the best way to solve the problem.
>
>
>>The state of the aggregates is a structure with a pointer to allocated
>>memory. The structure and memory are allocated in the
>>PortalMemoryContext and freed in the ffunc. We just discovered that the
>>ffunc function can be called multiple times with the same state on
>>certain queries. The double free causes a crash.
>
>
> AFAICS the ffunc should be called only once per aggregated group.
> Perhaps your code is confused about the handling of groups? If so,
> the double free is hardly your only worry --- you'll be computing wrong
> answers anyway till you fix that.
>

The aggregate is in a subquery which might make a difference. The ffunc
is only called multiple times when a nested loop is used in the query
plan. With other query plans, the ffunc is only called once. The ffunc
seems to be called once for every combination.

I can't get the following query to use a nested loop, but our query is
similar, but with a compound key and a custom aggregate. If x has N
rows, y has M, with N groups, then the ffunc can be called N*M times, M
times for each N group.

SELECT x.key_no, s.agg
FROM x
INNER JOIN (
SELECT y.key_no, custom_agg(num) AS agg
FROM y
GROUP BY key_no
) s
USING (key_no)
ORDER BY key_no

I'll try to come up with a test case that illustrates the problem.

- Ian