Question about using AggCheckCallContext in a C function

From: Matt Solnit <msolnit(at)soasta(dot)com>
To: "pgsql-general(at)postgresql(dot)org" <pgsql-general(at)postgresql(dot)org>
Subject: Question about using AggCheckCallContext in a C function
Date: 2013-08-12 17:45:04
Message-ID: 525FEA14-9548-4C3C-B88F-4201E8E9BA1E@soasta.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

Hi everyone. A while back, I wrote a function to sum array contents (think
1-D matrix addition), and am using it in a custom SUM aggregate. Here's an
example:

CREATE TABLE foo (arr INT[]);
INSERT INTO foo VALUES ('{1, 2, 3}'), ('{4, 5, 6}');

SELECT SUM(arr) FROM foo;
sum
---------
{5,7,9}
(1 row)

This works, but I got some great advice from Craig Ringer about how I can
make it even faster using AggCheckCallContext(). See http://stackoverflow.com/a/16996606/6198

I'm trying to implement this now, and I'm running into occasional memory
corruption. Sometimes the back-end segfaults, sometimes I get seemingly-
random "ERROR: could not open relation with OID xyz" errors.

Here's the relevant code snippet:

// Assumptions:
// 1. We will never be called with a null array. The CREATE FUNCTION call should specify STRICT to prevent PostgreSQL from doing this.
// 2. The arrays will never contain null values. It's up to the caller to ensure this.
// 3. The arrays will always be the same length. It's up to the caller to ensure this.

ArrayType *array1, *array2;
Datum *arrayData1, *arrayData2;
int arrayLength1;

array2 = PG_GETARG_ARRAYTYPE_P(1);

if (AggCheckCallContext(fcinfo, NULL))
{
// We are being called by an aggregate.

if (PG_ARGISNULL(0))
{
// This can happen on the very first call as PostgreSQL sets up the "temporary transition value" for the aggregate.

// NOTE: This never seems to happen! Is it because the function is STRICT?

// Return a copy of the second array.
PG_RETURN_ARRAYTYPE_P(copy_intArrayType(array2));
}
else
{
// This means that the first array is a "temporary transition value", and we can safely modify it directly with no side effects.
// This avoids the overhead of creating a new array object.

array1 = PG_GETARG_ARRAYTYPE_P(0);

// Get a direct pointer to each array's contents.
arrayData1 = ARR_DATA_PTR(array1);
arrayData2 = ARR_DATA_PTR(array2);

// Get the length of the first array (should be the same as the second).
arrayLength1 = (ARR_DIMS(array1))[0];

// Add the contents of the second array to the first.
for (i = 0; i < arrayLength1; i++)
{
arrayData1[i] += arrayData2[i];
}

// Return the updated array.
PG_RETURN_ARRAYTYPE_P(array1);
}
}
else
{
// We are not being called by an aggregate.
// <omitted>
}

After poring over the code in nodeAgg.c, and looking at the in8inc()
function, I think I know what the problem is: the typical use of
AggCheckCallContext() is not compatible with TOAST-able data types.

When the state transition function is STRICT (as mine is), and
advance_transition_function() comes across the first non-NULL value,
it makes a copy using datumCopy(). However, from what I can tell,
datumCopy() is not really "compatible" with TOAST for this
particular use case.

Am I on the right track here? If so, what's the best way to solve
this problem? I would appreciate any help you can offer.

Sincerely,
Matt Solnit <msolnit(at)soasta(dot)com>

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Tom Lane 2013-08-12 18:53:30 Re: Question about using AggCheckCallContext in a C function
Previous Message Day, David 2013-08-12 12:45:35 Re: plpgsql FOR LOOP CTE problem ?