Re: [HACKERS] out of memory with large queries

At 17:14 9/06/99 -0400, you wrote:
>Massimo Dal Zotto <dz(at)cs(dot)unitn(dot)it> writes:
>> This means that postgres is currently unable to execute large queries
>> that involve functions on text fields. A pretty bad limitation IMHO.
>
>Yup. This is one of the major projects that I was proposing for 6.6.
>I do not think the fix will be easy, but we need to do it.
>

In yet another attempt to go over my head, wouldn't it be possible to pass three parameters to the aggregate state functions, so their signatures become:

int (or void) = transfn1 (transtype1, basetype, *transtype1)
int (or void) = transfn2 (transtype2, *transtype2)
int (or void) = finalfn (transtype1, transtype2, *finaltype)

where the int return value *may* give status information, or may just be ignored.

This clearly breaks the ability to use the existing float8pl etc functions, but the cost is reasonably controlled: the creation of xxxxx3 wrappers (eg. float8pl3) which take three parameters and call the original function, then release the memory.

A better solution would be to have float8pl accept 2 or 3 parameters, and modify how it returns information based on the parameter count, but I *presume* that this would break on many platforms/compilers (is this right?), in any case it seems pretty nasty returning an int OR a float8*, depending on your parameters. Perhaps if the functions were modified as:

transtype1 = transfn1 (transtype1, basetype [, *transtype1])
transtype2 = transfn2 (transtype2 [, *transtype2])
finaltype = finalfn (transtype1, transtype2 [, *finaltype])

Then they are completely source-compatible with existing functions, but will accept an optional pointer to the storage for their return value (probably a local variable in the caller).

If it worked, then each of the aggregates could avoid memory allocation completely, which has to be a performance gain as well. Unfortunately, even if this worked on all platforms, I have no idea what it would do to the internals of PG.

Am I missing something (again?).

----------------------------------------------------------------
Philip Warner | __---_____
Albatross Consulting Pty. Ltd. |----/ - \
(A.C.N. 008 659 498) | /(@) ______---_
Tel: +61-03-5367 7422 | _________ \
Fax: +61-03-5367 7430 | ___________ |
Http://www.rhyme.com.au | / \|
| --________--
PGP key available upon request, | /
and from pgp5.ai.mit.edu:11371 |/