RE: Partial aggregates pushdown

From: "Fujii(dot)Yuki(at)df(dot)MitsubishiElectric(dot)co(dot)jp" <Fujii(dot)Yuki(at)df(dot)MitsubishiElectric(dot)co(dot)jp>
To: Jelte Fennema-Nio <postgres(at)jeltef(dot)nl>
Cc: Bruce Momjian <bruce(at)momjian(dot)us>, Robert Haas <robertmhaas(at)gmail(dot)com>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Stephen Frost <sfrost(at)snowman(dot)net>, Andres Freund <andres(at)anarazel(dot)de>, Tomas Vondra <tomas(dot)vondra(at)enterprisedb(dot)com>, Julien Rouhaud <rjuju123(at)gmail(dot)com>, Daniel Gustafsson <daniel(at)yesql(dot)se>, vignesh C <vignesh21(at)gmail(dot)com>, Alexander Pyhalov <a(dot)pyhalov(at)postgrespro(dot)ru>, "Fujii(dot)Yuki(at)df(dot)MitsubishiElectric(dot)co(dot)jp" <Fujii(dot)Yuki(at)df(dot)MitsubishiElectric(dot)co(dot)jp>
Subject: RE: Partial aggregates pushdown
Date: 2024-06-12 05:27:02
Message-ID: TYAPR01MB308841016A3F1AE8247BAA6495C02@TYAPR01MB3088.jpnprd01.prod.outlook.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi Jelte,

Thank you for your comment!
> From: Jelte Fennema-Nio <postgres(at)jeltef(dot)nl>
> Sent: Tuesday, June 11, 2024 11:00 PM
> How about instead of trying to serialize the output of serialfn/deserialfn, instead we don't use the "internal" type and
> create actual types in pg_type for these transtypes? Then we can simply use the in/out and recv/send functions of those
> types to serialize the values of the partial aggregate over the network.
> Instead of having to rely on serialfn/deserialfn to be network-safe (which they probably aren't).
>
> That indeed still leaves the pseudo types. Since non of those pseudotypes have a working in/recv function (they always
> error by definition), I agree that we can simply not support those.
>
> Basically that would mean that any aggregate with a non-internal and non-pseudotype as a transtype could be used in this
> multi-node partial aggregate pushdown.
Could you please clarify what you mean?
Are you referring to:
Option 1: Modifying existing aggregate functions to minimize the use of internal state values.
Option 2: Not supporting the push down of partial aggregates for functions with internal state values.
Option 3: Something other than Option 1 and Option 2.

There are many aggregate functions with internal state values, so if we go with Option 1,
we might need to change a lot of existing code, like transition functions and finalize functions.
Also, I'm not sure how many existing aggregate functions can be modified this way.

There are also many popular functions with internal state values,
like sum(int8) and avg(int8)(see [1]), so I don't think Option2 would be acceptable.

Best regards, Yuki Fujii

--
Yuki Fujii
Information Technology R&D Center, Mitsubishi Electric Corporation

[1] https://github.com/postgres/postgres/blob/REL_16_STABLE/src/include/catalog/pg_aggregate.dat

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Amit Kapila 2024-06-12 05:52:39 Re: Conflict Detection and Resolution
Previous Message Dilip Kumar 2024-06-12 05:21:29 Re: Logical Replication of sequences