Re: [E] Re: parallel aggregation

On Thu, 13 Apr 2023 at 14:31, Alexander Saydakov <saydakov(at)yahooinc(dot)com> wrote:
> 1. I wonder if there is a clean separation between the phases: once partial aggregation is done only combining is done after that (state transition function never receives results of combining).

Currently the transfn won't be called again on a state that has had
the combinefn called on it. I can't think of a future reason that we
might change that. My imagination might be lacking, however.

> 2. Can a combiner output also go through serial-deserial before hitting another combiner or finalizer?

Not currently. However, I *can* think of reasons why that might change
in the future. If we wanted to accept partially aggregated results
from foreign servers and then combine them locally then, if those
foreign servers did parallel aggregation, the foreign server's
combined states would need to be serialised before sending over the
network. It feels like just a matter of time before we grow the
ability to do that. Lots of work has been done on foreign data
wrappers in the past few years. It feels like it has tailed off a bit,
but I wouldn't be surprised if we had the ability to do that in the
next few years.

David