Re: Asynchronous execution support for Custom Scan

Le mardi 6 septembre 2022, 11:29:55 CET Etsuro Fujita a écrit :
> On Mon, Sep 5, 2022 at 10:32 PM Kazutaka Onishi <onishi(at)heterodb(dot)com> wrote:
> > I'm sorry for my error on your name...
>
> No problem.
>
> > > IIUC, it uses the proposed
> > >
> > > APIs, but actually executes ctidscans *synchronously*, so it does not
> > > improve performance. Right?
> >
> > Exactly.
> > The actual CustomScan that supports asynchronous execution will
> > start processing in CustomScanAsyncRequest,
> > configure to detect completion via file descriptor in
> > CustomScanAsyncConfigureWait,
> > and receive the result in CustomScanAsyncNotify.
>
> Ok, thanks!

Thanks for this patch, seems like a useful addition to the CustomScan API.
Just to nitpick: there are extraneous tabs in createplan.c on a blank line.

Sorry for the digression, but I know your ctidscan module had been proposed
for inclusion in contrib a long time ago, and I wonder if the rationale for
not including it could have changed. We still don't have tests which cover
CustomScan, and I can think of at least a few use cases where this customscan
is helpful and not merely testing code.

One of those use case is when performing migrations on a table, and one wants
to update the whole table by filling a new column with a computed value. You
obviously don't want to do it in a single transaction, so you end up batching
updates using an index looking for null values. If you want to do this, it's
much faster to update rows in a range of block, performing a first series of
batch updating all such block ranges, and then finally update the ones we
missed transactionally (inserted in a block we already processed while in the
middle of the batch, or in new blocks resulting from a relation extension).

Best regards,

--
Ronan Dunklau