Re: Weird failure with latches in curculio on v15

On Wed, Feb 08, 2023 at 02:25:54PM -0800, Nathan Bossart wrote:
> These are all good points. Perhaps there could be a base archiver
> implementation that shell_archive uses (and that other modules could use if
> desired, which might be important for backward compatibility with the
> existing callbacks). But if you want to do something fancier than
> archiving sequentially, you could write your own.

Which is basically the kind of things you can already achieve with a
background worker and a module of your own?

> In any case, I'm not really wedded to any particular approach at the
> moment, so I am likewise open to better ideas.

I am not sure how much we should try to move from core into the
modules when it comes to the current archiver process, with how much
control you'd like to give to users. It also looks like to me that
this is the kind of problem where we would not have the correct
callback design until someone comes in and develops a solution that
would shape around it. On top of that, this is the kind of things
achievable with just a bgworker, and perhaps simpler as the parallel
state can just be maintained in it, which is what the archiver process
is about at the end? Or were there some restrictions in the bgworker
APIs that would not fit with what an archiver process should do?
Perhaps these restrictions, if any, are what we'd better try to lift
first?
--
Michael