Re: parallelizing the archiver

On Fri, Sep 10, 2021 at 2:03 PM Andrey Borodin <x4mmm(at)yandex-team(dot)ru> wrote:
>
> > 10 сент. 2021 г., в 10:52, Julien Rouhaud <rjuju123(at)gmail(dot)com> написал(а):
> >
> > Yes, but it also means that it's up to every single archiving tool to
> > implement a somewhat hackish parallel version of an archive_command,
> > hoping that core won't break it.
> I'm not proposing to remove existing archive_command. Just deprecate it one-WAL-per-call form.

Which is a big API beak.

> It's a very simplistic approach. If some GUC is set - archiver will just feed ready files to stdin of archive command. What fundamental design changes we need?

I'm talking about the commands themselves. Your suggestion is to
change archive_command to be able to spawn a daemon, and it looks like
a totally different approach. I'm not saying that having a daemon
based approach to take care of archiving is a bad idea, I'm saying
that trying to fit that with the current archive_command + some new
GUC looks like a bad idea.