Re: parallelizing the archiver

From: Magnus Hagander <magnus(at)hagander(dot)net>
To: "Bossart, Nathan" <bossartn(at)amazon(dot)com>
Cc: Stephen Frost <sfrost(at)snowman(dot)net>, Andrey Borodin <x4mmm(at)yandex-team(dot)ru>, Robert Haas <robertmhaas(at)gmail(dot)com>, Julien Rouhaud <rjuju123(at)gmail(dot)com>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: parallelizing the archiver
Date: 2021-10-06 20:33:18
Message-ID: CABUevEx+7JD+gOBdemi_vjCVbH1GtTuAtHy40qx6K1STSWJXpw@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Tue, Oct 5, 2021 at 5:32 AM Bossart, Nathan <bossartn(at)amazon(dot)com> wrote:

> On 10/4/21, 8:19 PM, "Stephen Frost" <sfrost(at)snowman(dot)net> wrote:
> > It's also been discussed, at least around the water cooler (as it were
> > in pandemic times- aka our internal slack channels..) that the existing
> > archive command might be reimplemented as an extension using these. Not
> > sure if that's really necessary but it was a thought. In any case,
> > thanks for working on this!
>
> Interesting. I like the idea of having one code path for everything
> instead of branching for the hook and non-hook paths. Thanks for
> sharing your thoughts.
>

I remember having had this discussion a few times, I think mainly with
Stephen and David as well (but not on their internal slack channels :P).

I definitely think that's the way to go. It gives a single path for
everything which makes it simpler in the most critical parts. And once you
have picked an implementation other than it, you're now completely rid of
the old implementation. And of course the good old idea that having an
extension already using the API is a good way to show that the API is in a
good place.

As much as I dislike our current interface in archive_command, and would
like to see it go away completely, I do believe we need to ship something
that has it - if nothing else then for backwards compatibility. But an
extension like this would also make it easier to eventually, down the road,
deprecate this solution.

Oh, and please put said implementation in a better place than contrib :)

--
Magnus Hagander
Me: https://www.hagander.net/ <http://www.hagander.net/>
Work: https://www.redpill-linpro.com/ <http://www.redpill-linpro.com/>

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Robert Haas 2021-10-06 20:35:22 Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)
Previous Message Robert Haas 2021-10-06 20:27:41 Re: Role Self-Administration