Re: design for parallel backup

On Thu, Apr 30, 2020 at 3:52 PM Andres Freund <andres(at)anarazel(dot)de> wrote:
> Why 8kb? That's smaller than what we currently do in pg_basebackup,
> afaictl, and you're actually going to be bottlenecked by syscall
> overhead at that point (unless you disable / don't have the whole intel
> security mitigation stuff).

I just picked something. Could easily try other things.

> > , divided into 1, 2, 4, 8, or 16 equal size files, with each file
> > written by a separate process, and an fsync() at the end before
> > process exit. So in this test, there is no question of whether the
> > master can read the data fast enough, nor is there any issue of
> > network bandwidth. It's purely a test of whether it's faster to have
> > one process write a big file or whether it's faster to have multiple
> > processes each write a smaller file.
>
> That's not necessarily the only question though, right? There's also the
> approach one process writing out multiple files (via buffered, not async
> IO)? E.g. one basebackup connecting to multiple backends, or just
> shuffeling multiple files through one copy stream.

Sure, but that seems like it can't scale better than this. You have
the scaling limitations of the filesystem, plus the possibility that
the process is busy doing something else when it could be writing to
any particular file.

> If you can provide me with the test program, I'd happily run it on some
> decent, but not upper end, NVMe SSDs.

It was attached, but I forgot to mention that in the body of the email.

> I think you might also be seeing some interaction with write caching on
> the raid controller here. The file sizes are small enough to fit in
> there to a significant degree for the single file tests.

Yeah, that's possible.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company