Re: pg_basebackup inconsistent performance

On 5/6/20 5:44 AM, Jasen Lentz wrote:
> We have pg_basebackup running on two of our DB servers that are
> replicated.  We are running postgres 11, and it seems we are getting
> inconsistent performance from the backups and unsure as of why.  We
> start out at 5-6 hours over a dedicated 10G port for 7TB.  It creeps up
> to 8-9 hours then all of a sudden takes 12-16 hours.  There seems to be

I'm guessing the above happens from one run to another correct?

Where are the machines you are backing up from/to relative to each on
the network?

Is there increased activity on the database servers e.g. inserts,
updates, etc during the extended backups?

> no rhyme or reason for the extended backup times.  The command we use
> for backups is as follows:
>
> On server 2 (secondary), starts at 4PM
>
> pg_basebackup --pgdata=/opt/postgres/pgbackup/`echo $DATE`
> --format=plain --write-recovery-conf --no-sync --wal-method=stream
> --checkpoint=fast --label=`hostname`-`echo $DATE` --no-verify-checksums
> --host=<server1> --username=replication --port=5432
>
> On server 1 (Primary), starts at Midnight
>
> pg_basebackup --pgdata=/opt/postgres/pgbackup/`echo $DATE`
> --format=plain --write-recovery-conf --no-sync --wal-method=stream
> --checkpoint=fast --label=`hostname`-`echo $DATE` --no-verify-checksums
> --host=<server2> --username=replication --port=5432
>
> I’m not sure why or how we are running into the weeds.  I am the
> SysAdmin and am not familiar with the inner workings of the DB.  I can
> pass any commands that need run along to our DBA.
>
> From the OS perspective, we are not seeing any problems with CPU,
> memory or disk.  We are running on RHEL 7.7
>
> Thanks!
>

--
Adrian Klaver
adrian(dot)klaver(at)aklaver(dot)com