Re: rsync and streaming replication

Hi,

I tried many times with different options of rsync :
with rsync -a as explained in the wiki (
http://www.postgresql.org/docs/9.0/interactive/warm-standby.html#STANDBY-SERVER-SETUP)
it takes 1h40 for each distant slave and quite all data files are
transferred
with -c (checksum) or -z (compress) , it takes more time, probably due to
checksum time or compress/uncompress time

For example, the rsync commands I am using are for data and logs are (I
have 2 databases, 1 tablespace per database):
rsync -a /var/opt/hosting/db/slony/pg_xlog/*
slave-db01.profiles.bench.pns-si.s1.p.fti.net:
/var/opt/hosting/db/slony/pg_xlog/
rsync -a /var/opt/hosting/db/profiles/bench/*
slave-db01.profiles.bench.pns-si.s1.p.fti.net:
/var/opt/hosting/db/profiles/bench/
rsync -a /var/opt/hosting/db/profiles/profiles/*
slave-db01.profiles.bench.pns-si.s1.p.fti.net:
/var/opt/hosting/db/profiles/profiles/

More information about version of rsync :
jaluce(at)master-db01:~$ rsync --version
rsync version 3.0.7 protocol version 30

Jal

2011/11/14 Cédric Villemain <cedric(dot)villemain(dot)debian(at)gmail(dot)com>

> 2011/11/13 Jean-Armel Luce <jaluce06(at)gmail(dot)com>:
> > Hi Jerry and Kevin,
> >
> > Thanks for your answers.
> >
> > Jerry, I tried as you said with the parameter recovery_target_timeline =
> > 'latest' and it works.
> >
> > I tried on a smaller test database (only 15MB) with PG9.1.1 and only 1
> > slave.
> >
> > My switchover procedure was :
> >
> > Step 1 : stop the old master
> > /usr/local/pgsql/bin/pg_ctl stop -m immediate -D
> > /usr/local/pgsql91/server1/data
> >
> > Step 2 : promote slave as master :
> > touch /usr/local/pgsql91/server2/data/trigger_file
> >
> > Step 3 : declare the old master as a standby server
> > Step 3.1 : vi /usr/local/pgsql91/server1/data/postgresql.conf
> > Add hot_standby = on in the postgresql.conf
> >
> > Step 3.2 Set recovery.conf for old master server (including
> > recovery_target_timeline = 'latest')
> > cp /usr/local/pgsql91/server1/data/recovery.bkp
> > /usr/local/pgsql91/server1/data/recovery.conf
> >
> > Step 4 : start old master
> > /usr/local/pgsql/bin/pg_ctl start -D /usr/local/pgsql91/server1/data &
> >
> >
> > The old master is now a hot_standby of the new master. Replication works
> > without rsyncing all data from new master to new slave.
> >
> >
> > Tomorrow, I shall try with PG9.0.3, 3 slaves and a primary database with
> 100
> > GB.
> >
> > Thanks.
> >
> > Jal
>
> just for the value : rsync --checksum is the option to use to prevent
> copying of identical files (it computes checksum on both side before
> sending)
>
>
> --
> Cédric Villemain +33 (0)6 20 30 22 52
> http://2ndQuadrant.fr/
> PostgreSQL: Support 24x7 - Développement, Expertise et Formation
>