Re: Having trouble configuring a Master with multiple standby Servers in PostgreSQL 9.3.3

From: <fburgess(at)radiantblue(dot)com>
To: "Michael Paquier" <michael(dot)paquier(at)gmail(dot)com>
Cc: pgsql-bugs(at)postgresql(dot)org
Subject: Re: Having trouble configuring a Master with multiple standby Servers in PostgreSQL 9.3.3
Date: 2014-04-17 16:19:17
Message-ID: 20140417091917.5a830134ae84016b0174832fdc1a3173.8e79f63c53.wbe@email11.secureserver.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-bugs

<html><body><span style="font-family:Verdana; color:#000000; font-size:10pt;"><div>Hi Michael, thanks for your reply.</div><div><br></div><div>I discussed this my colleague, and we decided to change the archive_command to execute a shell script.</div><div><br></div><div>#!/bin/bash<br># archive_command script to replicate archivelogs to standby server slaves<br># <br># postgresql.conf parameter<br>#<br># archive_command = '&lt;$PGDATA&gt;/<a href="http://replica_achive_set.sh">replica_achive_set.sh</a> "%p" "%f"'<br>#<br>set -e<br>set -u<br>ARCHIVE1="/mnt/server/slave1_archivedir"<br>ARCHIVE2="/mnt/server/slave2_archivedir"<br>if [ -f ${ARCHIVE1}/$2 ] &amp;&amp; [ -f ${ARCHIVE2}/$2 ] ; then<br>&nbsp; echo Archive file $2 already exists in one of the replicated sets archive, skipping &gt;&amp;2<br>&nbsp; exit 0<br>fi<br>echoerr() { echo "$@" 1&gt;&amp;2; }<br>FAIL=0<br>`/usr/bin/rsync -aq $1 ${ARCHIVE1}/$2` &amp; pid_1=$! ; `/usr/bin/rsync -aq $1 ${ARCHIVE2}/$2` &amp; pid_2=$!<br>echoerr "Spawned replication processes $pid_1 AND $pid_2"<br>wait $pid_1 || let "FAIL+=1"<br>wait $pid_2 || let "FAIL+=1"<br>if [ "$FAIL" == "0" ];<br>then<br>echoerr "Replication success $1 $2"<br>else<br>echoerr "Replication failed $1 $2"<br>fi<br></div><div><br></div><div>This will copy the archivelogs from the master to both slaves. <b>Will that avoid the issue with removing needed WAL files?</b></div><div><br></div><div>I should be able to use these recovery.conf files</div><div><br></div><div>slave #1</div><div><br></div><div><span style="font-family:Verdana; color:#000000; font-size:10pt;">standby_mode = 'on'<br>primary_conninfo
= 'host=&lt;master database ip address&gt; port=5432 dbname=tumsdb
user=replication password=&lt;password&gt; application_name=slave1
sslmode=require'<br>restore_command = 'cp </span><span style="font-family:Verdana; color:#000000; font-size:10pt;">/mnt/server/slave1_archivedir/%f "%p%"'&nbsp;&nbsp;&nbsp;&nbsp; <br>archive_cleanup_command = 'pg_archivecleanup </span><span style="font-family:Verdana; color:#000000; font-size:10pt;">/mnt/server/slave1_archivedir/ %r'<br>trigger_file= '/opt/PostgreSQL/9.3/data/pgsql.trigger.file'</span></div><div><br></div><div><br></div><div>slaves #2</div><div><br></div><div><span style="font-family:Verdana; color:#000000; font-size:10pt;">standby_mode = 'on'<br>primary_conninfo
= 'host=&lt;master database ip address&gt; port=5432 dbname=tumsdb
user=replication password=&lt;password&gt; application_name=slave2
sslmode=require'<br>restore_command = 'cp </span><span style="font-family:Verdana; color:#000000; font-size:10pt;">/mnt/server/slave2_archivedir/%f "%p%"'&nbsp;&nbsp;&nbsp;&nbsp; <b></b><br>archive_cleanup_command = 'pg_archivecleanup </span><span style="font-family:Verdana; color:#000000; font-size:10pt;">/mnt/server/slave2_archivedir/ %r'<br>trigger_file= '/opt/PostgreSQL/9.3/data/pgsql.trigger.file'</span></div><div><span style="font-family:Verdana; color:#000000; font-size:10pt;"><br></span></div><div><b>Does this look correct?</b></div><div><br></div><div>Finally, question about the backup.</div><div><br></div><div>I did a pg_clt reload to change the archivelog destination from /mnt/server/master_archivedir to be redistributed to slave1 and slave2.<b> Do I need to redo this backup step? </b></div><div><br></div><div>psql -c "select pg_start_backup('initial_backup');"<br>rsync -cvar --inplace --exclude=*pg_xlog* /u01/fiber/postgreSQL_data/postgres(at)1(dot)2(dot)3(dot)5:/u01/fiber/postgreSQL_data/<br>psql -c " select pg_stop_backup ();"<br></div><div><span style="font-family:Verdana; color:#000000; font-size:10pt;"><br></span></div><div><span style="font-family:Verdana; color:#000000; font-size:10pt;">or can I just copy all of the missing archivelog files from the /mnt/server/master_archivedir to the slaves, and then restart the slaves in recovery mode?</span></div><div><span style="font-family:Verdana; color:#000000; font-size:10pt;"><br></span></div><div><span style="font-family:Verdana; color:#000000; font-size:10pt;">thanks</span></div><div><br></div>
<blockquote id="replyBlockquote" webmail="1" style="border-left: 2px solid blue; margin-left: 8px; padding-left: 8px; font-size:10pt; color:black; font-family:verdana;">
<div id="wmQuoteWrapper">
-------- Original Message --------<br>
Subject: Re: [BUGS] Having trouble configuring a Master with multiple<br>
standby Servers in PostgreSQL 9.3.3<br>
From: Michael Paquier &lt;<a href="mailto:michael(dot)paquier(at)gmail(dot)com">michael(dot)paquier(at)gmail(dot)com</a>&gt;<br>
Date: Wed, April 16, 2014 6:07 pm<br>
To: <a href="mailto:fburgess(at)radiantblue(dot)com">fburgess(at)radiantblue(dot)com</a><br>
Cc: <a href="mailto:pgsql-bugs(at)postgresql(dot)org">pgsql-bugs(at)postgresql(dot)org</a><br>
<br>
<div dir="ltr"><div>TODO<br><br>On Thu, Apr 17, 2014 at 1:29 AM, &nbsp;&lt;<a target="_blank" href="mailto:fburgess(at)radiantblue(dot)com">fburgess(at)radiantblue(dot)com</a>&gt; wrote:<br>&gt; Now the issue is with the recovery.conf file on slave1, should the<br> &gt; restore_command point to the archivelogs on the master?<br>Yes, this is where archive_command of master copies the WAL files. You need them for recovery operations on slaves.<br><br>&gt; Do I run the archive_cleanup_command when I recover slave1 or do I wait<br> &gt; until I have finished backup/copy from the slave2<br>Be careful here, this command may remove WAL files that are needed by other slaves. For example, if slave1 kicks this command, you may remove files still needed by slave2 that has not yet done any recovery operation and it may need them.<br> <br>&gt; postgresql.conf - Slave1<br>&gt; restore_command = 'cp /mnt/server/master_archivedir/%f "%p%"' &nbsp; &nbsp; &nbsp;&lt;--- ****<br>&gt; Is this correct! **** The master remains on-line and is producing archive<br> &gt; logs.<br>No need to have that much complexity for %p:<br>restore_command <span class=""></span>= 'cp -i /mnt/server/master_archivedir/%f %p'<br><br>&gt; postgresql.conf - Slave2 Server VM<br>&gt; restore_command = 'cp /mnt/server/slave2_archivedir/%f "%p%"' &nbsp; &nbsp; &nbsp;&lt;--- ****<br> &gt; Is this correct! **** The master remains on-line and is producing archive<br>&gt; logs.<br></div>Please see above, it could be more simple.<br><div>-- <br>Michael</div></div>
</div>
</blockquote></span></body></html>

Attachment Content-Type Size
unknown_filename text/html 6.1 KB

Responses

Browse pgsql-bugs by date

  From Date Subject
Next Message omar.pessoa 2014-04-17 19:15:32 BUG #10060: Distinct SQL results
Previous Message Ludovic POLLET 2014-04-17 15:43:59 Re: BUG #8842: lo_open/fastpath transaction inconsistency