Re: Trouble setting up replication

On 09/05/2015 11:00 AM, Chuck Martin wrote:
> Thanks for responding Adrian.
>
> After writing this, I noticed that the list is configure to reply to sender. I hope this is not inappropriate. Please tell me if I should re-send changing the address to the list.

The best way to do it is to use Reply All. This sends to both the list
and the user. Ensures the user gets the email even if the list is
running slow. While we are list etiquette, the custom on the list is to
not top post.

I am CCing the list.

>
> I apologize for leaving out important information, but plead ignorance. <g> I’m not a complete newbie with PG generally, but this is my first attempt at setting up replication.
>
> I am pretty sure that the directory was created by pg_basebackup when I tried to create the backup but the attempt failed due to an error. I did this quite a few times, getting many different errors.
>
> I have continued to study and experiment, and think that I caused most of my problems by not paying attention to the user I was logged in as when doing things. I found that even if I specify the user postgres when executing pg_basebackup, if I am logged in as root, the backup directory it creates, and its contents, are owned by root.

Yes the user specified by -U is the database user, not the system user.
The system user being whoever you are logged in as. Other then init
scripts you should not be doing anything with Postgres as root.

>
> So, I deleted the whole directory, changed to the postgres user, and then did:
>
> pg_basebackup -D /var/lib/pgsql/9.3/backups/rep -P -v -X s
>
> This produced errors but seemed to work despite them. This is what pg_basebackup reported:
>
> could not change directory to "/root": Permission denied

That was from being logged in as root.

> WARNING: could not read symbolic link "pg_tblspc/pg_hba.conf": Invalid argument

pg_tblspc contain symlinks to tablespaces. pg_hba.conf is a
configuration file, it should not be there. Looks like pg_basebackup
ignored it, but it would worth checking out what else is in pg_tblspc in
the database cluster you are backing up from.

> transaction log start point: 103/2E000028 on timeline 1
> pg_basebackup: starting background WAL receiver
> 260715040/260715040 kB (100%), 1/1 tablespace
> transaction log end point: 103/2E0009B0
> pg_basebackup: waiting for background process to finish streaming ...
> pg_basebackup: base backup completed
>
> I think the first line is because I logged in as root, changed to the postgres user, but didn’t change my working directory to the postgres directory. This doesn’t seem like anything to worry about.
> The second one didn’t stop it from working, but I like to fix things that are wrongly configured.
>
> Am I right in thinking I can rely on this backup despite the warning? And any thoughts on how to fix the warning?

See comments above.

>
> Could I have run pg_basebackup on the replicant/slave over ssh? I thought I could, and when I tried, the results suggest it is possible, but my configuration of one or the other server was incorrect. Can you see what I had wrong from this:
>
> pg_basebackup -h [main_server_url] -D /var/lib/pgsql/9.3/backups/rep -P -v -X s
> pg_basebackup: could not connect to server: FATAL: no pg_hba.conf entry for replication connection from host “[my replicant server’s outside IP]", user "root", SSL on
> FATAL: no pg_hba.conf entry for replication connection from host "[my replicant server’s outside IP]", user "root", SSL off
>
> I had added to my pg_hba.conf
>
> host replication rep 64.207.10.121/32 cert

From the above the only user that can use replication connecting from
64.207.10.121/32 is rep. You did not specify a -U in your connection
above and ran the command as root so pg_basebackup used that as the
user, which is the default behavior:

http://www.postgresql.org/docs/9.4/interactive/libpq-connect.html#LIBPQ-PARAMKEYWORDS
"
user

PostgreSQL user name to connect as. Defaults to be the same as the
operating system name of the user running the application.
"

There is no pg_hba entry for database replication and user root so the
connection was rejected. To repeat, get out of the habit of running
Postgres commands as root, it is not necessary. What matters is the
Postgres user you are connecting as. When using replication, which is
what pg_basebackup is doing, you need to connect as a user with
sufficient privileges:

http://www.postgresql.org/docs/9.4/interactive/app-pgbasebackup.html

".. The connection must be made with a superuser or a user having
REPLICATION permissions (see Section 20.2), and pg_hba.conf must
explicitly permit the replication connection. .."

Whatever user you choose to do this with then needs to authorized in
pg_hba.conf.

>
> as I have an ssh tunnel set up between the servers using certificates to authenticate. I also tried with md5 instead of cert, but both produced the error quoted above.

>
> Any suggestions?
>
> I very much appreciate you looking at this.
>
> Chuck
>
>
>> On Sep 5, 2015, at 12:38 PM, Adrian Klaver <adrian(dot)klaver(at)aklaver(dot)com> wrote:
>>
>> On 09/04/2015 04:07 PM, clmartin(at)theombudsman(dot)com wrote:
>>> I have been trying to set up "hot standby" replication, but have run
>>> into problems, and hope someone here can help.
>>>
>>> I have configured an ssh tunnel from a computer that is not on a public
>>> IP, and was unable to figure out how to run pg_basebackup from that
>>> server to backup the data from the main server, so I decided to run
>>> pb_basebackup on the main server and copy it later. So I did this:
>>>
>>> pg_basebackup -D /var/lib/pgsql/9.3/backups/rep/init -P -v -X s
>>>
>>> (after a number of other attempts), and got errors. First I got a
>>> permissions error, and found that the directory was owned by root
>>> instead of postgres. I changed that, but eventually I got this error:
>>>
>>> pg_basebackup: directory "/var/lib/pgsql/9.3/backups/rep" exists but is
>>> not empty
>>
>> So what is in it?
>>
>>>
>>> So my question is whether I can safely delete the directory and run
>>> pb_basebackup again?
>>
>> Was that directory present before you ran pg_basebackup or did pg_basebackup create it?
>>
>>>
>>> I appreciate any help you can give. Let me know what additional
>>> information is needed to answer the question.
>>>
>>> Chuck
>>>
>>>
>>
>>
>> --
>> Adrian Klaver
>> adrian(dot)klaver(at)aklaver(dot)com
>>
>>
>> --
>> Sent via pgsql-general mailing list (pgsql-general(at)postgresql(dot)org)
>> To make changes to your subscription:
>> http://www.postgresql.org/mailpref/pgsql-general
>

--
Adrian Klaver
adrian(dot)klaver(at)aklaver(dot)com