Re: replication timeout in pg_basebackup

From: "Aggarwal, Ajay" <aaggarwal(at)verizon(dot)com>
To: Haribabu Kommi <kommi(dot)haribabu(at)gmail(dot)com>
Cc: "pgsql-general(at)postgresql(dot)org" <pgsql-general(at)postgresql(dot)org>
Subject: Re: replication timeout in pg_basebackup
Date: 2014-03-11 01:46:46
Message-ID: 3B7431C850F4F347885C4CE5DD7B401993A93D8C@MIA20725MBX891A.apps.tmrk.corp
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

I have already tried experimenting with linux dirty_ratio etc. You can only fine tune up to a limit. The backup process still fills up the buffer cache very quickly. Yes, my database is about 5-6 GB in size and will grow bigger over time.

If wish there was a way to slow down pg_basebackup or force it to use direct I/O.
________________________________________
From: Haribabu Kommi [kommi(dot)haribabu(at)gmail(dot)com]
Sent: Monday, March 10, 2014 8:31 PM
To: Aggarwal, Ajay
Cc: pgsql-general(at)postgresql(dot)org
Subject: Re: [GENERAL] replication timeout in pg_basebackup

On Tue, Mar 11, 2014 at 7:07 AM, Aggarwal, Ajay <aaggarwal(at)verizon(dot)com> wrote:
> Thanks Hari Babu.
>
> I think what is happening is that my dirty cache builds up quickly for the
> volume where I am backing up. This would trigger flush of these dirty pages
> to the disk. While this flush is going on pg_basebackup tries to do fsync()
> on a received WAL file and gets blocked.

But the sync is executed for every WAL file finish. Does your database
is big in size?
Does your setup is write-heavy operations?

In Linux when it tries to write a bunch of buffers at once, the fysnc
call might block for some time.
In the following link there are some "Tuning Recommendations for
write-heavy operations" which might be useful to you.

http://www.westnet.com/~gsmith/content/linux-pdflush.htm

Any other ideas to handle these kind of problems?

Regards,
Hari Babu
Fujitsu Australia

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Aggarwal, Ajay 2014-03-11 01:51:03 Re: replication timeout in pg_basebackup
Previous Message Anand Kumar, Karthik 2014-03-11 01:04:25 Increase in max_connections