Re: pg_standby stuck on a wal file size <16MB

as Greg pointed out: my use of rsync --remove-sent-files option had
contributed to a short sized wal log file on standby.
changing master's postgres crontab to the following helped to resolve
the issue:

# ship logs to standby:
*/2 * * * * rsync -aq /wal_archive_local/ 10.10.10.12::wal_archive/
# remove files older then remove_check file mtime
*/5 * * * * find /wal_archive_local/ ! -newer
/wal_archive_local/remove_check -exec rm -f {} \; && touch
/wal_archive_local/remove_check

Thank you!
V.

Greg Smith wrote:
> On Sat, 17 May 2008, Ioannis Tambouras wrote:
>
>> The archive command tests if the wal segment exists and is a file,
>> but it does not check if the file is still being written.
>
> That's because it doesn't have to; the archive command doesn't get
> called until the writing is done.
>
>> I don't have sources of pg_standby near me, but I remember in the
>> C code checks for complete segment sizes.
>
> That's on the receiving side, to make sure it's not trying to process
> files that haven't finished copying to the standby yet. You don't
> have to do any of that in the archive_command.
>
> Anyway, back to the original question:
>
>> archive_command = 'test ! -f /usr/local/wal_archive_local/%f && cp %p
>> /usr/local/wal_archive_local/%f'
>> archive files are then moved on master to standby every other minute:
>> rsync -aq --remove-sent-files /usr/local/wal_archive_local/
>> slave::wal_archive/
>
> I don't see any mechanism here to keep rsync from copying over partial
> files to the standby before they've finished copying to the
> wal_archive_local directory. That's my guess for where the small
> files are coming from, rsync before the cp is done. If you're going
> to buffer in a transfer directory, you need some sort of test or
> locking to make sure the file is complete with exactly 16MB before it
> gets rsync'd over. I suspect no amount of poking at the standby will
> root out the issue because it's happening on the primary.
>
> --
> * Greg Smith gsmith(at)gregsmith(dot)com http://www.gregsmith.com Baltimore, MD
>

--
________________________________________
Vladimir (Vlad) Kosilov
Senior Systems Administrator
Contigo Systems Inc.
604.683.3106 (phone)
604.648.9886 (fax)
vkosilov(at)contigo(dot)com
www.contigo.com
________________________________________