Re: pg_basebackup: could not get write-ahead log end position from server: ERROR

Also you could change your pg_basebackup parameter and set it as -X stream
instead of -X fetch and you don't have to worry about increasing
wal_keep_segments in the future

El mar., 12 jun. 2018 a las 1:03, Saeed Ahmed (DB) (<
saeed(dot)ahmed1(at)telenor(dot)com(dot)pk>) escribió:

> Hi,
>
>
>
> Thanks for the support and your time.
>
>
>
> The issue was resolved after increasing wal_keep_segments to 20 means it
> will keep 20 files in directory /PGDATA/pg_wal .
>
>
>
>
>
> and ran "pg_ctl reload" to enable the configuration without rebooting the
> db.
>
>
>
>
>
> Regards,
>
>
>
> Saeed Ahmed
>
>
>
>
>
> *From:* Ahmed, Nawaz [mailto:Nawaz(at)fast(dot)au(dot)fujitsu(dot)com]
> *Sent:* Tuesday, June 12, 2018 2:44 AM
> *To:* Saeed Ahmed (DB); pgsql-admin(at)postgresql(dot)org
> *Cc:* Database Services
> *Subject:* RE: pg_basebackup: could not get write-ahead log end position
> from server: ERROR
>
>
>
> That explains it, the script deletes the WAL file as soon as it is
> archived but pg_basebackup is not aware of this so it generates that error.
>
>
>
> The second point is, you have to clean-up archives manually using
> pg_archivecleanup or something similar based on your situation as Pavan had
> noted. In order for the clean-up of unneeded WAL after a successful base
> backup should be implemented using bash (Linux / Unix) or batch/Powershell
> (Win) scripting.
>
>
>
> Best Regards,
>
>
>
>
>
>
> *Nawaz AhmedSoftware Development EngineerFujitsu Australia Software
> Technology Pty Ltd*
> 14 Rodborough Road, Frenchs Forest NSW 2086, Australia
> *T* +61 2 9452 9027
> Nawaz(at)fast(dot)au(dot)fujitsu(dot)com
> fastware.com.au
>
>
>
>
> *From:* Saeed Ahmed (DB) [mailto:saeed(dot)ahmed1(at)telenor(dot)com(dot)pk
> <saeed(dot)ahmed1(at)telenor(dot)com(dot)pk>]
> *Sent:* Saturday, 9 June 2018 9:48 PM
> *To:* Ahmed, Nawaz <Nawaz(at)fast(dot)au(dot)fujitsu(dot)com>; pgsql-admin(at)postgresql(dot)org
> *Cc:* Database Services <db(dot)is(at)telenor(dot)com(dot)pk>
> *Subject:* RE: pg_basebackup: could not get write-ahead log end position
> from server: ERROR
>
>
>
> Hi Nawaz,
>
>
>
> Many thanks for the update.
>
>
>
>
>
> 1) have you configured the archive_command to a script that archives the
> online WAL to the archive destination and deletes the file?
>
> Yes
>
> 2) As far as I know the cleanup of archive destination has to done
> manually.
>
>
>
> Ok
>
>
>
>
>
> Regards,
>
>
>
> Saeed Ahmed
>
>
>
>
>
>
>
> *From:* Ahmed, Nawaz [mailto:Nawaz(at)fast(dot)au(dot)fujitsu(dot)com
> <Nawaz(at)fast(dot)au(dot)fujitsu(dot)com>]
> *Sent:* Saturday, June 09, 2018 2:41 AM
> *To:* Saeed Ahmed (DB); pgsql-admin(at)postgresql(dot)org
> *Cc:* Database Services
> *Subject:* Re: pg_basebackup: could not get write-ahead log end position
> from server: ERROR
>
>
>
> Hi,
>
>
>
> 1) have you configured the archive_command to a script that archives the
> online WAL to the archive destination and deletes the file?
>
>
>
> 2) As far as I know the cleanup of archive destination has to done
> manually.
>
>
>
> Regards,
>
> Nawaz
>
>
> ------------------------------
>
>
> *From:* Saeed Ahmed (DB) <saeed(dot)ahmed1(at)telenor(dot)com(dot)pk>
> *Date:* 9 June 2018 at 4:27:25 am AEST
> *To:* pgsql-admin(at)postgresql(dot)org <pgsql-admin(at)postgresql(dot)org>
> *Cc:* Database Services <db(dot)is(at)telenor(dot)com(dot)pk>
> *Subject:* pg_basebackup: could not get write-ahead log end position from
> server: ERROR
>
>
>
> Dear Team,
>
>
>
> 1)Whilst taking full backup via pg_basebackup I have facing below error.
> Backup is schedule via cronjob.
>
>
>
>
> ================================================================================
>
> ****** Full Backup of posgresql database started Fri Jun 8 20:40:36 PKT
> 2018.
>
> 080618204036
>
> pg_basebackup: initiating base backup, waiting for checkpoint to complete
>
> pg_basebackup: checkpoint completed
>
> pg_basebackup: write-ahead log start point: 43/400252F8 on timeline 1
>
> 17624153/17624153 kB (100%), 2/2
> tablespaces
>
> pg_basebackup: could not get write-ahead log end position from server:
> ERROR: requested WAL segment 000000010000004300000040 has already been
> removed
>
> pg_basebackup: removing data directory
> "/db01/pgdata/hot_backup/080618204036"
>
>
>
>
>
>
>
>
>
> 2)Also after complete backup I have to manually delete the wal archive
> files using pg_archivecleanup. Is there any way to automatically delete
> after full backup is complete and wal archive are no longer required.
>
>
>
> pg_archivecleanup -d /db01/wal_archive 00000001000000390000000A
>
>
>
>
>
> Kindly let me know how to rectify.
>
>
>
> pg_basebackup -z -X fetch --progress -v --format=t -D
> /db01/pgdata/hot_backup/$DD --no-password
>
>
>
>
>
>
>
> *Current Version:*
>
>
>
>
> version
>
>
> ---------------------------------------------------------------------------------------------------------
>
> PostgreSQL 10.1 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 4.4.7
> 20120313 (Red Hat 4.4.7-18), 64-bit
>
>
>
>
>
> *LINUX : .10.0-327.el7.x86_64*
>
>
>
>
>
> Regards,
>
>
>
> Saeed Ahmed
>
>
> Disclaimer
>
> The information in this e-mail is confidential and may contain content
> that is subject to copyright and/or is commercial-in-confidence and is
> intended only for the use of the above named addressee. If you are not the
> intended recipient, you are hereby notified that dissemination, copying or
> use of the information is strictly prohibited. If you have received this
> e-mail in error, please telephone Fujitsu Australia Software Technology Pty
> Ltd on + 61 2 9452 9000 or by reply e-mail to the sender and delete the
> document and all copies thereof.
>
>
>
> Whereas Fujitsu Australia Software Technology Pty Ltd would not knowingly
> transmit a virus within an email communication, it is the receiver’s
> responsibility to scan all communication and any files attached for
> computer viruses and other defects. Fujitsu Australia Software Technology
> Pty Ltd does not accept liability for any loss or damage (whether direct,
> indirect, consequential or economic) however caused, and whether by
> negligence or otherwise, which may result directly or indirectly from this
> communication or any files attached.
>
>
>
> If you do not wish to receive commercial and/or marketing email messages
> from Fujitsu Australia Software Technology Pty Ltd, please email
> unsubscribe(at)fast(dot)au(dot)fujitsu(dot)com
> Disclaimer
>
> The information in this e-mail is confidential and may contain content
> that is subject to copyright and/or is commercial-in-confidence and is
> intended only for the use of the above named addressee. If you are not the
> intended recipient, you are hereby notified that dissemination, copying or
> use of the information is strictly prohibited. If you have received this
> e-mail in error, please telephone Fujitsu Australia Software Technology Pty
> Ltd on + 61 2 9452 9000 or by reply e-mail to the sender and delete the
> document and all copies thereof.
>
>
>
> Whereas Fujitsu Australia Software Technology Pty Ltd would not knowingly
> transmit a virus within an email communication, it is the receiver’s
> responsibility to scan all communication and any files attached for
> computer viruses and other defects. Fujitsu Australia Software Technology
> Pty Ltd does not accept liability for any loss or damage (whether direct,
> indirect, consequential or economic) however caused, and whether by
> negligence or otherwise, which may result directly or indirectly from this
> communication or any files attached.
>
>
>
> If you do not wish to receive commercial and/or marketing email messages
> from Fujitsu Australia Software Technology Pty Ltd, please email
> unsubscribe(at)fast(dot)au(dot)fujitsu(dot)com
>