From: | Magnus Hagander <magnus(at)hagander(dot)net> |
---|---|
To: | Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> |
Cc: | Andres Freund <andres(at)2ndquadrant(dot)com>, Fujii Masao <masao(dot)fujii(at)gmail(dot)com>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org> |
Subject: | Re: pg_basebackup failed to back up large file |
Date: | 2014-06-03 15:57:52 |
Message-ID: | CABUevExUwTadqP=ZoTnqA3gUKv6qiuPFJ4NRpa2XBevuX8oE6w@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
On Tue, Jun 3, 2014 at 5:42 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Andres Freund <andres(at)2ndquadrant(dot)com> writes:
> > On 2014-06-03 11:04:58 -0400, Tom Lane wrote:
> >> My point is that having backups crash on an overflow doesn't really seem
> >> acceptable. IMO we need to reconsider the basebackup protocol and make
> >> sure we don't *need* to put values over 4GB into this field. Where's
> the
> >> requirement coming from anyway --- surely all files in PGDATA ought to
> be
> >> 1GB max?
>
> > Fujii's example was logfiles in pg_log. But we allow to change the
> > segment size via a configure flag, so we should support that or remove
> > the ability to change the segment size...
>
> What we had better do, IMO, is fix things so that we don't have a filesize
> limit in the basebackup format. After a bit of googling, I found out that
> recent POSIX specs for tar format include "extended headers" that among
> other things support member files of unlimited size [1]. Rather than
> fooling with partial fixes, we should make the basebackup logic use an
> extended header when the file size is over INT_MAX.
>
Yeah, pax seems to be the way to go. It's at least supported by GNU tar -
is it also supported on say BSD, or other popular platforms? (The size
extension in the general ustar format seems to be, so it would be a shame
if this one is less portable)
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
From | Date | Subject | |
---|---|---|---|
Next Message | Andres Freund | 2014-06-03 16:06:18 | Re: pg_basebackup failed to back up large file |
Previous Message | Tom Lane | 2014-06-03 15:42:49 | Re: pg_basebackup failed to back up large file |