Re: Patch: incorrect array offset in backend replication tar header

On Tue, Sep 25, 2012 at 4:07 AM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Brian Weaver <cmdrclueless(at)gmail(dot)com> writes:
>> While researching the way streaming replication works I was examining
>> the construction of the tar file header. By comparing documentation on
>> the tar header format from various sources I certain the following
>> patch should be applied to so the group identifier is put into thee
>> header properly.
>
> Yeah, this is definitely wrong.
>
>> While I realize that wikipedia isn't always the best source of
>> information, the header offsets seem to match the other documentation
>> I've found. The format is just easier to read on wikipedia
>
> The authoritative specification can be found in the "pax" page in the
> POSIX spec, which is available here:
> http://pubs.opengroup.org/onlinepubs/9699919799/
>
> I agree that the 117 number is bogus, and also that the magic "ustar"
> string is written incorrectly. What's more, it appears that the latter
> error has been copied from pg_dump (but the 117 seems to be just a new
> bug in pg_basebackup). I wonder what else might be wrong hereabouts :-(
> Will sit down and take a closer look.
>
> I believe what we need to do about this is:
>
> 1. fix pg_dump and pg_basebackup output to conform to spec.
>
> 2. make sure pg_restore will accept both conformant and
> previous-generation files.
>
> Am I right in believing that we don't have any code that's expected to
> read pg_basebackup output? We just feed it to "tar", no?

Yes. We generate a .tarfile, but we have no tool that reads it
ourselves. (unlike pg_dump where we have pg_restore that actually
reads it)

> I'm a bit concerned about backwards compatibility issues. It looks to
> me like existing versions of pg_restore will flat out reject files that
> have a spec-compliant "ustar\0" MAGIC field. Is it going to be
> sufficient if we fix this in minor-version updates, or are we going to
> need to have a switch that tells pg_dump to emit the incorrect old
> format? (Ick.)

Do we officially support using an older pg_restore to reload a newer
dump? I think not? As long as we don't officially support that, I
think we'll be ok.

--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/