Re: Adding pipe support to pg_dump and pg_restore

Hi,

On Sat, Sep 29, 2018 at 8:03 PM, Stephen Frost <sfrost(at)snowman(dot)net> wrote:
> Greetings,
>
> * David Hedberg (david(dot)hedberg(at)gmail(dot)com) wrote:
>> On Sat, Sep 29, 2018 at 7:01 PM, Stephen Frost <sfrost(at)snowman(dot)net> wrote:
>> > * David Hedberg (david(dot)hedberg(at)gmail(dot)com) wrote:
>> >> On Sat, Sep 29, 2018 at 5:03 PM, Stephen Frost <sfrost(at)snowman(dot)net> wrote:
>> >> Generally, my thinking is that this can be pretty useful in general
>> >> besides encryption. For other formats the dumps can already be written
>> >> to standard output and piped through for example gpg or a custom
>> >> compression application of the administrators choice, so in a sense
>> >> this functionality would merely add the same feature to the directory
>> >> format.
>> >
>> > That's certainly not the same though. One of the great advantages of
>> > custom and directory format dumps is the TOC and the ability to
>> > selectively extract data from them without having to read the entire
>> > dump file. You end up losing that if you have to pass the entire dump
>> > through something else because you're using the pipe.
>>
>> I can maybe see the problem here, but I apologize if I'm missing the point.
>>
>> Since all the files are individually passed through separate instances
>> of the pipe, they can also be individually restored. I guess the
>> --list option could be (adopted to be) used to produce a clear text
>> TOC to further use in selective decryption of the rest of the archive?
>

I admit that my understanding of the custom format was naive (I have
never actually used it).

>> If this is simply outside the scope of the directory or the custom
>> format, that is certainly understandable (and, to me, somewhat
>> regrettable :-) ).
>
> What I think isn't getting through is that while this is an interesting
> approach, it really isn't a terribly good one, regardless of how
> flexible you view it to be. The way to move this forward seems pretty
> clearly to work on adding generalized encryption support to
> pg_dump/restore that doesn't depend on calling external programs
> underneath of the directory format with a pipe.

I did get the message that it wasn't the optimal way of doing it, and
I have now also gotten the message that it's probably not really
wanted at all.

Thanks you for your insights,
David