Re: Schema version management

On Sun, May 20, 2012 at 7:36 PM, Joel Jacobson <joel(at)trustly(dot)com> wrote:
> On Mon, May 21, 2012 at 8:08 AM, Daniel Farina <daniel(at)heroku(dot)com> wrote:
>> I think you are absolutely right, but I'm not sure if teaching pg_dump
>> a new option is the best idea.  It's a pretty complex program as-is.
>> I've also heard some people who really wish pg knew how to self-dump
>> for valid reasons.
>
> Complex program? Yes, pg_dump it is extremely complex, I wouldn't want
> to touch any of the code. A rewrite is probably close to impossible.

I wouldn't be so sure about that...

> Complex patch? No. It's 102 lines of code and doesn't change any of
> the existing code in pg_dump, it simply adds some lines writing out
> the objects to separate files. Have a look at the patch, it's super
> simple.

Ah. I did not know there was a patch already out there -- I did not
somehow get that , as it then can be audited in its precise functionality.

>> It sounds like some of the catalog wrangling and cycle-breaking
>> properties of pg_dump could benefit from being exposed stand-alone,
>> but unfortunately that's not a simple task, especially if you want to
>> do The Right Thing and have pg_dump link that code, given pg_dump's
>> criticality.
>
> It's just sad realizing people need to some up with hacks and
> work-arounds to solve a obvious real-life problem, easily fixed inside
> pg_dump with 102 lines of drop-dead simple code, not touching any of
> the logics or flows in pg_dump.
>
> I can't even image how many hours coders have wasted hacking together
> tools like pg_extractor just to circumvent the stupid fact pg_dump
> can't do this natively.

My next question would be how this might relate to the directory dump
format. For example, is it an embellishment of that? It seems at
fist glance that whatever this patch might be a cousin of that
feature. Or, is it superseded? The documentation is clear that tables
are given their own files, but doesn't say much about how other schema
objects are stored, so they may or may not be useful to your needs.

Also, now that I look more carefully, there was a lot of conversation
about this patch; it seems like what you are doing now is reporting
its successful use, and I did not understand that by reading the
abstract of your email. And, beyond that, do we have a summary of the
open questions that prevented it from being committed?

> I'm sure pg_extractor does it best to achieve the objective, but even
> if it does, I would never trust it for production usage, version
> controlling your production schema is far too important to trust any
> tool not part of the mainline distribution of postgres. And personally
> I don't have any problem, I've been using the --split option for two
> years, I just feel sorry for the rest of the postgres community,
> unaware of how to solve this problem, having to hack together their
> own little tools, or be "lucky" finding some existing hack.

My thinking is that confidence would be increased if there was a piece
of code that handled a lot of the catalog munging et al that is part
of pg_dump that *is* maintained by postgres so other projects can more
convincingly add a correct veneer.

As a meta-comment, all I did was ask some polite questions. You could
have politely disqualified pg_extractor and spared some of the
language without having gotten anything less done.

--
fdr