Re: [HACKERS] Logical replication and multimaster

Well,

Given my earlier message about Apple wanting to pay me for writing
patches now, maybe I can revisit this idea.

For background: I brought up the idea of an FDW that could read custom
dump files and expose them as tables so that you could grab just a
single record (or of course more) from a custom dump file without having
to load the whole thing up, if you were stuck reaching into a backup to
get at accidentally deleted tables, rows, etc.. The stopper, which was
communicated to me by Tom at the following pgcon was that the code for
parsing custom dumps is duplicated in pg_dump only, and would need to be
duplicated into the server for the FDW, or broken out into a library.

And for posterity, Dave Page said that was a stupid idea, while Magnus
said that it sounded useful. And IIRC Bruce and Robert H said it was
doable, just a good deal of work on the refactor needed.

This convo went down at the Amsterdam conf where I spoke about using
writeable LVM snapshots to expose each developer a copy of prod to
noodle on, without having to actually make a full copy for each dev.

Added trivia: I gave the talk with a can of Heineken in my hand at the
podium, and my lightning talk had the work F(&king Cool in the title ;)

That was also when I bought my plush Slony which was supposedly the very
last one. (turns out they made more)
--
Jon Erdman (aka StuckMojo)
PostgreSQL Zealot

On 12/15/15 9:48 PM, Jim Nasby wrote:
> On 12/13/15 7:37 AM, David Fetter wrote:
>> As I understand it, pushing these into a library has been proposed but
>> not rejected.  That it hasn't happened yet is mostly about the lack of
>> tuits (the round ones) to rewrite the functionality as libraries and
>> refactor pg_dump/pg_restore to use only calls to same.  As usual, it's
>> less about writing the code and more about the enormous amount of
>> testing any such a refactor would entail.
>
> My understanding as well. IIRC Jon Erdman brought this question up a
> couple years ago and the response was "It'd probably be accepted, it's
> just that no one has done the work."
>
>> I believe that refactoring much of pg_dump's functionality for the
>> current version of the server into SQL-accessible functions and making
>> pg_dump use only those functions is achievable with available
>> resources.
>>
>> Such a refactor need not be all-or-nothing.  For example, the
>> dependency resolution stuff is a first step that appears to be worth
>> doing by itself even if the effort then pauses, possibly for some
>> time.
>
> If someone wanted to spend time on this, I suspect it'd be worth looking
> at how bad some of the backward compatibility issues would be if done in
> the server. Maybe they wouldn't be that bad. I suspect the audience for
> this code would be much larger if it was in the server as opposed to a C
> library.