Re: The plan for FDW-based sharding

We do not have formal prove that proposed XTM is "general enough" to
handle all possible transaction manager implementations.
But there are two general ways of dealing with isolation: snapshot based
and CSN based.
pg_dtm and pg_tsdtm prove that both of them can be implemented using XTM.
If you know some approach to distributed transaction manager
implementation, please let us know.
Otherwise your statement "is not general enough" is not concrete enough.
Postgres-XL GTM can be in principle implemented as extension based on XTM.

This API is based on existed PostgreSQL TM functions: we do not
introduce some new abstractions.
Is it possible that some other TM function has to be encapsulated? Yes,
it is.
But I do not see much problems with adding this function to XTM in
future if it is actually needed.
It happens with most APIs. It is awful when API functions are changed,
breaking application based on this API.
But as far as functions encapsulated in XTM are in any case present in
PostgreSQL core, I do not think
that them will be changed in future unless there are some plans to
completely rewrite Postgres transaction manager...

Yes, it is certainly possible to develop cluster by cloning PostgreSQL.
But it cause big problems both for developers, which have to permanently
synchronize their branch with master,
and, what is more important, for customers, which can not use standard
version of PostgreSQL.
It may cause problems with system certification, with running Postgres
in cloud,...
Actually the history of Postgres-XL/XC and Greenplum IMHO shows that it
is wrong direction.

On 26.02.2016 19:06, Robert Haas wrote:
> On Fri, Feb 26, 2016 at 7:21 PM, Oleg Bartunov <obartunov(at)gmail(dot)com> wrote:
>> Right now tm is hardcoded and it's doesn't matter "if other people might
>> need" at all. We at least provide developers ("other people") ability to
>> work on their implementations and the patch is safe and doesn't sacrifices
>> anything in core.
> I don't believe that. When we install APIs into core, we're
> committing to keep those APIs around. And I think that we're far too
> early in the development of transaction managers for PostgreSQL to
> think that we know what APIs we want to commit to over the long term.
>
>>> And what makes us think we
>>> really need multiple transaction managers, anyway?
>> If you brave enough to say that one tm-fits-all and you are able to teach
>> existed tm to play well in various clustering environment during
>> development period, which is short, than probably we don't need multiple
>> tms. But It's too perfect to believe and practical solution is to let
>> multiple groups to work on their solutions.
> Nobody's preventing multiple groups for working on their solutions.
> That's not the question. The question is why we should install hooks
> in core at this early stage without waiting to see which
> implementations prove to be best and whether those hooks are actually
> general enough to cater to everything people want to do. There is
> talk of integrating XC/XL work into PostgreSQL; it has a GTM.
> Postgres Pro has several GTMs. Maybe there will be others.
>
> Frankly, I'd like to see a GTM in core at some point because I'd like
> everybody who uses PostgreSQL to have access to a GTM. What I don't
> want is for every PostgreSQL company to develop its own GTM and
> distribute it separately from everybody else's. IIUC, MySQL kinda did
> that with storage engines and it resulted in the fragmentation of the
> community. We've had the same thing happen with replication tools -
> every PostgreSQL company develops their own set. It would have been
> better to have ONE set that was distributed by the core project so
> that we didn't all do the same work over again.
>
> I don't understand the argument that without these hooks in core,
> people can't continue to work on this. It isn't hard to work on GTM
> without any core changes at all. You just patch your copy of
> PostgreSQL. We do this all the time, for every patch. We don't add
> hooks for every patch.
>
>> dtms. It's time to start working on dtm, I believe. The fact you don't
>> think about distributed transactions support doesn't mean there no "other
>> people", who has different ideas on postgres future. That's why we propose
>> this patch, let's play the game !
> I don't like to play games with the architecture of PostgreSQL.
>

--
Konstantin Knizhnik
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company