Re: Horizontal scalability/sharding

>
>
> At PGCon we agreed to have such meeting in Vienna at least. But I think we
> should be prepared and try to clean all our issues before. It looks like we
> already out of time,but probably we could meet in Hong Kong ?
>
> Honestly, I still don't know which approach is better, we already played
> with XL (ported to 9.4) and identified some very strong issues with
> inconsistency, which scared us, especially taking into account how easy we
> found them. XC people have fixed them, but I'm not sure if they were
> fundamental and if we could construct more sophisticated tests and find
> more issues in XC/XL. We also a bit disappointed by Huawei position about
> CSN patch, we hoped to use for our XTM. FDW approach has been actively
> criticized by pg_shard people and that's also made me a bit suspicious. It
> looks like we are doomed to continue several development forks, so we
> decided to work on very important common project, XTM, which we hoped could
> be accepted by all parties and eventually committed to 9.6. Now I see we
> were right, unfortunately.
>

Distributed transaction manager should support at least three things
1. Atomic commit
2. Atomic visibility
3. Consistent snapshots (e.g. required for repeatable reads and higher
isolation levels).

I have submitted patch for implementing first for FDWs. The patch adds
infrastructure to be used by all FDWs including postgres_fdw. It also adds
postgres_fdw code to use this infrastructure. The same can be used to
achieve atomic commit in postgres_fdw based sharding. Please see if XTM can
benefit from it. If there are things that are required by XTM, please post
the requirements on that thread and I will work on those. You can find the
latest patch at
http://www.postgresql.org/message-id/CAFjFpRfANWL53+x2HdM9TCNe5pup=oPkQSSJ-KGfr-d2efj+CQ@mail.gmail.com

>
> Again, could we organize meeting somewhere in September ? US is not good
> for us, but other places should be ok. I want to have an agreement at
> least on XTM. We still are testing various approaches, though. We could
> present results of our experiments and are open to discussion. It's not
> easy project, but it's something we could do for 9.6.
>
> I'm very glad Bruce started this discussion in -hackers, since it's silly
> to me to participate in both threads :) Let's meet in September !
>
>
>
>> --
>> Simon Riggs http://www.2ndQuadrant.com/
>> <http://www.2ndquadrant.com/>
>> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
>>
>
>

--
Best Wishes,
Ashutosh Bapat
EnterpriseDB Corporation
The Postgres Database Company