Re: I'd like to discuss scaleout at PGCon

On Sat, Jun 2, 2018 at 4:05 AM, Simon Riggs <simon(at)2ndquadrant(dot)com> wrote:
> On 1 June 2018 at 16:56, Ashutosh Bapat <ashutosh(dot)bapat(at)enterprisedb(dot)com> wrote:
>> On Fri, Jun 1, 2018 at 11:10 AM, Simon Riggs <simon(at)2ndquadrant(dot)com> wrote:
>>>
>>> Using a central coordinator also allows multi-node transaction
>>> control, global deadlock detection etc..
>>
>> But that becomes an SPOF and then we have to configure a standby for
>> that. I am not saying that that's a bad design but it's not very good
>> for many work-loads. But it would be good if we could avoid any
>> "central server" in this configuration.
>>
>>>
>>> And that is why both XL and "FDW approach" rely on a central coordinator.
>>
>> I don't think we ever specified that "FDW approach" "relies" on a
>> central coordinator. One could configure and setup a cluster with
>> multiple coordinators using FDWs.
>
> Yes, of course. You're just misunderstanding me. I'm talking about a
> query coordinator "role". There can be many coordinator components and
> they can be spread out in variours ways, but for any one SQL query
> there needs to be one coordinator node. Not a SPOF.

In your earlier mail, which is included above, you mentioned central
coordinator for multi-node transaction control and global deadlock
detection. That doesn't sound like a "query coordinator role". It
sounds more like GTM, which is an SPOF. Anyway I am happy to clarify
that "FDW approach" relies on a query coordinator, the server which
faces the client. But I don't think we have decided how would the
transaction management and deadlock detection work in the shared
nothing cluster of PostgreSQL servers. There was discussion in
developer unconference this year, but I was not part of that as I was
holding another session the same time. May be somebody who attended
that session can post a summary here or provide a link to the summary
written elsewhere.

>
>>>
>>> FDWs alone are not enough. It is clear that some more tight coupling
>>> is required to get things to work well. For example, supporting SQL
>>> query plans that allow for redistribution of data for joins.
>>
>> I think partitioning + FDW provide basic infrastructure for
>> distributing data, planning queries working with such data. We need
>> more glue to support node management, cluster configuration. So, I
>> agree with your statement. But I think it was clear from the beginning
>> that we need more than FDW and partitioning.
>
> No, it wasn't clear. But I'm glad to hear it. It might actually work then.

Good to see some agreement.

--
Best Wishes,
Ashutosh Bapat
EnterpriseDB Corporation
The Postgres Database Company