Re: Multimaster

From: Moreno Andreo <moreno(dot)andreo(at)evolu-s(dot)it>
To: pgsql-general(at)postgresql(dot)org
Subject: Re: Multimaster
Date: 2016-04-14 09:11:59
Message-ID: 570F5EDF.3080604@evolu-s.it
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

<html>
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">Il 14/04/2016 07:45, Craig Ringer ha
scritto:<br>
</div>
<blockquote
cite="mid:CAMsr+YHA-ijT0Cq1UTy=YNXyEf7WcfzHAXQXxdOCqwu4M=uD+g(at)mail(dot)gmail(dot)com"
type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">On 2 April 2016 at 02:15, Moreno
Andreo <span dir="ltr">&lt;<a moz-do-not-send="true"
href="mailto:moreno(dot)andreo(at)evolu-s(dot)it" target="_blank">moreno(dot)andreo(at)evolu-s(dot)it</a>&gt;</span>
wrote:<br>
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
Actually we have to improve what our replicator is doing:
it's only replicating the single user's database. The
improvement should that we can put it on the "server" (in
some cases there are groups of users sharing a dedicated
server) and, given a configuration of what and how to
replicate, it should replicate more than one DB a time.<br>
</blockquote>
<div><br>
</div>
<div>That's a layer on top as far as pglogical is concerned.
It's only interested in a single database at a time.</div>
<div><br>
</div>
<div>The same is true of BDR.</div>
</div>
</div>
</div>
</blockquote>
I'll try to explain.<br>
We have a "cloud server" which contains all users DBs and to which
all users connect to get replication and.<br>
Some users have more than one work site, i.e. in one they're alone
(and the PostgreSQL server is on their own work PC) and in another
they're teamed up and they have a dedicated server (and everyone can
write on teammates' databases).<br>
In the first case it's easy: the only database is replicated with
the corresponding one on the cloud server. Straightforward.<br>
At the current state, in the team case, replication occurs only when
the user connects to application, so if there's a user going to a
particular site say, only once a week, in the remaining days all
data input by team mates won't be replicated, and the user won't be
aware on what's been done until he replicates again...<br>
So we are trying to have a process, running on the server, that
checks for any database to be replicated and performs the action on
all of these.<br>
<br>
Case 1<br>
Local                         Remote<br>
[User1:DB1]    --&gt;    [DB1]<br>
<br>
Case 2<br>
[User1:DB1]    --&gt;    [DB1]<br>
[User2:DB2]    --&gt;    [DB2]<br>
...<br>
[UserN:DBn]   --&gt;    [DBn]<br>
<br>
<blockquote
cite="mid:CAMsr+YHA-ijT0Cq1UTy=YNXyEf7WcfzHAXQXxdOCqwu4M=uD+g(at)mail(dot)gmail(dot)com"
type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div><br>
</div>
<div>A tool that automatically configures newly found
databases to be replicated should be pretty trivial to
write, though.</div>
</div>
</div>
</div>
</blockquote>
All databases that have to be replicated have a flag in the app
configuration, and the replication config is set up, for each
database and each site, when the first synchronization is made, so
we could spawn a process per database and replicate.<br>
Not to mention we should be able, if necessary, to exclude one or
more tables from the replication.<br>
<blockquote
cite="mid:CAMsr+YHA-ijT0Cq1UTy=YNXyEf7WcfzHAXQXxdOCqwu4M=uD+g(at)mail(dot)gmail(dot)com"
type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
We were beginning to "translate" (and then improve) this
program in c#, when I bumped into articles pointing to
BDR, and I started taking a look. But it seems that is
good to replicahe whole servers, and still hasn't the
granularity we need.</blockquote>
<div><br>
</div>
<div>Huh?</div>
<div><br>
</div>
<div>BDR is configured database-by-database. The only
exception is with bdr_init_copy, for initial setup using
binary base backups; in that case all databases are
copied.</div>
</div>
</div>
</div>
</blockquote>
Hmmm... it seems I misunderstood something......<br>
<blockquote
cite="mid:CAMsr+YHA-ijT0Cq1UTy=YNXyEf7WcfzHAXQXxdOCqwu4M=uD+g(at)mail(dot)gmail(dot)com"
type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div><br>
</div>
<div>It sounds like you actually *want* to replicate all
databases at once. Presumably the reason you're not just
using physical streaming replication for that  is that
different hosts have a different set of writeable
databases? E.g.</div>
<div><br>
</div>
<div>[Node A]   [Node B]</div>
<div>[DB-1]   -&gt;  [DB-1]</div>
<div>[DB-2]   -&gt;  [DB-2]</div>
<div>[DB-3]   &lt;-  [DB-3]</div>
<div>[DB-4]   &lt;-  [DB-4]</div>
<div><br>
</div>
<div>so each DB is written from only one node at a time, but
both nodes have writeable DBs. Right?</div>
</div>
</div>
</div>
</blockquote>
Yes, I tried to explain this in the first part of the message.<br>
<br>
Thanks<br>
Moreno<br>
<blockquote
cite="mid:CAMsr+YHA-ijT0Cq1UTy=YNXyEf7WcfzHAXQXxdOCqwu4M=uD+g(at)mail(dot)gmail(dot)com"
type="cite">
<div dir="ltr">
<div class="gmail_extra"><br clear="all">
<div><br>
</div>
-- <br>
<div class="gmail_signature">
<div dir="ltr">
<div>
<div dir="ltr"> Craig Ringer                   <a
moz-do-not-send="true"
href="http://www.2ndQuadrant.com/" target="_blank"><a class="moz-txt-link-freetext" href="http://www.2ndQuadrant.com/">http://www.2ndQuadrant.com/</a></a><br>
 PostgreSQL Development, 24x7 Support, Training &amp;
Services<br>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<br>
</body>
</html>

Attachment Content-Type Size
unknown_filename text/html 6.9 KB

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message konstantin knizhnik 2016-04-14 09:14:40 Re: Multimaster
Previous Message Thomas Kellerer 2016-04-14 06:37:18 Re: Why is the comparison between timestamp and date so much slower then between two dates