Re: database split

From: Dave Owens <dave(at)teamunify(dot)com>
To: Jim Nasby <Jim(dot)Nasby(at)bluetreble(dot)com>
Cc: pgsql-general(at)postgresql(dot)org
Subject: Re: database split
Date: 2015-04-30 19:08:40
Message-ID: CA+OQrzjU0KDq8MRGbaftujVwGfb8L=yByPgq9bu1DYy++bB=jA@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

>
> All that said, my guess is you're doing this to support horizontal
> scale-out, which means you'll probably need to do this more than once, and
> it'd presumably be nice for you and your customers if this didn't require
> downtime. I would look at having a way to create a partial replica using
> londiste (or BDR if it'd support it). The trick there is having a way to
> identify whether you want each individual row on a replica. If you add some
> kind of cluster_id field to every table that makes doing that filtering
> pretty easy; IIRC londiste supports that out of the box.
>

Jim, thank you for your input. The reason for the split is partially for
horizontal scale-out (current system is well-provisioned for the near
future), partly for localization concerns arising from legacy code. We are
under a bit of a deadline to finish this split, so selective replication is
not feasible at this time. The tools you mention do look very promising
for our needs, however.

I decided to do a combination of A and C...
- load a schema-only dump from the original db to the new db (filtering out
triggers and constraints)
- do COPY TO/FROMs (I would have to generate all the WHEREs for Jim's
option D anyway, the data I am after is only tiny fraction of the original
db...)
- restore the filtered constraints/triggers

Dave Owens

541-359-2602
TU Corporate Website
<http://t.signauxun.com/link?url=http%3A%2F%2Fwww.teamunify.com%2F__corp__%2F&ukey=agxzfnNpZ25hbHNjcnhyGAsSC1VzZXJQcm9maWxlGICAgOCP-IMLDA&k=b64a8344-b9e5-4e6c-ab3f-782da4a277a6>
| Swimming is Our Passion
<http://info.teamunify.com/swimming-is-our-passion-whats-your-story>
TU Facebook
<http://t.signauxun.com/link?url=https%3A%2F%2Fwww.facebook.com%2Fteamunify&ukey=agxzfnNpZ25hbHNjcnhyGAsSC1VzZXJQcm9maWxlGICAgOCP-IMLDA&k=832bda8d-6593-4bad-fa2a-31c8623aa381>
| Free OnDeck Mobile Apps
<http://t.signauxun.com/link?url=http%3A%2F%2Fwww.teamunify.com%2F__corp__%2Fondeck%2F&ukey=agxzfnNpZ25hbHNjcnhyGAsSC1VzZXJQcm9maWxlGICAgOCP-IMLDA&k=3316db21-b178-4e7b-82d8-f6b37b655825>
| TeamUnify Blog
<http://t.signauxun.com/link?url=http%3A%2F%2Fblog.teamunify.com%2F&ukey=agxzfnNpZ25hbHNjcnhyGAsSC1VzZXJQcm9maWxlGICAgOCP-IMLDA&k=3ca8e429-69ba-43df-ef87-c59e9ca2f19f>

In response to

Browse pgsql-general by date

  From Date Subject
Next Message Charlton Galvarino 2015-04-30 20:49:00 Re: psql 8 warm standby strong start, weak finish
Previous Message Adrian Klaver 2015-04-30 19:01:11 Re: pgbench - prevent client from aborting on ERROR