| From: | Peter Wilson <petew(at)yellowhawk(dot)co(dot)uk> | 
|---|---|
| To: | pgsql-general(at)postgresql(dot)org | 
| Subject: | DBmirror replication - replacement for DBMirror.pl | 
| Date: | 2005-05-01 10:17:04 | 
| Message-ID: | 4274ACA0.20605@yellowhawk.co.uk | 
| Views: | Whole Thread | Raw Message | Download mbox | Resend email | 
| Thread: | |
| Lists: | pgsql-general | 
Not sure whether this is any use to anyone, or whether this is the right 
list to post to but...
I've just released a C++ implementation of the DBMirror.pl script as 
part of Whitebeam (http://www.whitebeam.org) We had *real* performance 
issues with the Perl implementation replicating large fields. It's an 
almost drop in replacement - except configuration file format is 
difffernt. It's also more fault tolerant than the Perl implementation 
and has some parameters to help distribute the replication load over 
time (if you delete 10,000 rows from a table, you don't really want 
you're main application to grind to a halt as replication soak up most 
of your resource!)
I needed to do this quickly - so it links to the utility classes in 
Whitebeam, things like a string class etc. It wouldn't be too difficult 
to decouple these and if there is any interest I'll do that when I get a 
little spare time. Once it's built though it's entirely generic and 
doesn't use anything else from Whitebeam.
If anyone has contact information for the original DBMirror author then 
I'd like to get in touch.
Would like to get feedback from anyone using DBmirror or thinks this 
would be useful..
Background
========
Our Whitebeam application server uses PostgreSQL for back-end data 
storage. We originally used IBM DB2, but ported to Postgres about 3 
years ago, but we never sorted out replication (partly because we were 
using Large Objects). I recently sorted that out and looked around for a 
replication scheme to use - first choice being Slony-I. I couldn't get 
it to work - the configuration scheme just kept claiming half our schema 
tables didn't have primary keys (they did!). Also the documentation was 
virtually non-existent.
Dropping back to DBmirror (which comes with the PostgreSQL distribution 
in the 'contrib' directory) worked fine-ish. Unfortunately the 
replication Perl script was un-usably slow, taking minutes to replicate 
a 100K BYTEA field, which we used to store images. The replication Perl 
script seemed to be rather inefficient, using a lot of regular 
expressions to decode field values etc. Perl isn't something I felt too 
confident in - and I needed a solution quickly and hence the C++ 
implementation
Pete
--
Peter Wilson
YellowHawk : http://www.yellowhawk.co.uk
Whitebeam : http:/www.whitebeam.org
-----
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Tom Lane | 2005-05-01 17:03:55 | Re: [GENERAL] Increasing statistics results in worse estimates | 
| Previous Message | A. Mous | 2005-05-01 06:36:46 | Re: VB ODBC connection to Postgresql, DAO,ADO,RDO,DSN??? |