Re: Migrate postgres to newer hardware

I've never worked with Bucardo and Londiste, so I cannot give you any
hint about it.
With slony it's not necessary to restore all database prior staring sync
process. You only have to restore your schema but not data. Slony will
keep them synced with source server.

Anyway, if you plan to migrate to 8.4 there are some issues with slony.
For 8.3 and higher, you will go with slony 2.X. For 8.2 with slony 1.X
I'm not sure if Slony 1.x is compatible with Postgres >= 8.3, but I'm
sure slony 2.x is not backwards compatible.

So, prior thinking about migrating to 8.4 you will have to check this.
And more important, it's possible you have to slightly change your
database schema to work in postgres 8.4, specially if you use tsearch2.

If using slony, I wouldn't upgrade to 8.4 before a deep analysis of
changes in your existing database.

-----Original Message-----
From: Renato Oliveira <renato(dot)oliveira(at)grant(dot)co(dot)uk>
To: Iñigo Martinez Lasala <imartinez(at)vectorsf(dot)com>
Cc: pgsql-admin <pgsql-admin(at)postgresql(dot)org>
Subject: RE: [ADMIN] Migrate postgres to newer hardware
Date: Wed, 31 Mar 2010 09:11:37 +0100


Hi Iñigo,

Thank you for your input, really appreciated.

I just had a thought; if I backup ‘pg_dump’ full database, then restore
to my new machine new postgres 8.4, which one of these programs would
work best to do the migration,

Slony, Bucardo or Londiste?

I would like to say that we did have slony and I was not impressed, it
fell behind and could not catch up and caused a very high load on the
system.

Also they way its philosophy works, it is very high maintenance, and the
idea of creating tables, and triggers on both dbs...

It might be, it was not setup properly or well, we removed it as the db
was screaming for some fresh air.

Ps I would like to point out that I am systems administrator and not a
dba, so you can understand sometimes my questions...

I think Bucardo seems best for the task, for what I have read so far,
but I do not know.

Thank you very much and I am sorry for this

Renato

Renato Oliveira
Systems Administrator
e-mail: renato(dot)oliveira(at)grant(dot)co(dot)uk

Tel: +44 (0)1763 260811
Fax: +44 (0)1763 262410
www.grant.co.uk

Grant Instruments (Cambridge) Ltd

Company registered in England, registration number 658133

Registered office address:
29 Station Road,
Shepreth,
CAMBS SG8 6GB
UK

From: Iñigo Martinez Lasala [mailto:imartinez(at)vectorsf(dot)com]
Sent: 30 March 2010 16:27
To: Renato Oliveira
Cc: pgsql-admin
Subject: RE: [ADMIN] Migrate postgres to newer hardware

Yes, you only have that two possibilities, I think.

PITR is not an option. I tested the same, from 7.4 32bit to 7.4 64bit
and didn't work. Later, when I asked here, I was told why not.

The problem with slony is that you have to manually create tables in
destination database and all database model (procedures, triggers,
sequences, views, etc). If your application creates new tables, you will
have to deal with this prior starting migration, or at least disable the
creation of new tables.

Slony is asynchronous, so you will have to ensure that all changes have
been committed to new database before changing your applications or
exchanging IP addresses.
Slony also add many triggers and special tables to both databases
(master and slave). So, after migration, you will have to delete them.
It's not difficult but don't forget to do it.

By the way, are you sure your database is 160GB? Including indexes?
There are strategies in order to perform a faster pg_restore...
For example, if you migrate your database schema but don't create
indexes, then migrate data and finally create pending indexes restore
will be faster. With pg 8.4 restore is very fast, so it will take less
time that export.

Anyway, if you cannot leave database down for a day, I think slony will
be your best bet, although it's not exempt of problems. :)

-----Original Message-----
From: Renato Oliveira <renato(dot)oliveira(at)grant(dot)co(dot)uk>
To: Iñigo Martinez Lasala <imartinez(at)vectorsf(dot)com>
Subject: RE: [ADMIN] Migrate postgres to newer hardware
Date: Tue, 30 Mar 2010 15:47:27 +0100

 Hey Iñigo,

Thank you very much for your reply.

I would love to do just that, but unfortunately I can’t it is not as
simple as that.

I would love if the application had been built in with this in mind…

To give you an idea; the pg_dump takes 15 hours and I attempted a
restore yesterday and it took 14 hours and 21 min.

It would not be viable for us and specially I cannot have the system
down more than maximum 30 min without the risk of losing data and
customers not having alerts.

I don’t think I will be able to use PITR to migrate to new servers
specially if it is 64 bit and to migrate to another 32 bit is no gain,
as we need more memory.

As far as can gather there are only two ways:

a) Slony type

b) Pg_dump

Is that correct ? Do you guys have any other ways?

Renato

Renato Oliveira
Systems Administrator
e-mail: renato(dot)oliveira(at)grant(dot)co(dot)uk Tel: +44 (0)1763 260811
Fax: +44 (0)1763 262410
www.grant.co.uk Grant Instruments (Cambridge) Ltd

Company registered in England, registration number 658133

Registered office address:
29 Station Road,
Shepreth,
CAMBS SG8 6GB
UK
From: Iñigo Martinez Lasala [mailto:imartinez(at)vectorsf(dot)com]
Sent:30 March 2010 15:29
To:Renato Oliveira
Cc:pgsql-admin
Subject: Re: [ADMIN] Migrate postgres to newer hardware

Hi Renato.

I would follow the ancient method: perform a pg_dump / pg_restore

Yes, you will have to take offline database for a long period.

And yes, it would be a great moment to perform a 8.4 upgrade.
Performance is far superior, restore is far faster...
... and yes, it could give you many problems if you don't perform many
test in order to address all queries without explicit type conversions
before real migration, but I think it's the best moment to deal with a
very convenient upgrade.

We have performed this upgrade last week with a gforge (with only 25GB
database) and having also to upgrade to new tsearch2 and everything is
running smooth.

-----Original Message-----
From: Renato Oliveira <renato(dot)oliveira(at)grant(dot)co(dot)uk>
To: pgsql-admin(at)postgresql(dot)org <pgsql-admin(at)postgresql(dot)org>
Subject: [ADMIN] Migrate postgres to newer hardware
Date: Tue, 30 Mar 2010 12:18:36 +0100

Dear All,

What would be the easiest and fastest way to migrate Postgres 8.2.24 32
BIT to a new server 64 Bit.

The existing server runs on 32 bit architecture and has a database as
big as 160GB.

We initially thought of using PITR, but as one of the PITR requirements
is both machines need to be similar.

This similarity needs to be even in architecture? I think I read
something which says “Yes”.

If we cannot use PITR what would be the best approach, we can’t have
down time I am afraid.

Any ideas or suggestions would be very welcome.

Thank you very much

Best regards

Renato

Renato Oliveira
Systems Administrator
e-mail: renato(dot)oliveira(at)grant(dot)co(dot)uk Tel: +44 (0)1763 260811
Fax: +44 (0)1763 262410
www.grant.co.uk Grant Instruments (Cambridge) Ltd

Company registered in England, registration number 658133

Registered office address:
29 Station Road,
Shepreth,
CAMBS SG8 6GB
UK

P Please consider the environment before printing this email

CONFIDENTIALITY: The information in this e-mail and any attachments is
confidential. It is intended only for the named recipients(s). If you
are not the named recipient please notify the sender immediately and do
not disclose the contents to another person or take copies.
VIRUSES: The contents of this e-mail or attachment(s) may contain
viruses which could damage your own computer system. Whilst Grant
Instruments (Cambridge) Ltd has taken every reasonable precaution to
minimise this risk, we cannot accept liability for any damage which you
sustain as a result of software viruses. You should therefore carry out
your own virus checks before opening the attachment(s).
OpenXML: For information about the OpenXML file format in use within
Grant Instruments please visit our website

P Please consider the environment before printing this email

CONFIDENTIALITY: The information in this e-mail and any attachments is
confidential. It is intended only for the named recipients(s). If you
are not the named recipient please notify the sender immediately and do
not disclose the contents to another person or take copies.
VIRUSES: The contents of this e-mail or attachment(s) may contain
viruses which could damage your own computer system. Whilst Grant
Instruments (Cambridge) Ltd has taken every reasonable precaution to
minimise this risk, we cannot accept liability for any damage which you
sustain as a result of software viruses. You should therefore carry out
your own virus checks before opening the attachment(s).
OpenXML: For information about the OpenXML file format in use within
Grant Instruments please visit our website

P Please consider the environment before printing this email

CONFIDENTIALITY: The information in this e-mail and any attachments is
confidential. It is intended only for the named recipients(s). If you
are not the named recipient please notify the sender immediately and do
not disclose the contents to another person or take copies.

VIRUSES: The contents of this e-mail or attachment(s) may contain
viruses which could damage your own computer system. Whilst Grant
Instruments (Cambridge) Ltd has taken every reasonable precaution to
minimise this risk, we cannot accept liability for any damage which you
sustain as a result of software viruses. You should therefore carry out
your own virus checks before opening the attachment(s).

OpenXML: For information about the OpenXML file format in use within
Grant Instruments please visit our website