Upgrading cluster with thousands of DBs

From: Alejandro Martínez <zen(at)itram(dot)es>
To: pgsql-admin(at)postgresql(dot)org
Subject: Upgrading cluster with thousands of DBs
Date: 2014-01-14 10:45:33
Message-ID: CAOx6PCJe-+=Y70fk7c5LcJ5aUqxpnXHtHpP34Sg8hS7nmMV6YQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-admin

Hi list,

We're working on upgrading from 9.1 to 9.3 some of our database
clusters, while seeking to minimize (write) downtime by making the
upgrades as fast as possible.

However, our clusters have the particularity that they get to host
many different *databases*, as in, ~2000 databases per cluster. Most
of those databases are small (only a table and PostGIS functions), so
it's more a problem of the number of DBs than the space they use. The
total size of the cluster is around 60gb.

I first started using pg_upgrade but, even when enabling
parallelization and hardlinks, as most of the steps (preliminary
check?) don't seem to be parallelized, the upgrade takes ~5 hours as
it goes database-per-database serially on most steps.

I'm experimenting with reducing the time by spawning parallel pg_dump
and pg_restore instances, so far I've reached a 3:15h mark that I
still think can reduce a bit more with some tweaking.

Does anyone have any tips on reducing the time of upgrading such a
kind of a cluster?

Thanks!

Responses

Browse pgsql-admin by date

  From Date Subject
Next Message Gabriele Bartolini 2014-01-14 10:52:54 Re: Upgrading cluster with thousands of DBs
Previous Message Andrew O. Zhukov 2014-01-11 11:46:54 Not fatal error