Re: Experience with large number of tables in single PostgreSQL instance

From: Marc Mamin <M(dot)Mamin(at)intershop(dot)de>
To: "'Vedran Krivokuca'" <vkrivokuca(at)gmail(dot)com>, "pgsql-admin(at)postgresql(dot)org" <pgsql-admin(at)postgresql(dot)org>
Subject: Re: Experience with large number of tables in single PostgreSQL instance
Date: 2013-04-11 08:14:09
Message-ID: B6F6FD62F2624C4C9916AC0175D56D8802ED87@jenmbs01.ad.intershop.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-admin


>
> 2) we can go with single instance of PostgreSQL service which would
> then contain 30.000 database tables.
>
> So, consider this purely theoretical discussion - does anyone here have
> experience of running PostgreSQL service with large number of database
> tables (couple of thousands up to couple of tens of thousands)?

Hello,

We are handling data for several customers, each of them having its own schema.
This allows us to easily bulk move data on other servers when place gets rare.
These are comparable to data warehouse data and are used by a reporting application.
A majority of the tables are seldom or never used which may moderate possible issues with very large pg_catalog.

We are overall impressed how well Postgres handle this :-)
On the other hand we've spent a *LOT* of work in table and query design and we do have some problems with the parser performance:

We do log all durations greater than 1 seconds. 1 % of these are "parse" statements
about 90 % of these long parse statements are below 10 seconds.

Here the figure of our server hosting the most tables.
I expect them to be somewhat above the maximum that we should allow.

DB size :3500 GB

pg_catalog size: 2GB

select count(*) from pg_tables;
120'884

select count(*) from pg_indexes;
219'082

select count(*) from pg_attribute;
2'779'199

Server: 48 GB RAM & 12 cores

regards,

Marc Mamin

In response to

Browse pgsql-admin by date

  From Date Subject
Next Message X.H.----WANG 2013-04-11 08:21:08 Re: with PostgreSQL 9.1.9,the stats collector process is not work!
Previous Message Hoàng Thanh Toàn - DB 2013-04-11 08:13:30 Master/Slave mode: Temp table used by view