Re: Thousands of schemas and ANALYZE goes out of memory

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: "Hugo <Nabble>" <hugo(dot)tech(at)gmail(dot)com>
Cc: pgsql-general(at)postgresql(dot)org
Subject: Re: Thousands of schemas and ANALYZE goes out of memory
Date: 2012-10-04 18:14:06
Message-ID: 923.1349374446@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

"Hugo <Nabble>" <hugo(dot)tech(at)gmail(dot)com> writes:
> We use Debian linux 32-bits, so the addressable space available really seems
> to be in the 3.0Gb to 3.5Gb range. Last night I decreased the shared_buffers
> from 2Gb to 1Gb and tried the global ANALYZE again. It went out of memory
> after 3 hours 40 minutes. That database has 12,197 schemas with 22 tables
> each, which means 268,334 tables.

It sure sounds to me like you're using the schema for what ought to be a
high-order key column in a much smaller number of tables ... having that
many tables is simply never going to perform well. We can play
whack-a-mole on individual bottlenecks all day, but that's never going
to fix the fundamental problem. Some of the bottlenecks will be at
kernel or filesystem level and thus out of our reach, anyway.

regards, tom lane

In response to

Browse pgsql-general by date

  From Date Subject
Next Message Jeff Janes 2012-10-04 19:56:47 Re: Thousands of schemas and ANALYZE goes out of memory
Previous Message Hugo <Nabble> 2012-10-04 17:54:44 Re: Thousands of schemas and ANALYZE goes out of memory