From: | Joe Conway <mail(at)joeconway(dot)com> |
---|---|
To: | Bruce Momjian <bruce(at)momjian(dot)us>, Jeff Janes <jeff(dot)janes(at)gmail(dot)com> |
Cc: | John R Pierce <pierce(at)hogranch(dot)com>, "pgsql-general(at)postgresql(dot)org" <pgsql-general(at)postgresql(dot)org> |
Subject: | Re: Uber migrated from Postgres to MySQL |
Date: | 2016-07-28 23:58:16 |
Message-ID: | da5b1b7b-9b41-895b-4561-adf77c85c4ce@joeconway.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-general |
On 07/28/2016 03:16 PM, Bruce Momjian wrote:
> On Thu, Jul 28, 2016 at 12:35:23AM -0700, Jeff Janes wrote:
>> On Wed, Jul 27, 2016 at 9:48 PM, John R Pierce <pierce(at)hogranch(dot)com> wrote:
>>> On 7/27/2016 9:39 PM, Jeff Janes wrote:
>>>>
>>>> That depends on how how many objects there are consuming that 1 TB.
>>>> With millions of small objects, you will have problems. Not as many
>>>> in 9.5 as there were in 9.1, but still it does not scale linearly in
>>>> the number of objects. If you only have thousands of objects, then as
>>>> far as I know -k works like a charm.
>>>
>>>
>>> millions of tables?
>>
>> Well, it was a problem at much smaller values, until we fixed many of
>> them. But the perversity is, if you are stuck on a version before the
>> fixes, the problems prevent you from getting to a version on which it
>> is not a problem any more.
>
> Uh, that is only true if the slowness was in _dumping_ many objects.
> Most of the fixes have been for _restoring_ many objects, and that is
> done in the new cluster, so they should be OK.
Not really true. I ran into two separate cases where on older (pre 9.3 I
believe) Postgres if you had hundreds of thousands of tables (in the
case I remember well, it was about 500k tables) the schema dump from the
old cluster basically never finished (ok, was killed after about a
week). I had to find the patch that fixed a good bit of the slowness and
backport it to the older version so we could successfully run pg_upgrade
(in something like 14 hours instead of 7+ days).
Joe
--
Crunchy Data - http://crunchydata.com
PostgreSQL Support for Secure Enterprises
Consulting, Training, & Open Source Development
From | Date | Subject | |
---|---|---|---|
Next Message | Joshua D. Drake | 2016-07-29 00:23:47 | Re: Uber migrated from Postgres to MySQL |
Previous Message | Bruce Momjian | 2016-07-28 22:53:58 | Re: Uber migrated from Postgres to MySQL |