Re: Performance for initial copy when using pg_logical to upgrade Postgres

From: Tim <timfosho(at)gmail(dot)com>
To: "Westwood, Giles" <g(dot)westwood(at)orcid(dot)org>
Cc: pgsql-performance(at)lists(dot)postgresql(dot)org
Subject: Re: Performance for initial copy when using pg_logical to upgrade Postgres
Date: 2021-09-24 16:02:31
Message-ID: CAKhLO5jd=n3WBGoRU9inqZgQxPN6gsg_crLwq5SgJ0Ky45_8cw@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-performance

I'm currently doing this with a 2.2 TB database.

Best way IMO is to (UPDATE pg_index SET indisready = false ... ) for non PK
indexes for the largest tables. Then just set it back to indisready = true
after its done and run a REINDEX CONCURRENTLY on the indexes that were
disabled.

Got about a transfer speed of 100GB per 50 minutes with this method with
consistent results.

On Fri, Sep 24, 2021 at 11:49 AM Westwood, Giles <g(dot)westwood(at)orcid(dot)org>
wrote:

>
>
>
>
> On Fri, Sep 24, 2021 at 3:39 PM Justin Pryzby <pryzby(at)telsasoft(dot)com>
> wrote:
>
>> On Fri, Sep 24, 2021 at 03:28:50PM +0100, Westwood, Giles wrote:
>>
>> Did you see this thread and its suggestions to 1) set bulk load
>> parameters;
>> and, 2) drop indexes and FKs ?
>>
>>
>> https://www.postgresql.org/message-id/flat/4A8EFC4E-A264-457D-A8E7-AE324ED9ABD4(at)thebuild(dot)com
>>
>>
> I did actually but I wanted to avoid getting my hands dirty with anything
> schema wise. I've found another person with another similar situation:-
>
> https://github.com/2ndQuadrant/pglogical/issues/325
>
>

In response to

Responses

Browse pgsql-performance by date

  From Date Subject
Next Message Andrew Dunstan 2021-09-24 20:56:30 Re: Big Performance drop of Exceptions in UDFs between V11.2 and 13.4
Previous Message Andrew Dunstan 2021-09-24 16:00:42 Re: Performance for initial copy when using pg_logical to upgrade Postgres