postgres_fdw insert extremely slow

From: Mats Julian Olsen <mats(at)duneanalytics(dot)com>
To: pgsql-general(at)lists(dot)postgresql(dot)org
Subject: postgres_fdw insert extremely slow
Date: 2020-11-25 16:37:28
Message-ID: CAARtqpE-7mZY8WKKMq2fWeHBcrAJu41_GnKgKqdvkidk7g+M3A@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

Helle pgsql-general,

We have just set up postgres_fdw between two postgres databases, x and y,
with the plan to periodically insert data from x into y.

We've successfully set up the connection with a few options:
`use_remote_estimate 'true'` and `fetch_size '50000'` (the latter we've
played around with). We've run ANALYZE on the foreign server.

SELECTs against the foreign table returns in milliseconds, however an
INSERT of 100 rows takes 10 seconds. we have roughly 200 000 rows for the
initial sync, which translates into ~6 hours.

Is this the expected performance of postgre_fdw? Is there anything we've
overlooked when setting this up? Very curious to hear experiences from the
community when doing read/write and not just read from foreign sources.

Best regards,

--
Mats
CTO @ Dune Analytics
We're hiring: https://careers.duneanalytics.com

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Adrian Klaver 2020-11-25 16:41:09 Re: postgres_fdw insert extremely slow
Previous Message Laurenz Albe 2020-11-25 16:36:37 Number of parallel workers chosen by the optimizer for parallel append