Moving delta data faster

From: yudhi s <learnerdatabase99(at)gmail(dot)com>
To: pgsql-general <pgsql-general(at)lists(dot)postgresql(dot)org>
Subject: Moving delta data faster
Date: 2024-04-03 20:38:36
Message-ID: CAEzWdqcYGi0U5_cK1FVykx2-OZHmEUD8EZ_VE=kpoVaZKYWJeg@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

Hi All,
It's postgresql database version 15.4. We have a requirement in which we
will be initially moving full table data for 3-4 tables, from source
database to target(i.e. postgres) . Maximum number of rows will be
~10million rows in those tables. Then subsequently these rows will be
inserted/updated based on the delta number of rows that got
inserted/updated in the source database. In some cases these changed data
can flow multiple times per day to the downstream i.e. postgres database
and in other cases once daily.

Want to understand , if we should use upsert(insert on conflict) or merge
statements or anything else in such a scenario so as to persist those delta
records faster in the target database, while making the system online to
the users?

Regards
Yudhi

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Thomas Munro 2024-04-03 20:56:12 Re: could not open file "global/pg_filenode.map": Operation not permitted
Previous Message sud 2024-04-03 20:09:32 Re: Timestamp conversion Error in dynamic sql script