From: | Adrian Klaver <adrian(dot)klaver(at)aklaver(dot)com> |
---|---|
To: | yudhi s <learnerdatabase99(at)gmail(dot)com>, pgsql-general <pgsql-general(at)lists(dot)postgresql(dot)org> |
Subject: | Re: Moving delta data faster |
Date: | 2024-04-03 21:11:37 |
Message-ID: | a9dd7961-dd3e-406e-a1d8-0feb233fa5e9@aklaver.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-general |
On 4/3/24 13:38, yudhi s wrote:
> Hi All,
> It's postgresql database version 15.4. We have a requirement in which
> we will be initially moving full table data for 3-4 tables, from source
> database to target(i.e. postgres) . Maximum number of rows will be
> ~10million rows in those tables. Then subsequently these rows will be
> inserted/updated based on the delta number of rows that got
> inserted/updated in the source database. In some cases these changed
> data can flow multiple times per day to the downstream i.e. postgres
> database and in other cases once daily.
What is the source database?
Can it be reached with a FDW?:
https://wiki.postgresql.org/wiki/Foreign_data_wrappers
Can the delta on the source be output as CSV?
>
> Want to understand , if we should use upsert(insert on conflict) or
> merge statements or anything else in such a scenario so as to persist
> those delta records faster in the target database, while making the
> system online to the users?
>
> Regards
> Yudhi
--
Adrian Klaver
adrian(dot)klaver(at)aklaver(dot)com
From | Date | Subject | |
---|---|---|---|
Next Message | Thomas Munro | 2024-04-03 21:14:33 | Re: could not open file "global/pg_filenode.map": Operation not permitted |
Previous Message | Thomas Munro | 2024-04-03 20:56:12 | Re: could not open file "global/pg_filenode.map": Operation not permitted |