Backfill bgworker Extension?

From: Jeremy Finzel <finzelj(at)gmail(dot)com>
To: PostgreSQL mailing lists <pgsql-hackers(at)postgresql(dot)org>
Subject: Backfill bgworker Extension?
Date: 2017-12-12 18:03:33
Message-ID: CAMa1XUg3yHHum1D4Mik-mgbsVWiH62f6ghY2vHjOyB6nLrhDOA@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

One of our challenges we have is that our engineers have written frameworks
to backfill data in several different DSLs, and every time they adopt a new
language, they maybe need to write another one.

To be clear, what I mean is batch updating a large set of data in small
pieces so as to avoid things like lock contention and replication lags.
Sometimes these have a driving table that has the source data to update in
a destination table based on a key column, but sometimes it is something
like setting just a single specific value for a huge table.

I would love instead to have a Postgres extension that uses postgres
background workers to accomplish this, especially if it were part of core.
Before I venture into exploring writing something like this as an
extension, would this ever be considered something appropriate as an
extension in Postgres core? Would that be appropriate?

Thanks,
Jeremy

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Andres Freund 2017-12-12 18:21:53 Re: money type's overflow handling is woefully incomplete
Previous Message Oleg Ivanov 2017-12-12 17:26:36 Re: Learned Index