Re: pg_background contrib module proposal

On Mon, Dec 12, 2016 at 10:47 PM, Andrew Borodin <borodin(at)octonica(dot)com> wrote:
> Hi!
>

Thanks a lot for your review.

> Just in case you'd like to include sleepsort as a test, here it is
> wrapped as a regression test(see attachment). But it has serious
> downside: it runs no less than 5 seconds.
>
> Also I'll list here every parallelism feature I managed to imagine. It
> is not to say that I suggest having some of these features at v1. We
> certainly have to start somewhere, this list is just for information
> purposes.
> 1. Poolable workers. Just to be sure you can reliably queue your task
> somewhere without having "increase max_connections" hint.
> 2. Inside one pool, you can have task chaining. After competition of
> task A (query A) start task B, in case of failure start task C. Task C
> is followed by task D.

I think background-session code is not that much deviated from
pg_background code, IIUC background-session patch we can manage to
reuse it, as suggested by Robert. This will allow us to maintain
session in long run, we could reuse this session to run subsequent
queries instead of maintaining separate worker pool. Thoughts?

> 3. Reliably read task status: running\awaiting\completed\errored\in a
> lock. Get a query plan of a task? (I know, there are reasons why it is
> not possible now)
+1, Let me check this.

> 4. Query as a future (actually it is implemented already by
> pg_background_result)
> 5. Promised result. Declare that you are waiting for data of specific
> format, execute a query, someone (from another backend) will
> eventually place a data to promised result and your query continues
> execution.

Could you please elaborate more?
Do you mean two way communication between foreground & background process?

> 6. Cancelation: a way to signal to background query that it's time to
> quit gracefully.
>
Let me check this too.

Thanks & Regards,
Amul