Re: [HACKERS] parallel.c oblivion of worker-startup failures

On Tue, Dec 19, 2017 at 9:01 PM, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
> On Tue, Dec 19, 2017 at 5:01 AM, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
>> I think it would have been much easier to fix this problem if we would
>> have some way to differentiate whether the worker has stopped
>> gracefully or not. Do you think it makes sense to introduce such a
>> state in the background worker machinery?
>
> I think it makes a lot more sense to just throw an ERROR when the
> worker doesn't shut down cleanly, which is currently what happens in
> nearly all cases. It only fails to happen for fork() failure and
> other errors that happen very early in startup. I don't think there's
> any point in trying to make this code more complicated to cater to
> such cases. If fork() is failing, the fact that parallel query is
> erroring out rather than running with fewer workers is likely to be a
> good thing. Your principle concern in that situation is probably
> whether your attempt to log into the machine and kill some processes
> is also going to die with 'fork failure', and having PostgreSQL
> consume every available process slot is not going to make that easier.
> On the other hand, if workers are failing so early in startup that
> they never attach to the error queue, then they're probably all
> failing the same way and trying to cope with that problem in any way
> other than throwing an error is going to result in parallelism being
> silently disabled with no notification to the user, which doesn't seem
> good to me either.
>

That is not the main point I am bothered about. I am concerned that
the patch proposed by you can lead to hang if there is a crash (abrupt
failure like proc_exit(1)) after attaching to the error queue. This is
explained in my email up thread [1]. I think we can fix it by trying
to read the queues as mentioned by you, but was not sure if that is
the best way to deal with it, so proposed an alternate solution.

[1] - https://www.postgresql.org/message-id/CAA4eK1LkHBbafduA8em1fb-in4kDBgmQf6qO5TV8CfyuTzEFaQ%40mail.gmail.com

--
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com