Re: PHJ file leak.

At Mon, 11 Nov 2019 17:24:45 -0500, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote in
> Thomas Munro <thomas(dot)munro(at)gmail(dot)com> writes:
> > On Tue, Nov 12, 2019 at 1:24 AM Kyotaro Horiguchi
> > <horikyota(dot)ntt(at)gmail(dot)com> wrote:
> >> Hello. While looking a patch, I found that PHJ sometimes complains for
> >> file leaks if accompanied by LIMIT.
>
> > Thanks for the patch! Yeah, this seems correct, but I'd like to think
> > about it some more before committing. I'm going to be a bit tied up
> > with travel so that might be next week.
>
> At this point we've missed the window for this week's releases, so
> there's no great hurry (and it'd be best not to push any noncritical
> patches into the back branches anyway, for the next 24 hours).
>
> Although the patch seems unobjectionable as far as it goes, I'd like
> to understand why we didn't see the need for it long since. Is there
> another call to ExecParallelHashCloseBatchAccessors somewhere, and
> if so, is that one wrongly placed?

It's a simple race conditions between leader and workers.

If a scan on workers reached to the end, no batch file is open, but a
batch file is open if it doesn't reaches to the end.

If a worker notices that the channel to the leader is already closed
before reaching the limit, it calls ExecEndNode and doesn't call
ExecShutdownNode. Otherwise, if the worker finds the limit first, the
worker *shutdowns* itself then ends. Everything's clean.

On second thought, it seems a issue of ExecutePlan, rather than PHJ
itself. ExecutePlan should shutdown nodes when output channel is
broken.

regards.

--
Kyotaro Horiguchi
NTT Open Source Software Center