From: | Melanie Plageman <melanieplageman(at)gmail(dot)com> |
---|---|
To: | Thomas Munro <thomas(dot)munro(at)gmail(dot)com> |
Cc: | Tomas Vondra <tomas(dot)vondra(at)2ndquadrant(dot)com>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org> |
Subject: | Re: Avoiding hash join batch explosions with extreme skew and weird stats |
Date: | 2019-06-06 23:37:19 |
Message-ID: | CAAKRu_Y++nBtuSzsKdpx=Rm8xminAEVaBGTDZA=+k7yJgi3KYQ@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
On Thu, May 16, 2019 at 3:22 PM Thomas Munro <thomas(dot)munro(at)gmail(dot)com> wrote:
> Admittedly I don't have a patch, just a bunch of handwaving. One
> reason I haven't attempted to write it is because although I know how
> to do the non-parallel version using a BufFile full of match bits in
> sync with the tuples for outer joins, I haven't figured out how to do
> it for parallel-aware hash join, because then each loop over the outer
> batch could see different tuples in each participant. You could use
> the match bit in HashJoinTuple header, but then you'd have to write
> all the tuples out again, which is more IO than I want to do. I'll
> probably start another thread about that.
>
>
Going back to the idea of using the match bit in the HashJoinTuple header
and writing out all of the outer side for every chunk of the inner
side, I was wondering if there was something we could do that was kind
of like mmap'ing the outer side file to give the workers in parallel
hashjoin the ability to update a match bit in the tuple in place and
avoid writing the whole outer side out each time.
--
Melanie Plageman
From | Date | Subject | |
---|---|---|---|
Next Message | Tom Lane | 2019-06-07 00:12:57 | Re: LLVM compile failing in seawasp |
Previous Message | Melanie Plageman | 2019-06-06 23:33:31 | Re: Avoiding hash join batch explosions with extreme skew and weird stats |