Re: WIP: [[Parallel] Shared] Hash

From: Thomas Munro <thomas(dot)munro(at)enterprisedb(dot)com>
To: Andres Freund <andres(at)anarazel(dot)de>
Cc: Peter Geoghegan <pg(at)bowt(dot)ie>, Rafia Sabih <rafia(dot)sabih(at)enterprisedb(dot)com>, Ashutosh Bapat <ashutosh(dot)bapat(at)enterprisedb(dot)com>, Haribabu Kommi <kommi(dot)haribabu(at)gmail(dot)com>, Pg Hackers <pgsql-hackers(at)postgresql(dot)org>, Robert Haas <robertmhaas(at)gmail(dot)com>
Subject: Re: WIP: [[Parallel] Shared] Hash
Date: 2017-03-26 22:41:30
Message-ID: CAEepm=1nsja-tr1v6GWNNHPpCLiuodV3CxNf5vqXYxhekJpUdQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Mon, Mar 27, 2017 at 11:03 AM, Thomas Munro
<thomas(dot)munro(at)enterprisedb(dot)com> >> Is there a risk that this ends up
running afoul of filename length
>> limits on some platforms?
>
> Hmm. I didn't think so. Do we have a project guideline on maximum
> path lengths based on some kind of survey? There are various limits
> involved (filesystem and OS per-path-component limits, total limits,
> and the confusing PATH_MAX, MAX_PATH etc macros), but I was under the
> impression that these numbers were always at least 255. This scheme
> seems capable of producing ~50 bytes in the final component
> (admittedly more if int is 64 bits), and then nowhere near enough to
> reach a limit of that order in the earlier components.

Err, plus prefix. Still seems unlikely to be too long.

>> I'm a bit unhappy with the partition terminology around this. It's
>> getting a bit confusing. We have partitions, participants and
>> segements. Most of them could be understood for something entirely
>> different than the meaning you have here...
>
> Ok. Let me try to explain and defend them and see if we can come up
> with something better.
>
> 1. Segments are what buffile.c already calls the individual
> capped-at-1GB files that it manages. They are an implementation
> detail that is not part of buffile.c's user interface. There seems to
> be no reason to change that.

After reading your next email I realised this is not quite true:
BufFileTell and BufFileSeek expose the existence of segments.

--
Thomas Munro
http://www.enterprisedb.com

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Peter Geoghegan 2017-03-26 23:12:37 Re: WIP: [[Parallel] Shared] Hash
Previous Message David Rowley 2017-03-26 22:28:39 Re: Performance improvement for joins where outer side is unique