Re: Parallel query and temp_file_limit

On Tue, May 17, 2016 at 6:40 PM, Peter Geoghegan <pg(at)heroku(dot)com> wrote:
> On Tue, May 17, 2016 at 3:33 PM, Peter Geoghegan <pg(at)heroku(dot)com> wrote:
>> Fundamentally, since temporary_files_size enforcement simply
>> piggy-backs on low-level fd.c file management, without any
>> consideration of what the temp files contain, it'll be hard to be sure
>> that parallel workers will not have issues. I think it'll be far
>> easier to fix the problem then it would be to figure out if it's
>> possible to get away with it.
>
> I'll write a patch to fix the issue, if there is a consensus on a solution.

I think for 9.6 we just have to document this issue. In the next
release, we could (and might well want to) try to do something more
clever.

What I'm tempted to do is trying to document that, as a point of
policy, parallel query in 9.6 uses up to (workers + 1) times the
resources that a single session might use. That includes not only CPU
but also things like work_mem and temp file space. This obviously
isn't ideal, but it's what could be done by the ship date.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company