Re: small parallel restore optimization

From: Magnus Hagander <magnus(at)hagander(dot)net>
To: Andrew Dunstan <andrew(at)dunslane(dot)net>
Cc: Alvaro Herrera <alvherre(at)commandprompt(dot)com>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: small parallel restore optimization
Date: 2009-03-09 09:53:04
Message-ID: 49B4E700.8000106@hagander.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Andrew Dunstan wrote:
>
>
> Alvaro Herrera wrote:
>> Andrew Dunstan wrote:
>>
>>
>>> I have found the source of the problem I saw. dumputils.c:fmtId()
>>> uses a static PQExpBuffer which it initialises the first time it's
>>> called. This gets clobbered by simultaneous calls by Windows threads.
>>>
>>> I could just make it auto and set it up on each call, but that could
>>> result in a non-trivial memory leak ... it's probably called a great
>>> many times. Or I could provide a parallel version where we pass in a
>>> PQExpBuffer that we create, one per thread, and is used by anything
>>> called by the parallel code. That seems like a bit of a potential
>>> footgun, though.
>>>
>>
>> Could you use a different static PQExpBuffer on each thread?
>> pthread_getspecific sort of thing, only to be used on Windows.
>>
>
> For MSVC we could declare it with "_declspec(thread)" and it would be
> allocated in thread-local storage, but it looks like this isn't
> supported on Mingw.

Yeah, _declspec(thread) would be the easy way to do it. But you can also
use the TlsAlloc(), TlsSetValue() and friends functions directly. We do
this to use TLS in ecpg.

It requires some coding around, but for ecpg we did a macro that makes
it behave like the posix functions (see
src/interfaces/ecpg/include/ecpg-pthread-win32.h)

//Magnus

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Alvaro Herrera 2009-03-09 12:18:28 problem inserting in GIN index
Previous Message Magnus Hagander 2009-03-09 09:48:10 Re: small parallel restore optimization