From: | Andrew Dunstan <andrew(at)dunslane(dot)net> |
---|---|
To: | Magnus Hagander <magnus(at)hagander(dot)net> |
Cc: | Alvaro Herrera <alvherre(at)commandprompt(dot)com>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org> |
Subject: | Re: small parallel restore optimization |
Date: | 2009-03-09 15:46:19 |
Message-ID: | 49B539CB.3040703@dunslane.net |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
Magnus Hagander wrote:
> Andrew Dunstan wrote:
>
>> Alvaro Herrera wrote:
>>
>>> Andrew Dunstan wrote:
>>>
>>>
>>>
>>>> I have found the source of the problem I saw. dumputils.c:fmtId()
>>>> uses a static PQExpBuffer which it initialises the first time it's
>>>> called. This gets clobbered by simultaneous calls by Windows threads.
>>>>
>>>> I could just make it auto and set it up on each call, but that could
>>>> result in a non-trivial memory leak ... it's probably called a great
>>>> many times. Or I could provide a parallel version where we pass in a
>>>> PQExpBuffer that we create, one per thread, and is used by anything
>>>> called by the parallel code. That seems like a bit of a potential
>>>> footgun, though.
>>>>
>>>>
>>> Could you use a different static PQExpBuffer on each thread?
>>> pthread_getspecific sort of thing, only to be used on Windows.
>>>
>>>
>> For MSVC we could declare it with "_declspec(thread)" and it would be
>> allocated in thread-local storage, but it looks like this isn't
>> supported on Mingw.
>>
>
> Yeah, _declspec(thread) would be the easy way to do it. But you can also
> use the TlsAlloc(), TlsSetValue() and friends functions directly. We do
> this to use TLS in ecpg.
>
> It requires some coding around, but for ecpg we did a macro that makes
> it behave like the posix functions (see
> src/interfaces/ecpg/include/ecpg-pthread-win32.h)
>
>
>
Yeah, we'll just have to call TlsAlloc to set the thread handle
somewhere near program start, but that shouldn't be too intrusive.
cheers
andrew
From | Date | Subject | |
---|---|---|---|
Next Message | Umar Farooq | 2009-03-09 16:39:47 | PostgreSQL Memory Management |
Previous Message | Greg Sabino Mullane | 2009-03-09 15:22:47 | One less footgun: deprecating pg_dump -d |