Re: libpq and multi-threading

Hi Michael,

Are pthread_* functions really such an improvement over clone? Does it make
an 'freely passing around' of PGresult objects possible? Like it matters,
process or thread.

We were talking about the documentation and this 'freely passing around'
PGresult object. I just don't think it is as simple as the documentation
makes you believe.

On Wed, 3 May 2023, 14:35 Michael Loftis, <mloftis(at)wgops(dot)com> wrote:

>
> That is not a thread. Linux man clone right at the start …
>
> “clone, __clone2, clone3 - create a child process”
>
> What you want is pthread_create (or similar)
>
> There’s a bunch of not well documented dragons if you’re trying to treat a
> child process as a thread. Use POSIX Threads, as pretty much anytime PG or
> anything else Linux based says thread they’re talking about a POSIX Thread
> environment.
>
>
> On Wed, May 3, 2023 at 05:12 Michael J. Baars <
> mjbaars1977(dot)pgsql(dot)hackers(at)gmail(dot)com> wrote:
>
>> Hi Peter,
>>
>> The shared common address space is controlled by the clone(2) CLONE_VM
>> option. Indeed this results in an environment in which both the parent and
>> the child can read / write each other's memory, but dynamic memory being
>> allocated using malloc(3) from two different threads simulaneously will
>> result in internal interference.
>>
>> Because libpq makes use of malloc to store results, you will come to find
>> that the CLONE_VM option was not the option you were looking for.
>>
>> On Tue, 2 May 2023, 19:58 Peter J. Holzer, <hjp-pgsql(at)hjp(dot)at> wrote:
>>
>>> On 2023-05-02 17:43:06 +0200, Michael J. Baars wrote:
>>> > I don't think it is, but let me shed some more light on it.
>>>
>>> One possibly quite important information you haven't told us yet is
>>> which OS you use.
>>>
>>> Or how you create the threads, how you pass the results around, what
>>> else you are possibly doing between getting the result and trying to use
>>> it ...
>>>
>>> A short self-contained test case might shed some light on this.
>>>
>>>
>>> > After playing around a little with threads and memory, I now know that
>>> the
>>> > PGresult is not read-only, it is read-once. The child can only read
>>> that
>>> > portion of parent memory, that was written before the thread started.
>>> Read-only
>>> > is not strong enough.
>>> >
>>> > Let me correct my first mail. Making libpq use mmap is not good enough
>>> either.
>>> > Shared memory allocated by the child can not be accessed by the parent.
>>>
>>> Are you sure you are talking about threads and not processes? In the OSs
>>> I am familiar with, threads (of the same process) share a common address
>>> space. You don't need explicit shared memory and there is no such thing
>>> as "parent memory" (there is thread-local storage, but that's more a
>>> compiler/library construct).
>>>
>>> hp
>>>
>>> --
>>> _ | Peter J. Holzer | Story must make more sense than reality.
>>> |_|_) | |
>>> | | | hjp(at)hjp(dot)at | -- Charles Stross, "Creative writing
>>> __/ | http://www.hjp.at/ | challenge!"
>>>
>> --
>
> "Genius might be described as a supreme capacity for getting its possessors
> into trouble of all kinds."
> -- Samuel Butler
>