Re: Processing very large TEXT columns (300MB+) using C/libpq

From: Bear Giles <bgiles(at)coyotesong(dot)com>
To: Geoff Winkless <pgsqladmin(at)geoff(dot)dj>
Cc: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Cory Nemelka <cnemelka(at)gmail(dot)com>, Aldo Sarmiento <aldo(at)bigpurpledot(dot)com>, pgsql-admin <pgsql-admin(at)postgresql(dot)org>
Subject: Re: Processing very large TEXT columns (300MB+) using C/libpq
Date: 2017-10-21 11:32:00
Message-ID: CALBNtw7TzuoJ91hjQHqVQzjXqQg8US-hF17BH+LbLNo_+ayGEg@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-admin

​In that case you must put a read lock on the string that covers the loop.
If you're in a multi-threaded environment and not using locks when
appropriate then all bets are off.

On Fri, Oct 20, 2017 at 12:05 PM, Geoff Winkless <pgsqladmin(at)geoff(dot)dj>
wrote:

> On 20 October 2017 at 18:52, Bear Giles <bgiles(at)coyotesong(dot)com> wrote:
>
>> As an aside any halfway decent optimizer would realize that the results
>> of strlen() are unchanging as long as the contents of what it's passed
>> isn't modified. That's a common enough pattern that it should be checked.
>>
>
> ​IME this is a myth perpetuated by bad computer science lecturers who
> haven't thought through the consequences of what they're saying.​ strlen()
> can change because of changes inside the loop but also because of also
> threads in the system; I've not yet seen a compiler optimise that away, and
> neither should it, IMO.
>
> G
>

In response to

Responses

Browse pgsql-admin by date

  From Date Subject
Next Message Ervin Weber 2017-10-22 14:14:25 confusing .pgpass behaviour for undocumented replication=true connection parameter
Previous Message Geoff Winkless 2017-10-20 18:05:46 Re: Processing very large TEXT columns (300MB+) using C/libpq