Re: Counting the occurences of a substring within a very large text

From: Marc Mamin <M(dot)Mamin(at)intershop(dot)de>
To: "'Geoff Winkless'" <pgsqladmin(at)geoff(dot)dj>, Postgres General <pgsql-general(at)postgresql(dot)org>
Subject: Re: Counting the occurences of a substring within a very large text
Date: 2015-06-24 15:05:18
Message-ID: B6F6FD62F2624C4C9916AC0175D56D8828BEA89E@jenmbs01.ad.intershop.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

>>On 24 June 2015 at 14:51, Marc Mamin <M(dot)Mamin(at)intershop(dot)de> wrote:
>>note that the 345MB text only contains 635 lines. This might be the issue...

>There's similar issue discussed here:

>http://www.postgresql.org/message-id/6046.1353874252@sss.pgh.pa.us

>Tom did seem to accept that the attempted alloc is larger than it needs to be, but even if it were fixed you would still run into problems further down the line managing values of that size.

Thanks,

does it means that the issue would also occur in a (not yet existing) function that would just count the substring, without having to manipulate the text itself ?
or in other words, are there string functions that can handle such large text in stream, or is it always necessary to alloc enough memory for the whole text first ?

Marc

In response to

Browse pgsql-general by date

  From Date Subject
Next Message Gauthier, Dave 2015-06-24 15:13:49 Re: DB access speeds, App(linux)<->PG(linux) vs App(linux) <->MSSql(Windows)
Previous Message Holger.Friedrich-Fa-Trivadis 2015-06-24 14:55:52 Re: mirroring a server and/or hot standby