Re: Processing very large TEXT columns (300MB+) using C/libpq

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Cory Nemelka <cnemelka(at)gmail(dot)com>
Cc: Aldo Sarmiento <aldo(at)bigpurpledot(dot)com>, pgsql-admin(at)postgresql(dot)org
Subject: Re: Processing very large TEXT columns (300MB+) using C/libpq
Date: 2017-10-20 16:55:22
Message-ID: 4750.1508518522@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-admin

Cory Nemelka <cnemelka(at)gmail(dot)com> writes:
> Yes, but I should be able to read them much faster. The psql client can
> display an 11MB column in a little over a minute, while in C using libpg
> library, it takes over an hour.

Well, of course psql relies on libpq, so it seems unlikely that libpq
itself is where the time is going. Have you tried applying a profiler?
"perf" or "oprofile" or similar tool ought to pinpoint the culprit
pretty easily.

regards, tom lane

In response to

Responses

Browse pgsql-admin by date

  From Date Subject
Next Message Bear Giles 2017-10-20 17:52:19 Re: Processing very large TEXT columns (300MB+) using C/libpq
Previous Message Cory Nemelka 2017-10-20 15:54:51 Re: Processing very large TEXT columns (300MB+) using C/libpq