Detoasting and memory usage

From: apb18(at)cornell(dot)edu
To: pgsql-general(at)postgresql(dot)org
Subject: Detoasting and memory usage
Date: 2003-07-14 15:03:27
Message-ID: Pine.SOL.3.91.1030714102925.2638D-100000@travelers.mail.cornell.edu
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

Hi,
I discovered a slightly misleading (or at least non-intuitive to
me) behaviour of the pg_detoast_datum call while processing variable length
objects. I was reading a large number of text fields on fetched
results and noticed some strange memory behavior. I was seeing that in
some situations, memory usage would shoot up dramatically, while in
others it would not. Puzzled, I tracked it down my usage of
DatumGetTextP(), which eventually reduces to pg_detoast_datum. As it turns
out, most of the text fields I was processing were small enough not to
be toasted and everything was OK. For the larger ones, pg_detoast_datum
would give me a palloced copy, and since I was making all these calls in
the same memory context and not pfreeing the results, I had a memory
leak. Just pfreeing the result would obviously not work since
pg_detoast_datum doesn't return a palloced result in most cases (i.e.
smaller varlength objects). I ended up using DatumGetTextPCopy (to
ensure that the result was palloced) with a pfree and all problems were
solved.

I guess my situation is an odd fringe case that probably doesn't happen
too much, but I thought that knowledge of it might be useful to someone
somewhere.

-Aaron

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Stephan Szabo 2003-07-14 15:37:22 Re: different transaction handling between postgresql and
Previous Message Vincent Hikida 2003-07-14 14:54:45 Re: Fw: select null + 0 question