Re: libpq: PQgetCopyData() and allocation overhead

From: Jeroen Vermeulen <jtvjtv(at)gmail(dot)com>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Jelte Fennema <postgres(at)jeltef(dot)nl>, daniel(at)yesql(dot)se, Bharath Rupireddy <bharath(dot)rupireddyforpostgres(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: libpq: PQgetCopyData() and allocation overhead
Date: 2023-03-03 19:30:37
Message-ID: CA+zULE7_7tLNiqiA4PZs5VQ9yzAxrNbSgfSBJxYN_q1bymo1Eg@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Interested, yes. But there may be reasons not to do that. Last time I
looked the binary format wasn't documented.

Anyway, I tried re-implementing pqGetCopyData3() using the callback.
Wasn't hard, but I did have to add a way for the callback to return an
error. Kept it pretty dumb for now, hoping a sensible rule will become
obvious later.

Saw no obvious performance impact, so that's good.

Jeroen

On Fri, 3 Mar 2023 at 19:53, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:

> Jeroen Vermeulen <jtvjtv(at)gmail(dot)com> writes:
> > The printf() is just the simplest example that sprang to mind though.
> > There may be other use-cases out there involving libraries that require
> > zero-terminated strings, and I figured an ability to set a sentinel could
> > help those.
>
> Well, since it won't help for binary COPY, I'm skeptical that this is
> something we should cater to. Anybody who's sufficiently worried about
> performance to be trying to remove malloc/free cycles ought to be
> interested in binary COPY as well.
>
> regards, tom lane
>

Attachment Content-Type Size
pqhandlecopydata.diff text/x-patch 11.4 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Andrew Dunstan 2023-03-03 19:43:25 Re: meson: Optionally disable installation of test modules
Previous Message Justin Pryzby 2023-03-03 18:55:46 Re: zstd compression for pg_dump