Using COPY command in pqlib

From: Lukáš Sobotka <sobotka(dot)luk(at)gmail(dot)com>
To: pgsql-interfaces(at)postgresql(dot)org
Subject: Using COPY command in pqlib
Date: 2019-07-18 13:57:55
Message-ID: CAJif3k+Q_nksEkS3SwpaButxxTcOkjgG91_sLLcc=986-2VZAg@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-interfaces

Hi guys,

I would like to ask you on correct way how to use pqlib for copying data.
We tried to do it according documentation but some problems occurred.

We have a multi-threaded application, all threads hold own non-blocking
connection into the remote database. Each thread is collecting data and
when a buffer is filled they copy data to remote DB as quick as possible.
Copying can run several times per minute. The postgresql related part of
thread main loop is briefly shown in attached file main.c.

Most of the time copying works well but big delay sometimes occurred. I
captured pcap (see https://ibb.co/2MhmSTZ) and the delay is between getting
PGRES_COPY_IN from PQresultStatus function and pushing data. But I am sure
that commands PQputCopyData and PQputCopyEnd are called immediately after
client gets PQexec.

What am I missing? Is there some other command which we should use?

We are using PostgreSQL 9.6 (PostgreSQL 9.6.10 on x86_64-pc-linux-gnu,
compiled by gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-28), 64-bit).

Best regards,
Lukas

Attachment Content-Type Size
main.c text/x-csrc 566 bytes

Responses

Browse pgsql-interfaces by date

  From Date Subject
Next Message Tom Lane 2019-07-18 22:26:41 Re: Using COPY command in pqlib
Previous Message Ashutosh Sharma 2019-07-18 11:05:02 Support for CALL statement in ecpg