Re: pg_dump: Error message from server: lost synchronization with server: got messag e type "d",

Here is the version for libpq

###########
Package: libpq-dev
Priority: optional
Section: libdevel
Installed-Size: 724
Maintainer: Ubuntu Developers <ubuntu-devel-discuss(at)lists(dot)ubuntu(dot)com>
Original-Maintainer: Debian PostgreSQL Maintainers
<pkg-postgresql-public(at)lists(dot)

alioth.debian.org>
Architecture: amd64
Source: postgresql-9.3
Version: 9.3.8-0ubuntu0.4.04
Depends: libc6 (>= 2.4), libpq5 (= 9.3.8-0ubuntu0.4.04), libssl-dev,
krb5-multid

ev, comerr-dev
Suggests: postgresql-doc-9.3
Filename: pool/main/p/postgresql-9.3/libpq-dev_9.3.8-0ubuntu0.4.04_amd64.deb
Size: 139652
MD5sum: 35758649b8688c4126a6b15bf8664dac
SHA1: f77498099374832be4a92c63f23dcc681a57c9dd
SHA256: fb64325c76bb1b435d80e3edc635d75e2febee2fb4fc606734a5afd7d2461962
Description-en: header files for libpq5 (PostgreSQL library)
Header files and static library for compiling C programs to link
with the libpq library in order to communicate with a PostgreSQL
database backend.
.
PostgreSQL is an object-relational SQL database management system.
Description-md5: 7f4362b106aae6b219ccc880faa1f04c
Homepage: http://www.postgresql.org/
Bugs: https://bugs.launchpad.net/ubuntu/+filebug
Origin: Ubuntu
Supported: 9m

Package: libpq-dev
Priority: optional
Section: libdevel
Installed-Size: 728
Maintainer: Ubuntu Developers <ubuntu-devel-discuss(at)lists(dot)ubuntu(dot)com>
Original-Maintainer: Debian PostgreSQL Maintainers
<pkg-postgresql-public(at)lists(dot)

alioth.debian.org>
Architecture: amd64
Source: postgresql-9.3
Version: 9.3.7-0ubuntu0.14.04
Depends: libc6 (>= 2.4), libpq5 (= 9.3.7-0ubuntu0.14.04), libssl-dev,
krb5-multi

dev, comerr-dev
Suggests: postgresql-doc-9.3
Filename:
pool/main/p/postgresql-9.3/libpq-dev_9.3.7-0ubuntu0.14.04_amd64.deb
Size: 139954
MD5sum: 28d32f1ad1e5080c97bce2d46c434aee
SHA1: 73c2d36a1054241cfea0612f7e34388f106bd98a
SHA256: 042770afe0a71be9bf5ca30ac49579e829b15cff0495da7d09224f9c635446a8
Description-en: header files for libpq5 (PostgreSQL library)
Header files and static library for compiling C programs to link
with the libpq library in order to communicate with a PostgreSQL
database backend.
.
PostgreSQL is an object-relational SQL database management system.
Description-md5: 7f4362b106aae6b219ccc880faa1f04c
Homepage: http://www.postgresql.org/
Bugs: https://bugs.launchpad.net/ubuntu/+filebug
Origin: Ubuntu
Supported: 9m

Package: libpq-dev
Priority: optional
Section: libdevel
Installed-Size: 720
Maintainer: Ubuntu Developers <ubuntu-devel-discuss(at)lists(dot)ubuntu(dot)com>
Original-Maintainer: Debian PostgreSQL Maintainers
<pkg-postgresql-public(at)lists(dot)

alioth.debian.org>
Architecture: amd64
Source: postgresql-9.3
Version: 9.3.4-1
Depends: libc6 (>= 2.4), libpq5 (= 9.3.4-1), libssl-dev, krb5-multidev,
comerr-d

ev
Suggests: postgresql-doc-9.3
Filename: pool/main/p/postgresql-9.3/libpq-dev_9.3.4-1_amd64.deb
Size: 138888
MD5sum: 56d8b168cd059c17ce29211ba8610111
SHA1: 28fb61e9b8791c7934e28f7ffe604ef2875e69b9
SHA256: e03839622bba3c67a876894aa9863b1be11f50c104a3e1582005393c4278c585
Description-en: header files for libpq5 (PostgreSQL library)
Header files and static library for compiling C programs to link
with the libpq library in order to communicate with a PostgreSQL
database backend.
.
PostgreSQL is an object-relational SQL database management system.
Description-md5: 7f4362b106aae6b219ccc880faa1f04c
Homepage: http://www.postgresql.org/
Bugs: https://bugs.launchpad.net/ubuntu/+filebug
Origin: Ubuntu
##################

On Tue, Jun 23, 2015 at 7:44 AM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:

> I wrote:
> > Hm. Given that the message type and length seem perfectly reasonable,
> > I suspect this must actually represent an out-of-memory condition within
> > pg_dump (*not* on the server end). But you'd have to be running it on a
> > toy box, or with a rather silly ulimit, for 6MB to be a problem...
>
> BTW, how old is your pg_dump (or really, libpq)? I wonder if you are
> hitting this bug in some form:
>
> Author: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
> Branch: master Release: REL9_4_BR [2f557167b] 2014-05-07 21:39:13 -0400
> Branch: REL9_3_STABLE Release: REL9_3_5 [b4f9c93ce] 2014-05-07 21:38:38
> -0400
> Branch: REL9_2_STABLE Release: REL9_2_9 [f7672c8ce] 2014-05-07 21:38:41
> -0400
> Branch: REL9_1_STABLE Release: REL9_1_14 [86888054a] 2014-05-07 21:38:44
> -0400
> Branch: REL9_0_STABLE Release: REL9_0_18 [77e662827] 2014-05-07 21:38:47
> -0400
> Branch: REL8_4_STABLE Release: REL8_4_22 [664ac3de7] 2014-05-07 21:38:50
> -0400
>
> Avoid buffer bloat in libpq when server is consistently faster than
> client.
>
> If the server sends a long stream of data, and the server + network are
> consistently fast enough to force the recv() loop in pqReadData() to
> iterate until libpq's input buffer is full, then upon processing the
> last
> incomplete message in each bufferload we'd usually double the buffer
> size,
> due to supposing that we didn't have enough room in the buffer to
> finish
> collecting that message. After filling the newly-enlarged buffer, the
> cycle repeats, eventually resulting in an out-of-memory situation
> (which
> would be reported misleadingly as "lost synchronization with server").
> Of course, we should not enlarge the buffer unless we still need room
> after discarding already-processed messages.
>
> This bug dates back quite a long time: pqParseInput3 has had the
> behavior
> since perhaps 2003, getCopyDataMessage at least since commit
> 70066eb1a1ad
> in 2008. Probably the reason it's not been isolated before is that in
> common environments the recv() loop would always be faster than the
> server
> (if on the same machine) or faster than the network (if not); or at
> least
> it wouldn't be slower consistently enough to let the buffer ramp up to
> a
> problematic size. The reported cases involve Windows, which perhaps
> has
> different timing behavior than other platforms.
>
> Per bug #7914 from Shin-ichi Morita, though this is different from his
> proposed solution. Back-patch to all supported branches.
>
> regards, tom lane
>

--
Best Regards,
Sanjay Veeramachaneni
Devops Engineer
989 E. Hillsdale Blvd., Suite 450, Foster City, CA 94404

[image: Oration logo]
<https://app.relateiq.com/r?url=http%3A%2F%2Fwww.oration.com%2F&t=AFwhZf25ZVW3GdSe1A7Wy2aORPtvFTNFaQlxnSXL4U9ZiYfs-EKrMdVj-aAHTKDmJ_qTViY0PR5CyTea_faPfi7gjFKPXtHHJgzXbLso79buSjigF-dKMMDp0qWE6vg31ReU4oVmtf5U>