From: | Lars Kanis <lars(at)greiz-reinsdorf(dot)de> |
---|---|
To: | pgsql-hackers(at)lists(dot)postgresql(dot)org |
Subject: | libpq: Process buffered SSL read bytes to support records >8kB on async API |
Date: | 2024-09-08 20:07:53 |
Message-ID: | 2039ac58-d3e0-434b-ac1a-2a987f3b4cb1@greiz-reinsdorf.de |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
Dear hackers,
I'm the maintainer of ruby-pg the ruby interface to the PostgreSQL
database. This binding uses the asynchronous API of libpq by default to
facilitate the ruby IO wait and scheduling mechanisms.
This works well with the vanilla postgresql server, but it leads to
starvation with other types of servers using the postgresql wire
protocol 3. This is because the current functioning of the libpq async
interface depends on a maximum size of SSL records of 8kB.
The following servers were reported to starve with ruby-pg:
* AWS RDS Aurora Serverless [1]
* YugabyteDb [2]
* CockroachDB [3]
They block infinitely on certain message sizes sent from the backend to
the libpq frontend. It is best described in [4]. A repro docker
composition is provided by YugabyteDB at [2].
To fix this issue the attached patch calls pqReadData() repeatedly in
PQconsumeInput() until there is no buffered SSL data left to be read.
Another solution could be to process buffered SSL read bytes in
PQisBusy() instead of PQconsumeInput() .
The synchronous libpq API isn't affected, since it supports arbitrary
SSL record sizes already. That's why I think that the asynchronous API
should also support bigger SSL record sizes.
Regards, Lars
[1] https://github.com/ged/ruby-pg/issues/325
[2] https://github.com/ged/ruby-pg/issues/588
[3] https://github.com/ged/ruby-pg/issues/583
[4] https://github.com/ged/ruby-pg/issues/325#issuecomment-737561270
Attachment | Content-Type | Size |
---|---|---|
0001-libpq-Process-buffered-SSL-read-bytes-to-support-rec.patch | text/x-patch | 1.7 KB |
From | Date | Subject | |
---|---|---|---|
Next Message | Thomas Munro | 2024-09-08 20:55:31 | Re: CI, macports, darwin version problems |
Previous Message | Noah Misch | 2024-09-08 16:53:55 | Re: Yet another way for pg_ctl stop to fail on Windows |