Re: psycopg2 conn.poll() hangs when used with sshtunnel

I have one more observation when i run this application one out of 10 times
it works. I have logged one more issue as COPY command hangs, so thought it
is the problem in psycopg2

On Tue, 30 Oct 2018, 21:22 Akshay Joshi <akshay(dot)joshi(at)enterprisedb(dot)com
wrote:

>
>
> On Tue, 30 Oct 2018, 21:06 Daniele Varrazzo <daniele(dot)varrazzo(at)gmail(dot)com
> wrote:
>
>> On Tue, Oct 30, 2018 at 1:55 PM Akshay Joshi <
>> akshay(dot)joshi(at)enterprisedb(dot)com> wrote:
>>
>>
>>> What changes should I made to fix this? Or it should be fixed in
>>> psycopg2 itself.
>>>
>>
>> Actually, I have another observation. I was surprised to see that the
>> whole test suite passed running the connection through an ssh tunnel: there
>> are definitely failing queries in the test suite. So I made other tests and
>> it seems the bad condition only happens using the `sshtunnel` module: If I
>> open a tunnel manually with something like:
>>
>> ssh -L 36421:localhost:5432 -N localhost
>>
>> and point your script to port 36421 everything works ok.
>>
>>
>> Conversely, I don't seem able to run the test suite through the tunnel
>> open by the sshtunnel module. It doesn't hang, but the tunnel breaks in a
>> test with the following reported on the sshtunnel side:
>>
>> 2018-10-30 15:24:25,550| ERROR | Socket exception: Bad file
>> descriptor (9)
>> 2018-10-30 15:24:25,551| ERROR | Could not establish connection
>> from ('127.0.0.1', 33743) to remote side of the tunnel
>>
>> and the following tests fail to run as the connection is broken.
>>
>>
>> Con-conversely, the patch I had in mind to fix #801, which very brutally
>> is:
>>
>> ```
>> @@ -1136,6 +1136,13 @@ pq_get_last_result(connectionObject *conn)
>> || status == PGRES_COPY_IN) {
>> break;
>> }
>> + if (PQisBusy(conn->pgconn)) {
>> + /* This happens connecting through ssl tunnel
>> + * TODO: just kill this function. The loop should happen
>> within
>> + * the async/green machinery. */
>> + Dprintf("pq_get_last_result: we are busy");
>> + break;
>> + }
>> }
>>
>> return result;
>> ```
>>
>> doesn't work, and failing queries leave the connection in an inconsistent
>> state.
>>
>>
>> So, wrapping up, I think there is something you can do on your side,
>> checking why the sshtunnel module behaves differently from a normal ssh
>> tunnel, and if you use the latter (or configure sshtunnel to behave so) you
>> shouldn't hit the problem. On our side I don't think we can fix #801 with a
>> quick band aid, and we should rather do #802, but as things stands now I
>> don't trust sshtunnel to do the right thing, and have less of an urgency to
>> do so.
>>
>> Please let us know if you understand what is the difference between the
>> module and the tunnel via `ssh -L`, thank you!
>>
>
> I have send the sample application and created an issue to sshtunnel
> git hub. Not sure where is the problem, as poll() function hangs so i
> thought its issue in psycopg2.
>
>>
>> -- Daniele
>>
>>>