Fwd: [LibPQ] Trying PQconnectStart: it doesn't seem to connect

From: Alessandro Agosto <the(dot)6one6(at)gmail(dot)com>
Date: 2010/1/24
Subject: Re: [GENERAL] [LibPQ] Trying PQconnectStart: it doesn't seem to
connect
To: Craig Ringer <craig(at)postnewspapers(dot)com(dot)au>

Hi, thank you for your reply.

2010/1/24 Craig Ringer <craig(at)postnewspapers(dot)com(dot)au>

What's wrong with psycopg2 for this purpose?
>
> Python's threading support (in the standard CPython interpreter) is awful
> due to the global interpreter lock. However, it works for waiting on
> blocking sockets as the GIL is released before entering most C-language
> routines. So you should be able to use psycopg2 in dedicated I/O worker
> threads just fine.
>
> If you're trying to use non-blocking sockets and select(...) with libpq,
> well, _then_ you'll have to go outside psycopg2. Be aware, though, that
> using SSL sockets in a non-blocking manner can be ... complicated ... so
> look into that in detail before deciding on this path. Multiple threads with
> blocking connections is likely to be a LOT easier.
>
Yes, i know psycopg2 and i like it. But as you said i'm trying to use
non-blocking socket and epoll (but writing a mock up i can use poll and work
on events later). Honestly i'm not aware on every problem that async
programming comport (like specific ssl problems). Anyway multiplexing
programming is, yes complicated, but also powerful and a program that uses
multiplexing has a lower footprint than thread/fork programs. In python, as
you said there is the "evil" GIL.

Frankly, though, you're never going to get wonderful results out of this.
> Twisted tries, but you've probably seen the issues it has working around the
> GIL and the limited success it has doing so. CPython's GIL dooms it to be a
> pretty shoddy language for high concurrency use, whether you use an async
> socket server model or a blocking threaded model.
>
I know twisted but sincerely i don't like to write code that makes a lot of
callbacks, but this is my personal opinion. I've used it a lot of times.
Probably you are right but on network programming async servers are faster
than classic threaded versions, also if mean more problems. The lack of an
asynchronous database interface is a problem that i could solve using more
processes of my webserver but this isn't a real solution. So i'm on this
project.

Can someone tell me what i wrong? And if you know some resource that
> explains with an example how i should to verify the connection during
> the select/poll, would be appreciated.
>

You can't reliably verify that a connection is alive with select/poll.
> TCP/IP timeouts are very long and until the connection times out at the
> TCP/IP level, it might appear fine even though the peer died hours ago.
>
I'm not yet within select/poll cycle, this is the first call that should
return CONNECTION_OK or CONNECTION_BAD (refering to docs). Probably i wrong
but this call seems to return another status, or as you said i cannot know
(also if i'm out of any cycle) if the connection is ok or not.

If the async apis became a problem , can i call the *synchronous* apis
through my webserver's main event loop? This should work or not? (i mean to
make the connection asynchronous).

Thank you again,
Greetings.
--
Alessandro A.

--
Alessandro A.