Re: Testing with concurrent sessions

Hi,

Jan Urbański wrote:
> sorry to butt in to the conversation, but I have spent some time
> wrapping/refining the concepts in dtester, and the results are here:
>
> http://git.wulczer.org/?p=twisted-psql.git;a=summary

That seems to cover the concurrent psql part of dtester. But I don't see
how that's wrapping or refining dtester.

> I borrowed the idea of wrapping a psql in a Twisted protocol and added a
> Deferred interface around it, which made it possible to run tests with
> trial: the Twisted unit testing framework.

dtester works pretty much the same way, except that it doesn't use
trial. But the Deferred interface is about the same.

> As a developer of a failry large Python system based on Twisted, that
> sports hundreds of trial-based tests, I very strongly recommend trial
> for asynchronous unit testing. It handles lots of boring details, is
> well maintained and Twisted itself is simply designed to do asynchronous
> programming. As an added bonus, the runnning and reporting
> infrastructure is already there, you just write the tests.

I'm using trial for two other projects as well and I've started with it
for Postgres. IMO trial is very good for unit tests, but fails horribly
for anything that involves complex setups and watch-guarding of multiple
processes. That's where dtester shines.

> My code is very rough and lacks good error reporting, for instance
> failed tests will probably result in a "test hung" and the need to
> Ctrl+C, but that can be easily improved.

Don't underestimate that. There are lots of sources of errors. Not
having a sentinel over the postmaster itself, unit tests from trial
simply fail to notice errors there. (And coding for Postgres, these are
the ones you are interested in). Now combine all of that and error
handling and (useful) reporting is *the* major challenge in automating
testing.

> A thing that would help
> tremendously would be a real Twisted protocol that talks to PG on the
> protocol level, not by parsing psql output (which is very clumsy and
> error prone IMHO).

I agree. (Well, now I do).

> I found one such project:
> http://www.jamwt.com/pgasync/
> but it had some issues with committing

Yeah, I didn't like that one, either.

Regards

Markus