Re: Feature test issues

On Mon, Feb 27, 2017 at 9:32 AM, Atira Odhner <aodhner(at)pivotal(dot)io> wrote:

> Cool, we already have a task about proper teardown and a couple other
> things in our backlog. we'll probably get to it in the next day or so. I'll
> take a look at the other stuff.
>
> Also, regarding speed, even without the app startup time, end to end tests
> are always going to be relatively slow. We definitely want to make sure
> that the time it takes to run the tests does not grow to where it is a
> deterrent to running them.
> There are a variety of things we can do to help address that as our suite
> grows. For instance, we could consider parallelizing the tests, making
> setup and teardown more efficient, combining related tests, or even
> breaking the tests into suites and running only some of them locally by
> default.
> Since we only have a couple feature tests so far the speed hasn't really
> felt like an issue for me yet, but I understand it may be different if you
> are trying to run in a variety of configurations.
>
> Out of curiosity, what is the goal in supporting multiple python versions?
>
We support Python 2.6, 2.7, 3.3+.

> Are we working on moving to 3.x and just haven't gotten fully there yet?
>
There is no plan to move to Python 3 only.
We do support Python 2.6 too, so that - it works on system like Cento OS 6+
out of box.

We (at EnterpriseDB) test pgAdmin 4 with Python 2.6, 2.7, 3.3, 3.4, 3.5 &
3.6.

--
Thanks,
Ashesh Vashi

> Tira
>
> On Sun, Feb 26, 2017, 3:39 AM Dave Page <dpage(at)pgadmin(dot)org> wrote:
>
>> Hi Tira, George,
>>
>> I've just been updating our internal automated test system to run the
>> feature tests, and ran into a couple of additional issues that need to
>> be addressed. Can you look into the following please?
>>
>> - When starting pgAdmin, it's using the default configuration database
>> (CONFIG['SQLITE_PATH']), however for testing we should be using
>> CONFIG['TEST_SQLITE_PATH']). This means that it's polluting the user's
>> default configuration (and if they don't have one, causing an
>> additional initialisation step).
>>
>> - With Python 2.6, the following failure is seen when the first
>> feature test is run:
>>
>> Traceback (most recent call last):
>> File "/var/lib/jenkins/workspace/pgadmin4-master-python26/web/
>> regression/runtests.py",
>> line 286, in <module>
>> verbosity=2).run(suite)
>> File "/var/lib/jenkins/workspace/pgadmin4-master-python26/
>> pgadmin-venv/lib/python2.6/site-packages/unittest2/runner.py",
>> line 172, in run
>> test(result)
>> File "/var/lib/jenkins/workspace/pgadmin4-master-python26/
>> pgadmin-venv/lib/python2.6/site-packages/unittest2/suite.py",
>> line 87, in __call__
>> return self.run(*args, **kwds)
>> File "/var/lib/jenkins/workspace/pgadmin4-master-python26/
>> pgadmin-venv/lib/python2.6/site-packages/unittest2/suite.py",
>> line 126, in run
>> test(result)
>> File "/var/lib/jenkins/workspace/pgadmin4-master-python26/
>> pgadmin-venv/lib/python2.6/site-packages/unittest2/case.py",
>> line 673, in __call__
>> return self.run(*args, **kwds)
>> File "/var/lib/jenkins/workspace/pgadmin4-master-python26/
>> pgadmin-venv/lib/python2.6/site-packages/unittest2/case.py",
>> line 633, in run
>> self._feedErrorsToResult(result, outcome.errors)
>> File "/var/lib/jenkins/workspace/pgadmin4-master-python26/
>> pgadmin-venv/lib/python2.6/site-packages/unittest2/case.py",
>> line 563, in _feedErrorsToResult
>> if issubclass(exc_info[0], self.failureException):
>> TypeError: issubclass() arg 2 must be a class or tuple of classes
>>
>> For completeness, other issues outstanding that we've previously
>> discussed:
>>
>> - pgAdmin processes may remain running after test failures.
>>
>> - The test suite may hang following a feature test failure, at the end
>> of the run.
>>
>> - The screenshot functionality should be fixed (ideally) or removed.
>>
>> - The tests really need to run with a single instantiation of pgAdmin.
>> It's clearly going to be far too slow to start/stop pgAdmin for every
>> test once we start adding more (and moving forward, I really want
>> feature tests to become the default to ensure we're end-to-end testing
>> everything). For reference, each test run (currently one version of
>> Python, against 5 different database servers) is now taking ~5 minutes
>> vs. 1m47s without the feature tests.
>>
>> On the plus side, test runs are now green across the board with
>> feature tests enabled, except for Python 2.6 :-)
>>
>> Thanks!
>>
>> --
>> Dave Page
>> Blog: http://pgsnake.blogspot.com
>> Twitter: @pgsnake
>>
>> EnterpriseDB UK: http://www.enterprisedb.com
>> The Enterprise PostgreSQL Company
>>
>