Re: [pgadmin4][patch] Change testing framework to pytest and grappa

Hi

On Mon, Apr 30, 2018 at 8:48 PM, Joao De Almeida Pereira <
jdealmeidapereira(at)pivotal(dot)io> wrote:

> Hi Hackers,
> Attached you can find a patch that converts 3 unit test files to pytest
> and grappa. This patch is a first step on to this migration.
> We started this work because in our regular development we do TDD and with
> the current setup we cannot run a single test or even a group of tests, the
> minimum about is always a packages that can contain 1 test or 10. This
> makes the Red - Green - Refactor cycle(http://blog.cleancoder.
> com/uncle-bob/2014/12/17/TheCyclesOfTDD.html) much harder and longer then
> it should be.
> Also running tests in parallel was not possible, even if they were against
> different databases.
>
> After some investigation we found a couple of options to replace the
> current setup and pytest looked like a good fit the only issue was the it
> did not had a assertions library. For that we found a library called grappa
> that is actively maintained and have a nice syntax to it.
>
> 0001 Patch contains the configuration files for pytests and also the 3
> files that we converted to start using pytest + grappa
>
> What do we know
> - This converting will need a change in the way we run the tests in CI
> - not all tests were converted, this is just a wip to see what people
> think about this new framework
> - Support for running the same test suite against multiple databases in a
> single task, can be done, but we would have to add more code and we believe
> there is a better way to accomplish this
>
> For the CI we envision something similar the picture below
> [image: Screen Shot 2018-04-30 at 3.40.13 PM.png]
> In the middle we run 2 separate tasks one that runs the karma tests+js
> linter and another one that lints the python code.
> If everything looks good, then we run the boxes the the left, each box
> represent a combination of database + python version.
>
> We believe that using concouse or even jenkins we can achieve this making
> the tests run in parallel for all environments and databases. The newer
> versions of Jenkins have a pipeline DSL language that would allow us to
> create pipelines like this one we have in concourse.
>
>
> Ask
> - Take a look at the new assertions syntax
>

Yes, I like Grappa.

> - Give us your thoughts about this new CI structure
>

I am definitely a fan of adding flexibility to the test suite. My personal
bug-bear is not being able to run individual feature tests which I would
hope this will resolve. I would also be extremely happy to be able to run
against different database server versions in parallel. A pipeline such as
the one you depicted is quite an attractive proposition (says the guy
that's been hacking on Jenkins for the last couple of weeks).

BTW; I had to manually "pip install setuptools_scm" before I could add the
additional requirements from web/regression/requirements.txt. We may need
to explicitly include that to avoid what I assume is a bug in the pytest
package.

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company