Re: [pgadmin4][patch] Use pytest test runner for unit tests

Hi

On Mon, Jun 4, 2018 at 3:26 PM, Joao De Almeida Pereira <
jdealmeidapereira(at)pivotal(dot)io> wrote:

> Known issues:
>>>
>>> - Python 2.7, the library we are using for assertions (Grappa) is
>>> failing while trying to assert on strings. We created a PR to the library:
>>> https://github.com/grappa-py/grappa/pull/43
>>> <https://github.com/grappa-py/grappa/pull/43> as soon as this gets
>>> in all the tests should pass
>>>
>>> Any guesses as to the ETA? Given that most of our dev, and our Windows
>> and Mac packages both run on 2.7 at the moment, it's clear that this is a
>> required fix before we can proceed.
>>
>
> Attached you can find the patch that bumps grappa to version 0.1.9 that
> support Python 2.7
>

I'll take a look.

>
>>> - Jenkins server need a change. Because we now run tests for a
>>> single database at a time the Jenkins flow need to change. Our proposal for
>>> this is to isolate each database in its own task something similar to the
>>> pipeline that we currently use internally:
>>>
>>> ​
>>>
>>> [image: Screen Shot 2018-06-01 at 3.36.37 PM.png]
>>>
>>> The boxes that we are pointing with the teal arrow represent 1 Python
>>> Version Against 1 Database. This can be accomplished using Jenkins as well.
>>> In order to accomplish that we are going to generate e Jenkinsfile(
>>> https://jenkins.io/doc/book/pipeline/jenkinsfile/) to do this in a more
>>> DSL manner, making it easier for us to deploy the pipeline in a more
>>> consistent way. The LTS version of jenkins is 2.107.3 the version that we
>>> have in CI should be ok, but is always good to be in a LTS version of
>>> Jenkins or newer because the community is pretty active.
>>> The idea would be to replace the 5 tasks that we have there and start
>>> using a pipeline that will spawn docker containers to isolate the testing
>>> between version/database. That is what we do with concourse as we depicted
>>> in the previous image.
>>> Does anyone have any thoughts about this?
>>>
>> Well the current public Jenkins system is going away soon, and the new
>> one has had a ton of jobs created that assume the tests run in series
>> automatically against each database server (and is completely different
>> from the current system). I do plan to change that in the longer term, but
>> it requires a change to the way I've been using (and know how to use)
>> Jenkins.
>>
>
> Do you have a link to the new CI that you can share with us so we can
> take a pick. We can give some advises on what we believe it is a a good
> flow for the Continuous Delivery pipeline.
>

No, because it's firewalled to the nines inside our network. There's no
chance I'm making production build machines internet-accessible.

> We have some examples from our pipeline that we can share:
> Script that we use to run the UT + Feature tests on a docker image that
> has python+yarn+selenium+postgres installed on it:
> https://github.com/greenplum-db/pgadmin4-ci/blob/master/
> tasks/run-postgres-tests/run.sh
>
> This type of scripts can be added to the Jenkinsfile to create a pipeline
> step. A good practice in a reproducible pipeline is to use Docker to ensure
> that every test runs in a clean and predictable environment, this make it
> easy to reproduce a problem found in testing.
>

Docker is of little use to us here, as 2 of the 4 build platforms cannot be
run in Docker (macOS and the Docker container), and the 3rd would be
extremely difficult to run that way (Windows)

>
>
>> However, the bigger problem for me is that I often run the tests against
>> multiple DB servers on my laptop, without Jenkins. How would that workflow
>> look now?
>>
>>
> We understand that and it looks like an interesting use case and it could
> be accomplished by creating a script of something similar that launch the
> tests multiple times. That is something that we can schedule as a future
> steps after this patch is reviewed and committed.
>

We don't commit patches that remove functionality that people use
regularly, even if it's only intended to be temporary - that's simply not
how we do things in the community, as the risk of the second part not being
done is too high.

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company