Re: Acceptance Tests against a browser (WIP)

Hi Dave,

Here is a new patch which includes the following:
- randomized ports
- delete the acceptance_test_db database in setup in case a prior run failed
- fixed size browser window

Cheers,
Tira & George

On Tue, Jan 31, 2017 at 11:25 AM, Dave Page <dpage(at)pgadmin(dot)org> wrote:

> Hi George,
>
> I just tried to do some debugging of pgAdmin, and found that I
> couldn't start it. On further investigation, I found that I had an
> instance running in the background on my system. I'm assuming this was
> started by the acceptance tests, but not shutdown. I killed it off,
> and re-ran the tests only to see failures because the database and
> table used in the acceptance tests were still present. When the tests
> completed, pgAdmin was again left running in the background.
>
> I've just re-run the tests, having first killed the backgrounded
> pgAdmin and then manually cleaned up the test objects. This time I do
> indeed only get the two errors below when it tests the first of 3
> servers I have configured. The second and third servers get three
> errors each, and pgAdmin is left running in the background again.
>
> So, you were right that I had another instance of pgAdmin running...
> but it was tests that caused it :-p
>
>
>
> On Tue, Jan 31, 2017 at 3:10 PM, Dave Page <dpage(at)pgadmin(dot)org> wrote:
> > Hi
> >
> > On Tue, Jan 31, 2017 at 2:54 PM, George Gelashvili
> > <ggelashvili(at)pivotal(dot)io> wrote:
> >> Hi Dave,
> >>
> >> We agree that a random port would be a nice addition. We think having
> >> randomized test database names can lead to polluting with lots of extra
> >> databases left around in the event that cleanup fails for whatever
> reason
> >> (e.g. a test errors out). We see this happen already with the
> randomized
> >> test databases you mention. We agree that there should probably be one
> >> strategy across the test suite. We could use randomized names and have a
> >> more general cleanup step that removes all databases of the form
> "test_...".
> >
> > I'm very wary about doing things like that. We had an early version of
> > the suite that managed to delete all databases :-/. Maybe we could use
> > a patterned name, but only delete databases that also have a comment
> > with some text in it that we can verify?
> >
> >> Dave, are those errors you saw when you shut down your application on
> :5050
> >> and did a fresh run of the tests? If not, could you please do a clean
> run?
> >> It's possible that the second error could be related to viewport size
> as you
> >> suggested, but the first error just looks like a problem with the test
> not
> >> being able to spin up its own server.
> >
> > That was on a second run of the tests, yes. I just did a careful
> > cleanup of left-over test databases, double-checked my server wasn't
> > running and re-ran the tests - I got the same results.
> >
> >>
> >> Thanks,
> >> George & Tira
> >>
> >> On Tue, Jan 31, 2017 at 9:41 AM, Dave Page <dpage(at)pgadmin(dot)org> wrote:
> >>>
> >>> Hi
> >>>
> >>> On Mon, Jan 30, 2017 at 9:23 PM, Atira Odhner <aodhner(at)pivotal(dot)io>
> wrote:
> >>> > Here's the patch with one more fix -- cleaning up the connections
> that
> >>> > get
> >>> > created in pgAdmin.
> >>>
> >>> Hmm, I had trouble with this one. I noticed a few issues:
> >>>
> >>> - The tests started pgAdmin listening on the default port (5050),
> >>> however, I already had an instance running on there;
> >>> a) It should have detected that something else was running on the
> port
> >>> b) Shouldn't we just use a random, unused port?
> >>>
> >>> - Errors were given because I already had an acceptance_test_db on a
> >>> number of servers, and that contained the test table. Obviously the
> >>> code now cleans up after itself, but I think we should use a random
> >>> database name as the main regression tests do (they append a random
> >>> number to the name iirc).
> >>>
> >>> - Some of the tests just seemed to time out. I *think* this might be
> >>> because the test browser window opens quite narrowly, and it looks
> >>> like the tests are probably trying to do things with nodes that aren't
> >>> actually visible.
> >>>
> >>> ======================================================================
> >>> ERROR: runTest
> >>> (pgadmin.acceptance.tests.connect_to_server_feature_test.
> ConnectsToServerFeatureTest)
> >>> ----------------------------------------------------------------------
> >>> Traceback (most recent call last):
> >>> File
> >>> "/Users/dpage/git/pgadmin4/web/pgadmin/acceptance/tests/
> connect_to_server_feature_test.py",
> >>> line 69, in tearDown
> >>> self.app_starter.stop_app()
> >>> File "/Users/dpage/git/pgadmin4/web/regression/utils/app_
> starter.py",
> >>> line 27, in stop_app
> >>> os.killpg(os.getpgid(self.pgadmin_process.pid), signal.SIGTERM)
> >>> OSError: [Errno 3] No such process
> >>>
> >>> ======================================================================
> >>> ERROR: runTest
> >>> (pgadmin.acceptance.tests.sql_template_selection_by_
> postgres_version_works_feature_test.SQLTemplateSelectionByPostgres
> VersionWorksFeatureTest)
> >>> ----------------------------------------------------------------------
> >>> Traceback (most recent call last):
> >>> File
> >>> "/Users/dpage/git/pgadmin4/web/pgadmin/acceptance/tests/
> sql_template_selection_by_postgres_version_works_feature_test.py",
> >>> line 37, in runTest
> >>> self.page.find_by_xpath("//*[(at)id='tree']//*[(at)class='aciTreeText'
> >>> and .='Trigger Functions']").click()
> >>> File "/Users/dpage/git/pgadmin4/web/regression/utils/pgadmin_
> page.py",
> >>> line 45, in find_by_xpath
> >>> return self.wait_for_element(lambda:
> >>> self.driver.find_element_by_xpath(xpath))
> >>> File "/Users/dpage/git/pgadmin4/web/regression/utils/pgadmin_
> page.py",
> >>> line 72, in wait_for_element
> >>> return self._wait_for("element to exist", element_if_it_exists)
> >>> File "/Users/dpage/git/pgadmin4/web/regression/utils/pgadmin_
> page.py",
> >>> line 106, in _wait_for
> >>> raise RuntimeError("timed out waiting for " + waiting_for_message)
> >>> RuntimeError: timed out waiting for element to exist
> >>>
> >>> ======================================================================
> >>> ERROR: runTest
> >>> (pgadmin.acceptance.tests.sql_template_selection_by_
> postgres_version_works_feature_test.SQLTemplateSelectionByPostgres
> VersionWorksFeatureTest)
> >>> ----------------------------------------------------------------------
> >>> Traceback (most recent call last):
> >>> File
> >>> "/Users/dpage/git/pgadmin4/web/pgadmin/acceptance/tests/
> sql_template_selection_by_postgres_version_works_feature_test.py",
> >>> line 60, in tearDown
> >>> self.page.find_by_xpath("//button[contains(.,'Cancel')]").click()
> >>> File "/Users/dpage/git/pgadmin4/web/regression/utils/pgadmin_
> page.py",
> >>> line 45, in find_by_xpath
> >>> return self.wait_for_element(lambda:
> >>> self.driver.find_element_by_xpath(xpath))
> >>> File "/Users/dpage/git/pgadmin4/web/regression/utils/pgadmin_
> page.py",
> >>> line 72, in wait_for_element
> >>> return self._wait_for("element to exist", element_if_it_exists)
> >>> File "/Users/dpage/git/pgadmin4/web/regression/utils/pgadmin_
> page.py",
> >>> line 106, in _wait_for
> >>> raise RuntimeError("timed out waiting for " + waiting_for_message)
> >>> RuntimeError: timed out waiting for element to exist
> >>>
> >>> ----------------------------------------------------------------------
> >>>
> >>> Thanks.
> >>>
> >>> --
> >>> Dave Page
> >>> Blog: http://pgsnake.blogspot.com
> >>> Twitter: @pgsnake
> >>>
> >>> EnterpriseDB UK: http://www.enterprisedb.com
> >>> The Enterprise PostgreSQL Company
> >>
> >>
> >
> >
> >
> > --
> > Dave Page
> > Blog: http://pgsnake.blogspot.com
> > Twitter: @pgsnake
> >
> > EnterpriseDB UK: http://www.enterprisedb.com
> > The Enterprise PostgreSQL Company
>
>
>
> --
> Dave Page
> Blog: http://pgsnake.blogspot.com
> Twitter: @pgsnake
>
> EnterpriseDB UK: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>