Re: pgAdmin4: Test result enhancement patch

Hi Navnath

I have run the updated patch. It is working fine with Python 2.7 but I am
facing following error with Python 3.5, can you please look into it:

======================================================================

ERROR: runTest
(pgadmin.feature_tests.connect_to_server_feature_test.ConnectsToServerFeatureTest)

Test database connection which can be created from the UI

----------------------------------------------------------------------

Traceback (most recent call last):

File
"/Users/akshay/Development/pgadmin4/web/regression/feature_utils/base_feature_test.py",
line 33, in setUp

self.page.reset_layout()

File
"/Users/akshay/Development/pgadmin4/web/regression/feature_utils/pgadmin_page.py",
line 33, in reset_layout

self.click_modal_ok()

File
"/Users/akshay/Development/pgadmin4/web/regression/feature_utils/pgadmin_page.py",
line 38, in click_modal_ok

self.click_element(self.find_by_xpath("//button[contains(.,'OK')]"))

File
"/Users/akshay/Development/pgadmin4/web/regression/feature_utils/pgadmin_page.py",
line 71, in find_by_xpath

return self.wait_for_element(lambda driver:
driver.find_element_by_xpath(xpath))

File
"/Users/akshay/Development/pgadmin4/web/regression/feature_utils/pgadmin_page.py",
line 128, in wait_for_element

return self._wait_for("element to exist", element_if_it_exists)

File
"/Users/akshay/Development/pgadmin4/web/regression/feature_utils/pgadmin_page.py",
line 162, in _wait_for

"Timed out waiting for " + waiting_for_message)

File
"/Users/akshay/Development/Workspace/lib/python3.5/site-packages/selenium/webdriver/support/wait.py",
line 80, in until

raise TimeoutException(message, screen, stacktrace)

selenium.common.exceptions.TimeoutException: Message: Timed out waiting for
element to exist

======================================================================

ERROR: runTest
(pgadmin.feature_tests.table_ddl_feature_test.TableDdlFeatureTest)

Test scenarios for acceptance tests

----------------------------------------------------------------------

Traceback (most recent call last):

File
"/Users/akshay/Development/pgadmin4/web/regression/feature_utils/base_feature_test.py",
line 33, in setUp

self.page.reset_layout()

File
"/Users/akshay/Development/pgadmin4/web/regression/feature_utils/pgadmin_page.py",
line 31, in reset_layout

self.click_element(self.find_by_partial_link_text("File"))

File
"/Users/akshay/Development/pgadmin4/web/regression/feature_utils/pgadmin_page.py",
line 90, in click_element

return self._wait_for("clicking the element not to throw an exception",
click_succeeded)

File
"/Users/akshay/Development/pgadmin4/web/regression/feature_utils/pgadmin_page.py",
line 162, in _wait_for

"Timed out waiting for " + waiting_for_message)

File
"/Users/akshay/Development/Workspace/lib/python3.5/site-packages/selenium/webdriver/support/wait.py",
line 80, in until

raise TimeoutException(message, screen, stacktrace)

selenium.common.exceptions.TimeoutException: Message: Timed out waiting for
clicking the element not to throw an exception

----------------------------------------------------------------------

Ran 153 tests in 45.493s

FAILED (errors=2, skipped=16)

======================================================================

Test Result Summary

======================================================================

Traceback (most recent call last):

File "runtests.py", line 354, in <module>

skipped_cases)

File
"/Users/akshay/Development/pgadmin4/web/regression/python_test_utils/test_utils.py",
line 442, in get_scenario_name

key, value = case_name_dict.items()[0]

TypeError: 'dict_items' object does not support indexing

On Thu, Mar 30, 2017 at 8:04 PM, Navnath Gadakh <
navnath(dot)gadakh(at)enterprisedb(dot)com> wrote:

> Hi Dave,
>
> Please find the revised patch for test result enhancement.
>
> *What's in the patch:*
> 1. The test result summary will store in JSON file.
> 2. Removed some redundant code from *regression/test_utils.py*
> *3. A*dded the scenario names for feature tests.
> 4. To print test scenario names in failed and skipped test cases, I o
> verride *apply_scenario()* function in *regression/test_utils.py*
>
> I have also attached the sample JSON file with the test result as per your
> suggestions.
>
> Thanks!
>
>
>
> On Wed, Mar 29, 2017 at 6:03 PM, Dave Page <dpage(at)pgadmin(dot)org> wrote:
>
>> On Wed, Mar 29, 2017 at 4:12 AM, Navnath Gadakh
>> <navnath(dot)gadakh(at)enterprisedb(dot)com> wrote:
>> > Hi,
>> >
>> > On Mon, Mar 27, 2017 at 5:37 PM, Dave Page <dpage(at)pgadmin(dot)org> wrote:
>> >>
>> >> Hi
>> >>
>> >> On Mon, Mar 27, 2017 at 12:15 AM, Navnath Gadakh
>> >> <navnath(dot)gadakh(at)enterprisedb(dot)com> wrote:
>> >> > Hello Dave,
>> >> >
>> >> > On Fri, Mar 24, 2017 at 9:10 PM, Dave Page <dpage(at)pgadmin(dot)org>
>> wrote:
>> >> >>
>> >> >> Hi
>> >> >>
>> >> >> On Fri, Mar 24, 2017 at 3:13 PM, Navnath Gadakh
>> >> >> <navnath(dot)gadakh(at)enterprisedb(dot)com> wrote:
>> >> >> >
>> >> >> >> When running with the patch:
>> >> >> >>
>> >> >> >> 1) The browser isn't closed, and the script never exits - it just
>> >> >> >> sits
>> >> >> >> indefinitely at:
>> >> >> >>
>> >> >> >> =====
>> >> >> >> Please check output in file:
>> >> >> >> /Users/dpage/git/pgadmin4/web/regression/regression.log
>> >> >> >>
>> >> >> >> make: *** [check] Error 1
>> >> >> >> =====
>> >> >> >>
>> >> >> >> without returning to a shell prompt. The browser exits when I hit
>> >> >> >> Ctrl+C.
>> >> >>
>> >> >> The above is still a problem. In fact, not only do I have to hit
>> >> >> Ctrl+C, but then the browser prompts me to check I really do want to
>> >> >> exit.
>> >> >>
>> >> >> There's also another problem that just showed up. I got the
>> following
>> >> >> failure on PG 9.4 (due to a known intermittent bug that Ashesh and
>> >> >> Tira(at)Pivotal are working on). Note how it's not reported in the
>> >> >> summary (or the JSON output):
>> >> >
>> >> >
>> >> > I found the issue, In the feature tests we need to add a scenario
>> name
>> >> > for
>> >> > each test case. the purpose of this patch is to print the
>> failed/skipped
>> >> > test class name with the scenario name like:
>> >> >
>> >> > 152 tests passed
>> >> >
>> >> > 1 test failed:
>> >> >
>> >> > LoginRoleGetTestCase (Check Role Node)
>> >> >
>> >> > 16 tests skipped:
>> >> >
>> >> > SynonymGetTestCase (Fetch synonym Node URL)
>> >> >
>> >> > But our in-built test framework does not provide that scenario name
>> with
>> >> > failed/skipped test case that's why I override apply_scenario()
>> >> > function.
>> >> >
>> >> > def apply_scenario(scenario, test):
>> >> >
>> >> > name, parameters = scenario
>> >> >
>> >> > parameters["scenario_name"] = name
>> >> >
>> >> > While printing the result, I have checked the if 'scenario_name' in
>> test
>> >> > as
>> >> > we need to print scenario name in test summary as well as in JSON
>> file.
>> >> >
>> >> > I can do it without scenario name but for better understanding which
>> >> > test
>> >> > scenario is failed it's good to add a scenario name with each test
>> case.
>> >>
>> >> OK.
>> >>
>> >> > See this is how test cases looks like while printing on console
>> >> >
>> >> > API:
>> >> >
>> >> > runTest
>> >> >
>> >> > (pgadmin.browser.server_groups.servers.databases.schemas.
>> types.tests.test_types_put.TypesUpdateTestCase)
>> >> >
>> >> > Update type under schema node ... ok
>> >> >
>> >> > Feature tests:
>> >> >
>> >> > runTest
>> >> >
>> >> > (pgadmin.utils.tests.test_versioned_template_loader.TestVers
>> ionedTemplateLoader)
>> >> > ... ok
>> >> >
>> >> > No scenario name in feature tests.
>> >> >
>> >>
>> >> OK, is that easy to fix while you're at it?
>> >
>> >
>> > I have two solutions-
>> >
>> > 1. Need a little hack to skip scenario/test name if that does not
>> exist, but
>> > that's not the best idea.
>> >
>> > 2. Owner of feature tests should add scenario/test name to each feature
>> > test. In the summary also we will know for which scenario test is
>> failing or
>> > skipping.
>> > This is ideal and long term solution and I prefer it.
>>
>> Agreed - and as there are only 2 feature tests, you should be able to
>> fix them up pretty quickly :-p
>>
>> Once code is in the repo, it's "ours", meaning the entire communities.
>> I wouldn't expect us to ping all issues back to Pivotal - we're one
>> team on this.
>>
>> Thanks!
>>
>> --
>> Dave Page
>> Blog: http://pgsnake.blogspot.com
>> Twitter: @pgsnake
>>
>> EnterpriseDB UK: http://www.enterprisedb.com
>> The Enterprise PostgreSQL Company
>>
>
>
>
> --
> Regards,
> Navnath Gadakh
>
> EnterpriseDB Corporation
> The Enterprise PostgreSQL Company
>
>
>
>
> --
> Sent via pgadmin-hackers mailing list (pgadmin-hackers(at)postgresql(dot)org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgadmin-hackers
>
>

--
*Akshay Joshi*
*Principal Software Engineer *

*Phone: +91 20-3058-9517Mobile: +91 976-788-8246*