Re: [pgAdmin4][Patch] Feature #3270 Add support for running regression tests against Firefox

Thanks Akshay. The patch looks basically OK to me, except:

- I also see the same test failures
- The option added to test_config.json should be moved to the top section
of the file, alongside the headless_chrome option.
- I dislike the the sleeps. I'd really like to find a way around those if
we can.

Joao, Anthony; can you provide any input on this?

Thanks!

On Fri, May 4, 2018 at 10:32 AM, Akshay Joshi <akshay(dot)joshi(at)enterprisedb(dot)com
> wrote:

> Ignore the previous patch. Attached is the latest one
>
> On Fri, May 4, 2018 at 2:59 PM, Akshay Joshi <
> akshay(dot)joshi(at)enterprisedb(dot)com> wrote:
>
>> Hi Hackers,
>>
>> On Fri, May 4, 2018 at 2:48 PM, Dave Page <dpage(at)pgadmin(dot)org> wrote:
>>
>>> Any progress on this Akshay?
>>>
>>
>> I have fixed Linter issues and try to fix the test cases as many as
>> possible. Apart from this I have added "default_browser" parameter in "
>> test_config.json.in" file and also added logic to provide default
>> browser from command line. To run from command line use as below:
>>
>> python runtests.py --pkg feature_tests --default_browser firefox
>> Command line argument will overwrite the settings of "test_config.json.in"
>> file.
>>
>> The showstopper issue when I run feature tests in FireFox is every time
>> "fe_sendauth" error comes on my machine when a new server has been added
>> for testing. Browser stops here and rest of the test cases will fail. To
>> fix this I have added sleep of 0.5 seconds between entering the password
>> and click on 'Save' button.
>>
>> Apart from above following changes I have made:
>>
>> - We faced lots of Timeout issues, to fixed that i have increase the
>> timeout from *0.01* seconds to *0.05 *for WebDriverWait in
>> *pgadmin_page.py.*
>> - Added time.sleep for "Tests to check if file manager is vulnerable
>> to XSS".
>>
>> As I know time.sleep is not a good idea and I am new to feature test and
>> in learning phase, can someone correct/suggest the way to fix those issues.
>>
>> Following test cases are still failing and I am not able to resolve those:
>>
>> ERROR: runTest (pgadmin.feature_tests.view_da
>> ta_dml_queries.CheckForViewDataTest)
>>
>> Validate Insert, Update operations in View/Edit data with given test data
>>
>> ----------------------------------------------------------------------
>>
>> Traceback (most recent call last):
>>
>> File "/Users/akshay/Development/pgadmin4/web/pgadmin/feature_tests/view_data_dml_queries.py",
>> line 123, in runTest
>>
>> self._add_row()
>>
>> File "/Users/akshay/Development/pgadmin4/web/pgadmin/feature_tests/view_data_dml_queries.py",
>> line 296, in _add_row
>>
>> self._update_cell(cell_xpath, config_data[str(idx)])
>>
>> File "/Users/akshay/Development/pgadmin4/web/pgadmin/feature_tests/view_data_dml_queries.py",
>> line 189, in _update_cell
>>
>> cell_el
>>
>> File "/Users/akshay/Development/Workspace_3.5/lib/python3.5/site-
>> packages/selenium/webdriver/common/action_chains.py", line 80, in perform
>>
>> self.w3c_actions.perform()
>>
>> File "/Users/akshay/Development/Workspace_3.5/lib/python3.5/site-
>> packages/selenium/webdriver/common/actions/action_builder.py", line 76,
>> in perform
>>
>> self.driver.execute(Command.W3C_ACTIONS, enc)
>>
>> File "/Users/akshay/Development/Workspace_3.5/lib/python3.5/site-
>> packages/selenium/webdriver/remote/webdriver.py", line 312, in execute
>>
>> self.error_handler.check_response(response)
>>
>> File "/Users/akshay/Development/Workspace_3.5/lib/python3.5/site-
>> packages/selenium/webdriver/remote/errorhandler.py", line 242, in
>> check_response
>>
>> raise exception_class(message, screen, stacktrace)
>>
>> selenium.common.exceptions.MoveTargetOutOfBoundsException: Message:
>> (1120.5, 289.29998779296875) is out of bounds of viewport width (1067) and
>> height (728)
>>
>>
>>
>> ======================================================================
>>
>> FAIL: runTest (pgadmin.feature_tests.copy_selected_query_results_feature_
>> test.CopySelectedQueryResultsFeatureTest)
>>
>> Copy rows, column using button and keyboard shortcut
>>
>> ----------------------------------------------------------------------
>>
>> Traceback (most recent call last):
>>
>> File "/Users/akshay/Development/pgadmin4/web/pgadmin/feature_test
>> s/copy_selected_query_results_feature_test.py", line 64, in runTest
>>
>> self._mouseup_outside_grid_still_makes_a_selection()
>>
>> File "/Users/akshay/Development/pgadmin4/web/pgadmin/feature_test
>> s/copy_selected_query_results_feature_test.py", line 203, in
>> _mouseup_outside_grid_still_makes_a_selection
>>
>> self.assertEqual('"cool info"', pyperclip.paste())
>>
>> AssertionError: '"cool info"' != '"some info"\n"some other info"\n"cool
>> info"'
>>
>> - "cool info"
>>
>> + "some info"
>>
>> "some other info"
>>
>> "cool info"
>>
>> Attached is the modified patch, please review and let me know the review
>> comments.
>>
>>
>>>
>>> On Tue, Apr 24, 2018 at 3:34 PM, Joao De Almeida Pereira <
>>> jdealmeidapereira(at)pivotal(dot)io> wrote:
>>>
>>>> Hi Akshay,
>>>>
>>>> Linter fails and tests are failing when we run the patch on our machine
>>>> using Firefox. The linter output is:
>>>>
>>>> ./regression/runtests.py:184: [E125] continuation line with same indent as next logical line
>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-tests/builds/25#L5ad0f359:255>
>>>> ./regression/runtests.py:184: [E501] line too long (80 > 79 characters)
>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-tests/builds/25#L5ad0f359:256>
>>>> 1 E125 continuation line with same indent as next logical line
>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-tests/builds/25#L5ad0f359:257>
>>>> 1 E501 line too long (80 > 79 characters)
>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-tests/builds/25#L5ad0f359:258>
>>>> 2
>>>>
>>>>
>>>> The tests output is:
>>>>
>>>> =============Running the test cases for 'PostgreSQL 10'=============
>>>> runTest (pgadmin.feature_tests.copy_selected_query_results_feature_test.CopySelectedQueryResultsFeatureTest)
>>>> Copy rows, column using button and keyboard shortcut ... ERROR
>>>> runTest (pgadmin.feature_tests.keyboard_shortcut_test.KeyboardShortcutFeatureTest)
>>>> Test for keyboard shortcut ... Executing shortcut: File main menu...OK
>>>> Executing shortcut: Object main menu...OK
>>>> ok
>>>> runTest (pgadmin.feature_tests.pg_datatype_validation_test.PGDataypeFeatureTest)
>>>> Test checks for PG data-types output ... ERROR
>>>> runTest (pgadmin.feature_tests.query_tool_journey_test.QueryToolJourneyTest)
>>>> Tests the path through the query tool ... ERROR
>>>> runTest (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest)
>>>> Query tool feature test ...
>>>> On demand query result...
>>>> On demand result set on scrolling... OK.
>>>> On demand result set on grid select all... OK.
>>>> On demand result set on column select all... OK.
>>>> Explain query with verbose and cost... ERROR
>>>> runTest (pgadmin.feature_tests.table_ddl_feature_test.TableDdlFeatureTest)
>>>> Test table DDL generation ... ok
>>>> runTest (pgadmin.feature_tests.view_data_dml_queries.CheckForViewDataTest)
>>>> Validate Insert, Update operations in View/Edit data with given test data ... ERROR
>>>> ERROR
>>>> runTest (pgadmin.feature_tests.xss_checks_file_manager_test.CheckFileManagerFeatureTest)
>>>> Tests to check if File manager is vulnerable to XSS ... ERROR
>>>> runTest (pgadmin.feature_tests.xss_checks_panels_and_query_tool_test.CheckForXssFeatureTest)
>>>> Test XSS check for panels and query tool ... ERROR
>>>> ERROR
>>>> runTest (pgadmin.feature_tests.xss_checks_pgadmin_debugger_test.CheckDebuggerForXssFeatureTest)
>>>> Tests to check if Debugger is vulnerable to XSS ... ERROR
>>>> ERROR
>>>> runTest (pgadmin.feature_tests.xss_checks_roles_control_test.CheckRoleMembershipControlFeatureTest)
>>>> Tests to check if Role membership control is vulnerable to XSS^[ ... Traceback (most recent call last):
>>>> File "/home/pivotal/workspace/pgadmin4/web/regression/python_test_utils/test_utils.py", line 338, in create_role
>>>> sql_query
>>>> psycopg2.ProgrammingError: role "test_role" already exists
>>>>
>>>> Traceback (most recent call last):
>>>> File "/home/pivotal/workspace/pgadmin4/web/regression/python_test_utils/test_utils.py", line 338, in create_role
>>>> sql_query
>>>> psycopg2.ProgrammingError: role "<h1>test</h1>" already exists
>>>>
>>>> ERROR
>>>> ERROR
>>>>
>>>> ======================================================================
>>>> ERROR: runTest (pgadmin.feature_tests.copy_selected_query_results_feature_test.CopySelectedQueryResultsFeatureTest)
>>>> Copy rows, column using button and keyboard shortcut
>>>> ----------------------------------------------------------------------
>>>> Traceback (most recent call last):
>>>> File "/home/pivotal/workspace/pgadmin4/web/pgadmin/feature_tests/copy_selected_query_results_feature_test.py", line 57, in runTest
>>>> self._copies_rows()
>>>> File "/home/pivotal/workspace/pgadmin4/web/pgadmin/feature_tests/copy_selected_query_results_feature_test.py", line 67, in _copies_rows
>>>> pyperclip.copy("old clipboard contents")
>>>> File "/home/pivotal/.pyenv/versions/python36/lib/python3.6/site-packages/pyperclip/__init__.py", line 574, in lazy_load_stub_copy
>>>> return copy(text)
>>>> File "/home/pivotal/.pyenv/versions/python36/lib/python3.6/site-packages/pyperclip/__init__.py", line 284, in __call__
>>>> raise PyperclipException(EXCEPT_MSG)
>>>> pyperclip.PyperclipException:
>>>> Pyperclip could not find a copy/paste mechanism for your system.
>>>> For more information, please visit https://pyperclip.readthedocs.io/en/latest/introduction.html#not-implemented-error
>>>>
>>>> ======================================================================
>>>> ERROR: runTest (pgadmin.feature_tests.pg_datatype_validation_test.PGDataypeFeatureTest)
>>>> Test checks for PG data-types output
>>>> ----------------------------------------------------------------------
>>>> Traceback (most recent call last):
>>>> File "/home/pivotal/workspace/pgadmin4/web/pgadmin/feature_tests/pg_datatype_validation_test.py", line 135, in runTest
>>>> self._check_datatype()
>>>> File "/home/pivotal/workspace/pgadmin4/web/pgadmin/feature_tests/pg_datatype_validation_test.py", line 169, in _check_datatype
>>>> "contains(.,'{}')]".format(batch['datatype'][0])
>>>> File "/home/pivotal/.pyenv/versions/python36/lib/python3.6/site-packages/selenium/webdriver/support/wait.py", line 80, in until
>>>> raise TimeoutException(message, screen, stacktrace)
>>>> selenium.common.exceptions.TimeoutException: Message:
>>>>
>>>> ======================================================================
>>>> ERROR: runTest (pgadmin.feature_tests.query_tool_journey_test.QueryToolJourneyTest)
>>>> Tests the path through the query tool
>>>> ----------------------------------------------------------------------
>>>> Traceback (most recent call last):
>>>> File "/home/pivotal/workspace/pgadmin4/web/pgadmin/feature_tests/query_tool_journey_test.py", line 49, in runTest
>>>> self._test_copies_rows()
>>>> File "/home/pivotal/workspace/pgadmin4/web/pgadmin/feature_tests/query_tool_journey_test.py", line 54, in _test_copies_rows
>>>> pyperclip.copy("old clipboard contents")
>>>> File "/home/pivotal/.pyenv/versions/python36/lib/python3.6/site-packages/pyperclip/__init__.py", line 284, in __call__
>>>> raise PyperclipException(EXCEPT_MSG)
>>>> pyperclip.PyperclipException:
>>>> Pyperclip could not find a copy/paste mechanism for your system.
>>>> For more information, please visit https://pyperclip.readthedocs.io/en/latest/introduction.html#not-implemented-error
>>>>
>>>> ======================================================================
>>>> ERROR: runTest (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest)
>>>> Query tool feature test
>>>> ----------------------------------------------------------------------
>>>> Traceback (most recent call last):
>>>> File "/home/pivotal/workspace/pgadmin4/web/pgadmin/feature_tests/query_tool_tests.py", line 59, in runTest
>>>> self._query_tool_explain_with_verbose_and_cost()
>>>> File "/home/pivotal/workspace/pgadmin4/web/pgadmin/feature_tests/query_tool_tests.py", line 262, in _query_tool_explain_with_verbose_and_cost
>>>> self.page.find_by_id("btn-explain-costs").click()
>>>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 174, in find_by_id
>>>> lambda driver: driver.find_element_by_id(element_id)
>>>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 261, in wait_for_element
>>>> return self._wait_for("element to exist", element_if_it_exists)
>>>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 327, in _wait_for
>>>> "Timed out waiting for " + waiting_for_message
>>>> File "/home/pivotal/.pyenv/versions/python36/lib/python3.6/site-packages/selenium/webdriver/support/wait.py", line 80, in until
>>>> raise TimeoutException(message, screen, stacktrace)
>>>> selenium.common.exceptions.TimeoutException: Message: Timed out waiting for element to exist
>>>>
>>>> ======================================================================
>>>> ERROR: runTest (pgadmin.feature_tests.view_data_dml_queries.CheckForViewDataTest)
>>>> Validate Insert, Update operations in View/Edit data with given test data
>>>> ----------------------------------------------------------------------
>>>> Traceback (most recent call last):
>>>> File "/home/pivotal/workspace/pgadmin4/web/pgadmin/feature_tests/view_data_dml_queries.py", line 114, in runTest
>>>> self.page.add_server(self.server)
>>>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 55, in add_server
>>>> self.find_by_partial_link_text("Server...").click()
>>>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 185, in find_by_partial_link_text
>>>> EC.element_to_be_clickable((By.PARTIAL_LINK_TEXT, link_text))
>>>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 327, in _wait_for
>>>> "Timed out waiting for " + waiting_for_message
>>>> File "/home/pivotal/.pyenv/versions/python36/lib/python3.6/site-packages/selenium/webdriver/support/wait.py", line 80, in until
>>>> raise TimeoutException(message, screen, stacktrace)
>>>> selenium.common.exceptions.TimeoutException: Message: Timed out waiting for link with text "Server..."
>>>>
>>>> ======================================================================
>>>> ERROR: runTest (pgadmin.feature_tests.view_data_dml_queries.CheckForViewDataTest)
>>>> Validate Insert, Update operations in View/Edit data with given test data
>>>> ----------------------------------------------------------------------
>>>> Traceback (most recent call last):
>>>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/base_feature_test.py", line 66, in tearDown
>>>> self.after()
>>>> File "/home/pivotal/workspace/pgadmin4/web/pgadmin/feature_tests/view_data_dml_queries.py", line 132, in after
>>>> self.page.remove_server(self.server)
>>>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 122, in remove_server
>>>> "' and @class='aciTreeItem']")
>>>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 169, in find_by_xpath
>>>> lambda driver: driver.find_element_by_xpath(xpath)
>>>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 261, in wait_for_element
>>>> return self._wait_for("element to exist", element_if_it_exists)
>>>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 327, in _wait_for
>>>> "Timed out waiting for " + waiting_for_message
>>>> File "/home/pivotal/.pyenv/versions/python36/lib/python3.6/site-packages/selenium/webdriver/support/wait.py", line 80, in until
>>>> raise TimeoutException(message, screen, stacktrace)
>>>> selenium.common.exceptions.TimeoutException: Message: Timed out waiting for element to exist
>>>>
>>>> ======================================================================
>>>> ERROR: runTest (pgadmin.feature_tests.xss_checks_file_manager_test.CheckFileManagerFeatureTest)
>>>> Tests to check if File manager is vulnerable to XSS
>>>> ----------------------------------------------------------------------
>>>> Traceback (most recent call last):
>>>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/base_feature_test.py", line 43, in setUp
>>>> self.before()
>>>> File "/home/pivotal/workspace/pgadmin4/web/pgadmin/feature_tests/xss_checks_file_manager_test.py", line 37, in before
>>>> self.page.add_server(self.server)
>>>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 55, in add_server
>>>> self.find_by_partial_link_text("Server...").click()
>>>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 185, in find_by_partial_link_text
>>>> EC.element_to_be_clickable((By.PARTIAL_LINK_TEXT, link_text))
>>>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 327, in _wait_for
>>>> "Timed out waiting for " + waiting_for_message
>>>> File "/home/pivotal/.pyenv/versions/python36/lib/python3.6/site-packages/selenium/webdriver/support/wait.py", line 80, in until
>>>> raise TimeoutException(message, screen, stacktrace)
>>>> selenium.common.exceptions.TimeoutException: Message: Timed out waiting for link with text "Server..."
>>>>
>>>> ======================================================================
>>>> ERROR: runTest (pgadmin.feature_tests.xss_checks_panels_and_query_tool_test.CheckForXssFeatureTest)
>>>> Test XSS check for panels and query tool
>>>> ----------------------------------------------------------------------
>>>> Traceback (most recent call last):
>>>> File "/home/pivotal/workspace/pgadmin4/web/pgadmin/feature_tests/xss_checks_panels_and_query_tool_test.py", line 57, in runTest
>>>> self.page.add_server(self.server)
>>>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 55, in add_server
>>>> self.find_by_partial_link_text("Server...").click()
>>>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 185, in find_by_partial_link_text
>>>> EC.element_to_be_clickable((By.PARTIAL_LINK_TEXT, link_text))
>>>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 327, in _wait_for
>>>> "Timed out waiting for " + waiting_for_message
>>>> File "/home/pivotal/.pyenv/versions/python36/lib/python3.6/site-packages/selenium/webdriver/support/wait.py", line 80, in until
>>>> raise TimeoutException(message, screen, stacktrace)
>>>> selenium.common.exceptions.TimeoutException: Message: Timed out waiting for link with text "Server..."
>>>>
>>>> ======================================================================
>>>> ERROR: runTest (pgadmin.feature_tests.xss_checks_panels_and_query_tool_test.CheckForXssFeatureTest)
>>>> Test XSS check for panels and query tool
>>>> ----------------------------------------------------------------------
>>>> Traceback (most recent call last):
>>>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/base_feature_test.py", line 66, in tearDown
>>>> self.after()
>>>> File "/home/pivotal/workspace/pgadmin4/web/pgadmin/feature_tests/xss_checks_panels_and_query_tool_test.py", line 69, in after
>>>> self.page.remove_server(self.server)
>>>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 122, in remove_server
>>>> "' and @class='aciTreeItem']")
>>>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 169, in find_by_xpath
>>>> lambda driver: driver.find_element_by_xpath(xpath)
>>>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 261, in wait_for_element
>>>> return self._wait_for("element to exist", element_if_it_exists)
>>>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 327, in _wait_for
>>>> "Timed out waiting for " + waiting_for_message
>>>> File "/home/pivotal/.pyenv/versions/python36/lib/python3.6/site-packages/selenium/webdriver/support/wait.py", line 80, in until
>>>> raise TimeoutException(message, screen, stacktrace)
>>>> selenium.common.exceptions.TimeoutException: Message: Timed out waiting for element to exist
>>>>
>>>> ======================================================================
>>>> ERROR: runTest (pgadmin.feature_tests.xss_checks_pgadmin_debugger_test.CheckDebuggerForXssFeatureTest)
>>>> Tests to check if Debugger is vulnerable to XSS
>>>> ----------------------------------------------------------------------
>>>> Traceback (most recent call last):
>>>> File "/home/pivotal/workspace/pgadmin4/web/pgadmin/feature_tests/xss_checks_pgadmin_debugger_test.py", line 41, in runTest
>>>> self.page.add_server(self.server)
>>>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 55, in add_server
>>>> self.find_by_partial_link_text("Server...").click()
>>>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 185, in find_by_partial_link_text
>>>> EC.element_to_be_clickable((By.PARTIAL_LINK_TEXT, link_text))
>>>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 327, in _wait_for
>>>> "Timed out waiting for " + waiting_for_message
>>>> File "/home/pivotal/.pyenv/versions/python36/lib/python3.6/site-packages/selenium/webdriver/support/wait.py", line 80, in until
>>>> raise TimeoutException(message, screen, stacktrace)
>>>> selenium.common.exceptions.TimeoutException: Message: Timed out waiting for link with text "Server..."
>>>>
>>>> ======================================================================
>>>> ERROR: runTest (pgadmin.feature_tests.xss_checks_pgadmin_debugger_test.CheckDebuggerForXssFeatureTest)
>>>> Tests to check if Debugger is vulnerable to XSS
>>>> ----------------------------------------------------------------------
>>>> Traceback (most recent call last):
>>>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/base_feature_test.py", line 66, in tearDown
>>>> self.after()
>>>> File "/home/pivotal/workspace/pgadmin4/web/pgadmin/feature_tests/xss_checks_pgadmin_debugger_test.py", line 46, in after
>>>> self.page.remove_server(self.server)
>>>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 122, in remove_server
>>>> "' and @class='aciTreeItem']")
>>>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 169, in find_by_xpath
>>>> lambda driver: driver.find_element_by_xpath(xpath)
>>>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 261, in wait_for_element
>>>> return self._wait_for("element to exist", element_if_it_exists)
>>>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 327, in _wait_for
>>>> "Timed out waiting for " + waiting_for_message
>>>> File "/home/pivotal/.pyenv/versions/python36/lib/python3.6/site-packages/selenium/webdriver/support/wait.py", line 80, in until
>>>> raise TimeoutException(message, screen, stacktrace)
>>>> selenium.common.exceptions.TimeoutException: Message: Timed out waiting for element to exist
>>>>
>>>> ======================================================================
>>>> ERROR: runTest (pgadmin.feature_tests.xss_checks_roles_control_test.CheckRoleMembershipControlFeatureTest)
>>>> Tests to check if Role membership control is vulnerable to XSS
>>>> ----------------------------------------------------------------------
>>>> Traceback (most recent call last):
>>>> File "/home/pivotal/workspace/pgadmin4/web/pgadmin/feature_tests/xss_checks_roles_control_test.py", line 37, in runTest
>>>> self.page.add_server(self.server)
>>>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 55, in add_server
>>>> self.find_by_partial_link_text("Server...").click()
>>>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 185, in find_by_partial_link_text
>>>> EC.element_to_be_clickable((By.PARTIAL_LINK_TEXT, link_text))
>>>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 327, in _wait_for
>>>> "Timed out waiting for " + waiting_for_message
>>>> File "/home/pivotal/.pyenv/versions/python36/lib/python3.6/site-packages/selenium/webdriver/support/wait.py", line 80, in until
>>>> raise TimeoutException(message, screen, stacktrace)
>>>> selenium.common.exceptions.TimeoutException: Message: Timed out waiting for link with text "Server..."
>>>>
>>>> ======================================================================
>>>> ERROR: runTest (pgadmin.feature_tests.xss_checks_roles_control_test.CheckRoleMembershipControlFeatureTest)
>>>> Tests to check if Role membership control is vulnerable to XSS
>>>> ----------------------------------------------------------------------
>>>> Traceback (most recent call last):
>>>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/base_feature_test.py", line 66, in tearDown
>>>> self.after()
>>>> File "/home/pivotal/workspace/pgadmin4/web/pgadmin/feature_tests/xss_checks_roles_control_test.py", line 42, in after
>>>> self.page.remove_server(self.server)
>>>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 122, in remove_server
>>>> "' and @class='aciTreeItem']")
>>>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 169, in find_by_xpath
>>>> lambda driver: driver.find_element_by_xpath(xpath)
>>>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 261, in wait_for_element
>>>> return self._wait_for("element to exist", element_if_it_exists)
>>>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 327, in _wait_for
>>>> "Timed out waiting for " + waiting_for_message
>>>> File "/home/pivotal/.pyenv/versions/python36/lib/python3.6/site-packages/selenium/webdriver/support/wait.py", line 80, in until
>>>> raise TimeoutException(message, screen, stacktrace)
>>>> selenium.common.exceptions.TimeoutException: Message: Timed out waiting for element to exist
>>>>
>>>> ----------------------------------------------------------------------
>>>> Ran 11 tests in 441.881s
>>>>
>>>> FAILED (errors=13)
>>>>
>>>> ======================================================================
>>>> Test Result Summary
>>>> ======================================================================
>>>>
>>>> PostgreSQL 10:
>>>>
>>>> 2 tests passed
>>>> 9 tests failed:
>>>> CopySelectedQueryResultsFeatureTest (Copy rows, column using button and keyboard shortcut)
>>>> PGDataypeFeatureTest (Test checks for PG data-types output)
>>>> QueryToolJourneyTest (Tests the path through the query tool)
>>>> QueryToolFeatureTest (Query tool feature test)
>>>> CheckForViewDataTest (Validate Insert, Update operations in View/Edit data with given test data)
>>>> CheckFileManagerFeatureTest (Tests to check if File manager is vulnerable to XSS)
>>>> CheckForXssFeatureTest (Test XSS check for panels and query tool)
>>>> CheckDebuggerForXssFeatureTest (Tests to check if Debugger is vulnerable to XSS)
>>>> CheckRoleMembershipControlFeatureTest (Tests to check if Role membership control is vulnerable to XSS)
>>>> 0 tests skipped
>>>>
>>>> ======================================================================
>>>>
>>>> ​
>>>>
>>>> All the above tests fail in the following place:
>>>> [image:
>>>> CheckDebuggerForXssFeatureTest-2018.04.24_10.28.27-Python-3.6.4.png]
>>>>
>>>>
>>>> Also the tests will never end because firefox is waiting for the user
>>>> to click "Leave the page" button.
>>>>
>>>> Thanks
>>>> Victoria & Joao
>>>>
>>>> On Tue, Apr 24, 2018 at 2:26 AM Akshay Joshi <
>>>> akshay(dot)joshi(at)enterprisedb(dot)com> wrote:
>>>>
>>>>> Hi Hackers,
>>>>>
>>>>> On Mon, Apr 23, 2018 at 9:07 PM, Anthony Emengo <aemengo(at)pivotal(dot)io>
>>>>> wrote:
>>>>>
>>>>>> We also tried running the tests with this patch. It didn't launch
>>>>>> without some code changes and several tests were failing. We should really
>>>>>> defer pulling this in until we have more robust results on Firefox
>>>>>>
>>>>>> In order to have the tests running we had to do the following change:
>>>>>>
>>>>>> diff --git a/web/regression/feature_utils/app_starter.py
>>>>>> b/web/regression/feature_utils/app_starter.py
>>>>>> index 77b0400c..50d3e307 100644
>>>>>> --- a/web/regression/feature_utils/app_starter.py
>>>>>> +++ b/web/regression/feature_utils/app_starter.py
>>>>>> @@ -42,10 +44,18 @@ class AppStarter:
>>>>>> )
>>>>>>
>>>>>> self.driver.set_window_size(1280, 1024)
>>>>>> - self.driver.get(
>>>>>> - "http://" + self.app_config.DEFAULT_SERVER + ":" +
>>>>>> - random_server_port
>>>>>> - )
>>>>>> + # self.driver.implicitly_wait(60)
>>>>>> +
>>>>>> + def launch_browser():
>>>>>> + try:
>>>>>> + self.driver.get(
>>>>>> + "http://" + self.app_config.DEFAULT_SERVER +
>>>>>> ":" +
>>>>>> + random_server_port
>>>>>> + )
>>>>>> + except WebDriverException as e:
>>>>>> + time.sleep(5)
>>>>>> + launch_browser()
>>>>>> + launch_browser()
>>>>>>
>>>>>> This change was required because firefox was throwing an exception
>>>>>> when we tried to get the address and the server was not running. We saw
>>>>>> this behavior in Ubuntu.
>>>>>>
>>>>>
>>>>> Yes I have faced the same problem in Ubuntu. As per suggestion by
>>>>> Dave I have added parameter in test_config.json.in and also include
>>>>> the above code given by Anthony.
>>>>> Attached is the modified patch.
>>>>>
>>>>>>
>>>>>>
>>>>>> - Anthony and Joao
>>>>>>
>>>>>> On Mon, Apr 23, 2018 at 9:12 AM, Dave Page <dpage(at)pgadmin(dot)org> wrote:
>>>>>>
>>>>>>> Hi
>>>>>>>
>>>>>>> On Mon, Apr 23, 2018 at 2:05 PM, Akshay Joshi <
>>>>>>> akshay(dot)joshi(at)enterprisedb(dot)com> wrote:
>>>>>>>
>>>>>>>> Hi Hackers,
>>>>>>>>
>>>>>>>> I have added support for running feature tests against FireFox. For
>>>>>>>> that user will have to download gecko driver from https://github
>>>>>>>> .com/mozilla/geckodriver/releases and follow the below steps:
>>>>>>>>
>>>>>>>> - Extract the gecko driver.
>>>>>>>> - Run chmod +x geckodriver.
>>>>>>>> - Either copy the geckodriver to /usr/local/bin or the path of
>>>>>>>> the geckodriver must be specified in PATH.
>>>>>>>> - Apply the attached patch.
>>>>>>>> - Change the parameter "DEFAULT_TEST_BROWSER = 'Firefox' "
>>>>>>>> - Start the feature test.
>>>>>>>>
>>>>>>>> The config option needs to be in test_config.json. We don't really
>>>>>>> want test-specific config options in config.py (arguably, TEST_SQLITE_PATH
>>>>>>> shoudn't be there either, as it's useless for end users).
>>>>>>>
>>>>>>> I'm surprised to not see any updates to tests. When I tried firefox
>>>>>>> manually a few weeks back, it was failing at lot.
>>>>>>>
>>>>>>> --
>>>>>>> Dave Page
>>>>>>> Blog: http://pgsnake.blogspot.com
>>>>>>> Twitter: @pgsnake
>>>>>>>
>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com
>>>>>>> The Enterprise PostgreSQL Company
>>>>>>>
>>>>>>
>>>>>>
>>>>> --
>>>>> *Akshay Joshi*
>>>>>
>>>>> *Sr. Software Architect *
>>>>>
>>>>>
>>>>>
>>>>> *Phone: +91 20-3058-9517 <+91%2020%203058%209517>Mobile: +91
>>>>> 976-788-8246 <+91%2097678%2088246>*
>>>>>
>>>>
>>>
>>>
>>> --
>>> Dave Page
>>> Blog: http://pgsnake.blogspot.com
>>> Twitter: @pgsnake
>>>
>>> EnterpriseDB UK: http://www.enterprisedb.com
>>> The Enterprise PostgreSQL Company
>>>
>>
>>
>>
>> --
>> *Akshay Joshi*
>>
>> *Sr. Software Architect *
>>
>>
>>
>> *Phone: +91 20-3058-9517Mobile: +91 976-788-8246*
>>
>
>
>
> --
> *Akshay Joshi*
>
> *Sr. Software Architect *
>
>
>
> *Phone: +91 20-3058-9517Mobile: +91 976-788-8246*
>

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company