Re: [pgAdmin4][Patch] Feature #3270 Add support for running regression tests against Firefox

Hi Hackers,

On Fri, May 4, 2018 at 2:48 PM, Dave Page <dpage(at)pgadmin(dot)org> wrote:

> Any progress on this Akshay?
>

I have fixed Linter issues and try to fix the test cases as many as
possible. Apart from this I have added "default_browser" parameter in "
test_config.json.in" file and also added logic to provide default browser
from command line. To run from command line use as below:

python runtests.py --pkg feature_tests --default_browser firefox
Command line argument will overwrite the settings of "test_config.json.in"
file.

The showstopper issue when I run feature tests in FireFox is every time
"fe_sendauth" error comes on my machine when a new server has been added
for testing. Browser stops here and rest of the test cases will fail. To
fix this I have added sleep of 0.5 seconds between entering the password
and click on 'Save' button.

Apart from above following changes I have made:

- We faced lots of Timeout issues, to fixed that i have increase the
timeout from *0.01* seconds to *0.05 *for WebDriverWait in
*pgadmin_page.py.*
- Added time.sleep for "Tests to check if file manager is vulnerable to
XSS".

As I know time.sleep is not a good idea and I am new to feature test and in
learning phase, can someone correct/suggest the way to fix those issues.

Following test cases are still failing and I am not able to resolve those:

ERROR: runTest
(pgadmin.feature_tests.view_data_dml_queries.CheckForViewDataTest)

Validate Insert, Update operations in View/Edit data with given test data

----------------------------------------------------------------------

Traceback (most recent call last):

File
"/Users/akshay/Development/pgadmin4/web/pgadmin/feature_tests/view_data_dml_queries.py",
line 123, in runTest

self._add_row()

File
"/Users/akshay/Development/pgadmin4/web/pgadmin/feature_tests/view_data_dml_queries.py",
line 296, in _add_row

self._update_cell(cell_xpath, config_data[str(idx)])

File
"/Users/akshay/Development/pgadmin4/web/pgadmin/feature_tests/view_data_dml_queries.py",
line 189, in _update_cell

cell_el

File
"/Users/akshay/Development/Workspace_3.5/lib/python3.5/site-packages/selenium/webdriver/common/action_chains.py",
line 80, in perform

self.w3c_actions.perform()

File
"/Users/akshay/Development/Workspace_3.5/lib/python3.5/site-packages/selenium/webdriver/common/actions/action_builder.py",
line 76, in perform

self.driver.execute(Command.W3C_ACTIONS, enc)

File
"/Users/akshay/Development/Workspace_3.5/lib/python3.5/site-packages/selenium/webdriver/remote/webdriver.py",
line 312, in execute

self.error_handler.check_response(response)

File
"/Users/akshay/Development/Workspace_3.5/lib/python3.5/site-packages/selenium/webdriver/remote/errorhandler.py",
line 242, in check_response

raise exception_class(message, screen, stacktrace)

selenium.common.exceptions.MoveTargetOutOfBoundsException: Message:
(1120.5, 289.29998779296875) is out of bounds of viewport width (1067) and
height (728)

======================================================================

FAIL: runTest
(pgadmin.feature_tests.copy_selected_query_results_feature_test.CopySelectedQueryResultsFeatureTest)

Copy rows, column using button and keyboard shortcut

----------------------------------------------------------------------

Traceback (most recent call last):

File
"/Users/akshay/Development/pgadmin4/web/pgadmin/feature_tests/copy_selected_query_results_feature_test.py",
line 64, in runTest

self._mouseup_outside_grid_still_makes_a_selection()

File
"/Users/akshay/Development/pgadmin4/web/pgadmin/feature_tests/copy_selected_query_results_feature_test.py",
line 203, in _mouseup_outside_grid_still_makes_a_selection

self.assertEqual('"cool info"', pyperclip.paste())

AssertionError: '"cool info"' != '"some info"\n"some other info"\n"cool
info"'

- "cool info"

+ "some info"

"some other info"

"cool info"

Attached is the modified patch, please review and let me know the review
comments.

>
> On Tue, Apr 24, 2018 at 3:34 PM, Joao De Almeida Pereira <
> jdealmeidapereira(at)pivotal(dot)io> wrote:
>
>> Hi Akshay,
>>
>> Linter fails and tests are failing when we run the patch on our machine
>> using Firefox. The linter output is:
>>
>> ./regression/runtests.py:184: [E125] continuation line with same indent as next logical line
>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-tests/builds/25#L5ad0f359:255>
>> ./regression/runtests.py:184: [E501] line too long (80 > 79 characters)
>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-tests/builds/25#L5ad0f359:256>
>> 1 E125 continuation line with same indent as next logical line
>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-tests/builds/25#L5ad0f359:257>
>> 1 E501 line too long (80 > 79 characters)
>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-tests/builds/25#L5ad0f359:258>
>> 2
>>
>>
>> The tests output is:
>>
>> =============Running the test cases for 'PostgreSQL 10'=============
>> runTest (pgadmin.feature_tests.copy_selected_query_results_feature_test.CopySelectedQueryResultsFeatureTest)
>> Copy rows, column using button and keyboard shortcut ... ERROR
>> runTest (pgadmin.feature_tests.keyboard_shortcut_test.KeyboardShortcutFeatureTest)
>> Test for keyboard shortcut ... Executing shortcut: File main menu...OK
>> Executing shortcut: Object main menu...OK
>> ok
>> runTest (pgadmin.feature_tests.pg_datatype_validation_test.PGDataypeFeatureTest)
>> Test checks for PG data-types output ... ERROR
>> runTest (pgadmin.feature_tests.query_tool_journey_test.QueryToolJourneyTest)
>> Tests the path through the query tool ... ERROR
>> runTest (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest)
>> Query tool feature test ...
>> On demand query result...
>> On demand result set on scrolling... OK.
>> On demand result set on grid select all... OK.
>> On demand result set on column select all... OK.
>> Explain query with verbose and cost... ERROR
>> runTest (pgadmin.feature_tests.table_ddl_feature_test.TableDdlFeatureTest)
>> Test table DDL generation ... ok
>> runTest (pgadmin.feature_tests.view_data_dml_queries.CheckForViewDataTest)
>> Validate Insert, Update operations in View/Edit data with given test data ... ERROR
>> ERROR
>> runTest (pgadmin.feature_tests.xss_checks_file_manager_test.CheckFileManagerFeatureTest)
>> Tests to check if File manager is vulnerable to XSS ... ERROR
>> runTest (pgadmin.feature_tests.xss_checks_panels_and_query_tool_test.CheckForXssFeatureTest)
>> Test XSS check for panels and query tool ... ERROR
>> ERROR
>> runTest (pgadmin.feature_tests.xss_checks_pgadmin_debugger_test.CheckDebuggerForXssFeatureTest)
>> Tests to check if Debugger is vulnerable to XSS ... ERROR
>> ERROR
>> runTest (pgadmin.feature_tests.xss_checks_roles_control_test.CheckRoleMembershipControlFeatureTest)
>> Tests to check if Role membership control is vulnerable to XSS^[ ... Traceback (most recent call last):
>> File "/home/pivotal/workspace/pgadmin4/web/regression/python_test_utils/test_utils.py", line 338, in create_role
>> sql_query
>> psycopg2.ProgrammingError: role "test_role" already exists
>>
>> Traceback (most recent call last):
>> File "/home/pivotal/workspace/pgadmin4/web/regression/python_test_utils/test_utils.py", line 338, in create_role
>> sql_query
>> psycopg2.ProgrammingError: role "<h1>test</h1>" already exists
>>
>> ERROR
>> ERROR
>>
>> ======================================================================
>> ERROR: runTest (pgadmin.feature_tests.copy_selected_query_results_feature_test.CopySelectedQueryResultsFeatureTest)
>> Copy rows, column using button and keyboard shortcut
>> ----------------------------------------------------------------------
>> Traceback (most recent call last):
>> File "/home/pivotal/workspace/pgadmin4/web/pgadmin/feature_tests/copy_selected_query_results_feature_test.py", line 57, in runTest
>> self._copies_rows()
>> File "/home/pivotal/workspace/pgadmin4/web/pgadmin/feature_tests/copy_selected_query_results_feature_test.py", line 67, in _copies_rows
>> pyperclip.copy("old clipboard contents")
>> File "/home/pivotal/.pyenv/versions/python36/lib/python3.6/site-packages/pyperclip/__init__.py", line 574, in lazy_load_stub_copy
>> return copy(text)
>> File "/home/pivotal/.pyenv/versions/python36/lib/python3.6/site-packages/pyperclip/__init__.py", line 284, in __call__
>> raise PyperclipException(EXCEPT_MSG)
>> pyperclip.PyperclipException:
>> Pyperclip could not find a copy/paste mechanism for your system.
>> For more information, please visit https://pyperclip.readthedocs.io/en/latest/introduction.html#not-implemented-error
>>
>> ======================================================================
>> ERROR: runTest (pgadmin.feature_tests.pg_datatype_validation_test.PGDataypeFeatureTest)
>> Test checks for PG data-types output
>> ----------------------------------------------------------------------
>> Traceback (most recent call last):
>> File "/home/pivotal/workspace/pgadmin4/web/pgadmin/feature_tests/pg_datatype_validation_test.py", line 135, in runTest
>> self._check_datatype()
>> File "/home/pivotal/workspace/pgadmin4/web/pgadmin/feature_tests/pg_datatype_validation_test.py", line 169, in _check_datatype
>> "contains(.,'{}')]".format(batch['datatype'][0])
>> File "/home/pivotal/.pyenv/versions/python36/lib/python3.6/site-packages/selenium/webdriver/support/wait.py", line 80, in until
>> raise TimeoutException(message, screen, stacktrace)
>> selenium.common.exceptions.TimeoutException: Message:
>>
>> ======================================================================
>> ERROR: runTest (pgadmin.feature_tests.query_tool_journey_test.QueryToolJourneyTest)
>> Tests the path through the query tool
>> ----------------------------------------------------------------------
>> Traceback (most recent call last):
>> File "/home/pivotal/workspace/pgadmin4/web/pgadmin/feature_tests/query_tool_journey_test.py", line 49, in runTest
>> self._test_copies_rows()
>> File "/home/pivotal/workspace/pgadmin4/web/pgadmin/feature_tests/query_tool_journey_test.py", line 54, in _test_copies_rows
>> pyperclip.copy("old clipboard contents")
>> File "/home/pivotal/.pyenv/versions/python36/lib/python3.6/site-packages/pyperclip/__init__.py", line 284, in __call__
>> raise PyperclipException(EXCEPT_MSG)
>> pyperclip.PyperclipException:
>> Pyperclip could not find a copy/paste mechanism for your system.
>> For more information, please visit https://pyperclip.readthedocs.io/en/latest/introduction.html#not-implemented-error
>>
>> ======================================================================
>> ERROR: runTest (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest)
>> Query tool feature test
>> ----------------------------------------------------------------------
>> Traceback (most recent call last):
>> File "/home/pivotal/workspace/pgadmin4/web/pgadmin/feature_tests/query_tool_tests.py", line 59, in runTest
>> self._query_tool_explain_with_verbose_and_cost()
>> File "/home/pivotal/workspace/pgadmin4/web/pgadmin/feature_tests/query_tool_tests.py", line 262, in _query_tool_explain_with_verbose_and_cost
>> self.page.find_by_id("btn-explain-costs").click()
>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 174, in find_by_id
>> lambda driver: driver.find_element_by_id(element_id)
>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 261, in wait_for_element
>> return self._wait_for("element to exist", element_if_it_exists)
>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 327, in _wait_for
>> "Timed out waiting for " + waiting_for_message
>> File "/home/pivotal/.pyenv/versions/python36/lib/python3.6/site-packages/selenium/webdriver/support/wait.py", line 80, in until
>> raise TimeoutException(message, screen, stacktrace)
>> selenium.common.exceptions.TimeoutException: Message: Timed out waiting for element to exist
>>
>> ======================================================================
>> ERROR: runTest (pgadmin.feature_tests.view_data_dml_queries.CheckForViewDataTest)
>> Validate Insert, Update operations in View/Edit data with given test data
>> ----------------------------------------------------------------------
>> Traceback (most recent call last):
>> File "/home/pivotal/workspace/pgadmin4/web/pgadmin/feature_tests/view_data_dml_queries.py", line 114, in runTest
>> self.page.add_server(self.server)
>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 55, in add_server
>> self.find_by_partial_link_text("Server...").click()
>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 185, in find_by_partial_link_text
>> EC.element_to_be_clickable((By.PARTIAL_LINK_TEXT, link_text))
>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 327, in _wait_for
>> "Timed out waiting for " + waiting_for_message
>> File "/home/pivotal/.pyenv/versions/python36/lib/python3.6/site-packages/selenium/webdriver/support/wait.py", line 80, in until
>> raise TimeoutException(message, screen, stacktrace)
>> selenium.common.exceptions.TimeoutException: Message: Timed out waiting for link with text "Server..."
>>
>> ======================================================================
>> ERROR: runTest (pgadmin.feature_tests.view_data_dml_queries.CheckForViewDataTest)
>> Validate Insert, Update operations in View/Edit data with given test data
>> ----------------------------------------------------------------------
>> Traceback (most recent call last):
>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/base_feature_test.py", line 66, in tearDown
>> self.after()
>> File "/home/pivotal/workspace/pgadmin4/web/pgadmin/feature_tests/view_data_dml_queries.py", line 132, in after
>> self.page.remove_server(self.server)
>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 122, in remove_server
>> "' and @class='aciTreeItem']")
>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 169, in find_by_xpath
>> lambda driver: driver.find_element_by_xpath(xpath)
>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 261, in wait_for_element
>> return self._wait_for("element to exist", element_if_it_exists)
>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 327, in _wait_for
>> "Timed out waiting for " + waiting_for_message
>> File "/home/pivotal/.pyenv/versions/python36/lib/python3.6/site-packages/selenium/webdriver/support/wait.py", line 80, in until
>> raise TimeoutException(message, screen, stacktrace)
>> selenium.common.exceptions.TimeoutException: Message: Timed out waiting for element to exist
>>
>> ======================================================================
>> ERROR: runTest (pgadmin.feature_tests.xss_checks_file_manager_test.CheckFileManagerFeatureTest)
>> Tests to check if File manager is vulnerable to XSS
>> ----------------------------------------------------------------------
>> Traceback (most recent call last):
>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/base_feature_test.py", line 43, in setUp
>> self.before()
>> File "/home/pivotal/workspace/pgadmin4/web/pgadmin/feature_tests/xss_checks_file_manager_test.py", line 37, in before
>> self.page.add_server(self.server)
>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 55, in add_server
>> self.find_by_partial_link_text("Server...").click()
>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 185, in find_by_partial_link_text
>> EC.element_to_be_clickable((By.PARTIAL_LINK_TEXT, link_text))
>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 327, in _wait_for
>> "Timed out waiting for " + waiting_for_message
>> File "/home/pivotal/.pyenv/versions/python36/lib/python3.6/site-packages/selenium/webdriver/support/wait.py", line 80, in until
>> raise TimeoutException(message, screen, stacktrace)
>> selenium.common.exceptions.TimeoutException: Message: Timed out waiting for link with text "Server..."
>>
>> ======================================================================
>> ERROR: runTest (pgadmin.feature_tests.xss_checks_panels_and_query_tool_test.CheckForXssFeatureTest)
>> Test XSS check for panels and query tool
>> ----------------------------------------------------------------------
>> Traceback (most recent call last):
>> File "/home/pivotal/workspace/pgadmin4/web/pgadmin/feature_tests/xss_checks_panels_and_query_tool_test.py", line 57, in runTest
>> self.page.add_server(self.server)
>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 55, in add_server
>> self.find_by_partial_link_text("Server...").click()
>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 185, in find_by_partial_link_text
>> EC.element_to_be_clickable((By.PARTIAL_LINK_TEXT, link_text))
>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 327, in _wait_for
>> "Timed out waiting for " + waiting_for_message
>> File "/home/pivotal/.pyenv/versions/python36/lib/python3.6/site-packages/selenium/webdriver/support/wait.py", line 80, in until
>> raise TimeoutException(message, screen, stacktrace)
>> selenium.common.exceptions.TimeoutException: Message: Timed out waiting for link with text "Server..."
>>
>> ======================================================================
>> ERROR: runTest (pgadmin.feature_tests.xss_checks_panels_and_query_tool_test.CheckForXssFeatureTest)
>> Test XSS check for panels and query tool
>> ----------------------------------------------------------------------
>> Traceback (most recent call last):
>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/base_feature_test.py", line 66, in tearDown
>> self.after()
>> File "/home/pivotal/workspace/pgadmin4/web/pgadmin/feature_tests/xss_checks_panels_and_query_tool_test.py", line 69, in after
>> self.page.remove_server(self.server)
>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 122, in remove_server
>> "' and @class='aciTreeItem']")
>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 169, in find_by_xpath
>> lambda driver: driver.find_element_by_xpath(xpath)
>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 261, in wait_for_element
>> return self._wait_for("element to exist", element_if_it_exists)
>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 327, in _wait_for
>> "Timed out waiting for " + waiting_for_message
>> File "/home/pivotal/.pyenv/versions/python36/lib/python3.6/site-packages/selenium/webdriver/support/wait.py", line 80, in until
>> raise TimeoutException(message, screen, stacktrace)
>> selenium.common.exceptions.TimeoutException: Message: Timed out waiting for element to exist
>>
>> ======================================================================
>> ERROR: runTest (pgadmin.feature_tests.xss_checks_pgadmin_debugger_test.CheckDebuggerForXssFeatureTest)
>> Tests to check if Debugger is vulnerable to XSS
>> ----------------------------------------------------------------------
>> Traceback (most recent call last):
>> File "/home/pivotal/workspace/pgadmin4/web/pgadmin/feature_tests/xss_checks_pgadmin_debugger_test.py", line 41, in runTest
>> self.page.add_server(self.server)
>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 55, in add_server
>> self.find_by_partial_link_text("Server...").click()
>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 185, in find_by_partial_link_text
>> EC.element_to_be_clickable((By.PARTIAL_LINK_TEXT, link_text))
>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 327, in _wait_for
>> "Timed out waiting for " + waiting_for_message
>> File "/home/pivotal/.pyenv/versions/python36/lib/python3.6/site-packages/selenium/webdriver/support/wait.py", line 80, in until
>> raise TimeoutException(message, screen, stacktrace)
>> selenium.common.exceptions.TimeoutException: Message: Timed out waiting for link with text "Server..."
>>
>> ======================================================================
>> ERROR: runTest (pgadmin.feature_tests.xss_checks_pgadmin_debugger_test.CheckDebuggerForXssFeatureTest)
>> Tests to check if Debugger is vulnerable to XSS
>> ----------------------------------------------------------------------
>> Traceback (most recent call last):
>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/base_feature_test.py", line 66, in tearDown
>> self.after()
>> File "/home/pivotal/workspace/pgadmin4/web/pgadmin/feature_tests/xss_checks_pgadmin_debugger_test.py", line 46, in after
>> self.page.remove_server(self.server)
>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 122, in remove_server
>> "' and @class='aciTreeItem']")
>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 169, in find_by_xpath
>> lambda driver: driver.find_element_by_xpath(xpath)
>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 261, in wait_for_element
>> return self._wait_for("element to exist", element_if_it_exists)
>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 327, in _wait_for
>> "Timed out waiting for " + waiting_for_message
>> File "/home/pivotal/.pyenv/versions/python36/lib/python3.6/site-packages/selenium/webdriver/support/wait.py", line 80, in until
>> raise TimeoutException(message, screen, stacktrace)
>> selenium.common.exceptions.TimeoutException: Message: Timed out waiting for element to exist
>>
>> ======================================================================
>> ERROR: runTest (pgadmin.feature_tests.xss_checks_roles_control_test.CheckRoleMembershipControlFeatureTest)
>> Tests to check if Role membership control is vulnerable to XSS
>> ----------------------------------------------------------------------
>> Traceback (most recent call last):
>> File "/home/pivotal/workspace/pgadmin4/web/pgadmin/feature_tests/xss_checks_roles_control_test.py", line 37, in runTest
>> self.page.add_server(self.server)
>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 55, in add_server
>> self.find_by_partial_link_text("Server...").click()
>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 185, in find_by_partial_link_text
>> EC.element_to_be_clickable((By.PARTIAL_LINK_TEXT, link_text))
>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 327, in _wait_for
>> "Timed out waiting for " + waiting_for_message
>> File "/home/pivotal/.pyenv/versions/python36/lib/python3.6/site-packages/selenium/webdriver/support/wait.py", line 80, in until
>> raise TimeoutException(message, screen, stacktrace)
>> selenium.common.exceptions.TimeoutException: Message: Timed out waiting for link with text "Server..."
>>
>> ======================================================================
>> ERROR: runTest (pgadmin.feature_tests.xss_checks_roles_control_test.CheckRoleMembershipControlFeatureTest)
>> Tests to check if Role membership control is vulnerable to XSS
>> ----------------------------------------------------------------------
>> Traceback (most recent call last):
>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/base_feature_test.py", line 66, in tearDown
>> self.after()
>> File "/home/pivotal/workspace/pgadmin4/web/pgadmin/feature_tests/xss_checks_roles_control_test.py", line 42, in after
>> self.page.remove_server(self.server)
>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 122, in remove_server
>> "' and @class='aciTreeItem']")
>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 169, in find_by_xpath
>> lambda driver: driver.find_element_by_xpath(xpath)
>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 261, in wait_for_element
>> return self._wait_for("element to exist", element_if_it_exists)
>> File "/home/pivotal/workspace/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 327, in _wait_for
>> "Timed out waiting for " + waiting_for_message
>> File "/home/pivotal/.pyenv/versions/python36/lib/python3.6/site-packages/selenium/webdriver/support/wait.py", line 80, in until
>> raise TimeoutException(message, screen, stacktrace)
>> selenium.common.exceptions.TimeoutException: Message: Timed out waiting for element to exist
>>
>> ----------------------------------------------------------------------
>> Ran 11 tests in 441.881s
>>
>> FAILED (errors=13)
>>
>> ======================================================================
>> Test Result Summary
>> ======================================================================
>>
>> PostgreSQL 10:
>>
>> 2 tests passed
>> 9 tests failed:
>> CopySelectedQueryResultsFeatureTest (Copy rows, column using button and keyboard shortcut)
>> PGDataypeFeatureTest (Test checks for PG data-types output)
>> QueryToolJourneyTest (Tests the path through the query tool)
>> QueryToolFeatureTest (Query tool feature test)
>> CheckForViewDataTest (Validate Insert, Update operations in View/Edit data with given test data)
>> CheckFileManagerFeatureTest (Tests to check if File manager is vulnerable to XSS)
>> CheckForXssFeatureTest (Test XSS check for panels and query tool)
>> CheckDebuggerForXssFeatureTest (Tests to check if Debugger is vulnerable to XSS)
>> CheckRoleMembershipControlFeatureTest (Tests to check if Role membership control is vulnerable to XSS)
>> 0 tests skipped
>>
>> ======================================================================
>>
>> ​
>>
>> All the above tests fail in the following place:
>> [image:
>> CheckDebuggerForXssFeatureTest-2018.04.24_10.28.27-Python-3.6.4.png]
>>
>>
>> Also the tests will never end because firefox is waiting for the user to
>> click "Leave the page" button.
>>
>> Thanks
>> Victoria & Joao
>>
>> On Tue, Apr 24, 2018 at 2:26 AM Akshay Joshi <
>> akshay(dot)joshi(at)enterprisedb(dot)com> wrote:
>>
>>> Hi Hackers,
>>>
>>> On Mon, Apr 23, 2018 at 9:07 PM, Anthony Emengo <aemengo(at)pivotal(dot)io>
>>> wrote:
>>>
>>>> We also tried running the tests with this patch. It didn't launch
>>>> without some code changes and several tests were failing. We should really
>>>> defer pulling this in until we have more robust results on Firefox
>>>>
>>>> In order to have the tests running we had to do the following change:
>>>>
>>>> diff --git a/web/regression/feature_utils/app_starter.py
>>>> b/web/regression/feature_utils/app_starter.py
>>>> index 77b0400c..50d3e307 100644
>>>> --- a/web/regression/feature_utils/app_starter.py
>>>> +++ b/web/regression/feature_utils/app_starter.py
>>>> @@ -42,10 +44,18 @@ class AppStarter:
>>>> )
>>>>
>>>> self.driver.set_window_size(1280, 1024)
>>>> - self.driver.get(
>>>> - "http://" + self.app_config.DEFAULT_SERVER + ":" +
>>>> - random_server_port
>>>> - )
>>>> + # self.driver.implicitly_wait(60)
>>>> +
>>>> + def launch_browser():
>>>> + try:
>>>> + self.driver.get(
>>>> + "http://" + self.app_config.DEFAULT_SERVER + ":" +
>>>> + random_server_port
>>>> + )
>>>> + except WebDriverException as e:
>>>> + time.sleep(5)
>>>> + launch_browser()
>>>> + launch_browser()
>>>>
>>>> This change was required because firefox was throwing an exception when
>>>> we tried to get the address and the server was not running. We saw this
>>>> behavior in Ubuntu.
>>>>
>>>
>>> Yes I have faced the same problem in Ubuntu. As per suggestion by
>>> Dave I have added parameter in test_config.json.in and also include the
>>> above code given by Anthony.
>>> Attached is the modified patch.
>>>
>>>>
>>>>
>>>> - Anthony and Joao
>>>>
>>>> On Mon, Apr 23, 2018 at 9:12 AM, Dave Page <dpage(at)pgadmin(dot)org> wrote:
>>>>
>>>>> Hi
>>>>>
>>>>> On Mon, Apr 23, 2018 at 2:05 PM, Akshay Joshi <
>>>>> akshay(dot)joshi(at)enterprisedb(dot)com> wrote:
>>>>>
>>>>>> Hi Hackers,
>>>>>>
>>>>>> I have added support for running feature tests against FireFox. For
>>>>>> that user will have to download gecko driver from https://github.com/
>>>>>> mozilla/geckodriver/releases and follow the below steps:
>>>>>>
>>>>>> - Extract the gecko driver.
>>>>>> - Run chmod +x geckodriver.
>>>>>> - Either copy the geckodriver to /usr/local/bin or the path of
>>>>>> the geckodriver must be specified in PATH.
>>>>>> - Apply the attached patch.
>>>>>> - Change the parameter "DEFAULT_TEST_BROWSER = 'Firefox' "
>>>>>> - Start the feature test.
>>>>>>
>>>>>> The config option needs to be in test_config.json. We don't really
>>>>> want test-specific config options in config.py (arguably, TEST_SQLITE_PATH
>>>>> shoudn't be there either, as it's useless for end users).
>>>>>
>>>>> I'm surprised to not see any updates to tests. When I tried firefox
>>>>> manually a few weeks back, it was failing at lot.
>>>>>
>>>>> --
>>>>> Dave Page
>>>>> Blog: http://pgsnake.blogspot.com
>>>>> Twitter: @pgsnake
>>>>>
>>>>> EnterpriseDB UK: http://www.enterprisedb.com
>>>>> The Enterprise PostgreSQL Company
>>>>>
>>>>
>>>>
>>> --
>>> *Akshay Joshi*
>>>
>>> *Sr. Software Architect *
>>>
>>>
>>>
>>> *Phone: +91 20-3058-9517 <+91%2020%203058%209517>Mobile: +91
>>> 976-788-8246 <+91%2097678%2088246>*
>>>
>>
>
>
> --
> Dave Page
> Blog: http://pgsnake.blogspot.com
> Twitter: @pgsnake
>
> EnterpriseDB UK: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>

--
*Akshay Joshi*

*Sr. Software Architect *

*Phone: +91 20-3058-9517Mobile: +91 976-788-8246*