Re: [pgadmin4][Patch]: Test cases for the backup module

Hi there,

We've been noticing some issues with the tests on both our CI and local Mac
workstations.

1. When the following code blocks are invoked - we get plenty of
app.context() issues. It must not be valid when running tests.

​

from pgadmin.utils.driver import get_driver
driver = get_driver(PG_DEFAULT_DRIVER)
manager = driver.connection_manager(self.sid)

host = manager.local_bind_host if manager.use_ssh_tunnel else s.host
port = manager.local_bind_port if manager.use_ssh_tunnel else s.port

2. When we finally enable

"default_binary_paths": {

in our test_config, we get more failing tests that look like:

======================================================================
FAIL: runTest (pgadmin.tools.restore.tests.test_restore_create_job_unit_test.RestoreCreateJobTest)
When restore object with option - Miscellaneous
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/pivotal/.pyenv/versions/3.6.5/lib/python3.6/unittest/mock.py",
line 1179, in patched
return func(*args, **keywargs)
File "/Users/pivotal/workspace/pgadmin4/web/pgadmin/tools/restore/tests/test_restore_create_job_unit_test.py",
line 295, in runTest
self.assertEquals(response.status_code, 200)
AssertionError: 410 != 200

And

When restore object with the sections options ... 2018-05-31
12:24:42,988: ERROR pgadmin: illegal environment variable name
Traceback (most recent call last):
File "/Users/pivotal/workspace/pgadmin4/web/pgadmin/tools/restore/__init__.py",
line 352, in create_restore_job
manager.export_password_env(p.id)
File "/Users/pivotal/workspace/pgadmin4/web/pgadmin/utils/driver/psycopg2/server_manager.py",
line 365, in export_password_env
os.environ[str(env)] = password
File "/Users/pivotal/.pyenv/versions/3.6.5/lib/python3.6/os.py",
line 675, in __setitem__
self.putenv(key, value)
ValueError: illegal environment variable name
FAIL

​

Sincerely,

Victoria && Anthony

On Thu, May 31, 2018 at 1:16 AM Khushboo Vashi <
khushboo(dot)vashi(at)enterprisedb(dot)com> wrote:

> Hi,
>
> Please find the attached updated patch with the fixes.
> The test cases were only failing on MAC not on Linux.
>
> Thanks,
> Khushboo
>
> On Wed, May 30, 2018 at 10:13 AM, Khushboo Vashi <
> khushboo(dot)vashi(at)enterprisedb(dot)com> wrote:
>
>>
>>
>> On Wed, May 30, 2018 at 1:05 AM, Dave Page <dpage(at)pgadmin(dot)org> wrote:
>>
>>> Hi
>>>
>>> On Mon, May 28, 2018 at 8:09 AM, Khushboo Vashi <
>>> khushboo(dot)vashi(at)enterprisedb(dot)com> wrote:
>>>
>>>> Hi,
>>>>
>>>> please find the attached updated patch for the test cases of Backup,
>>>> Restore and Maintenance modules which includes:
>>>>
>>>> 1. Unit test cases
>>>> 2. End to end regression test cases
>>>> 3. Feature test cases
>>>>
>>>
>>> Thanks. I've yet to be able to run the feature tests successfully.
>>> Here's what I've found so far:
>>>
>>> 1) DEFAULT_BINARY_PATHS should be default_binary_paths in the JSON
>>> config file.
>>>
>>> Will do.
>>
>>> 2) I've hit screensize related issues:
>>>
>>> ======================================================================
>>>
>>> ERROR: runTest
>>> (pgadmin.feature_tests.pg_utilities_maintenance_test.PGUtilitiesMaintenanceFeatureTest)
>>>
>>> Test for PG maintenance: database
>>>
>>> ----------------------------------------------------------------------
>>>
>>> Traceback (most recent call last):
>>>
>>> File
>>> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/pg_utilities_maintenance_test.py",
>>> line 56, in runTest
>>>
>>> self._open_maintenance_dialogue()
>>>
>>> File
>>> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/pg_utilities_maintenance_test.py",
>>> line 75, in _open_maintenance_dialogue
>>>
>>> "*[.='" + self.table_name + "']/../*[(at)class='aciTreeItem'"
>>>
>>> File
>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webelement.py",
>>> line 80, in click
>>>
>>> self._execute(Command.CLICK_ELEMENT)
>>>
>>> File
>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webelement.py",
>>> line 628, in _execute
>>>
>>> return self._parent.execute(command, params)
>>>
>>> File
>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py",
>>> line 312, in execute
>>>
>>> self.error_handler.check_response(response)
>>>
>>> File
>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py",
>>> line 242, in check_response
>>>
>>> raise exception_class(message, screen, stacktrace)
>>>
>>> WebDriverException: Message: unknown error: Element <span
>>> class="aciTreeItem">...</span> is not clickable at point (223, 604). Other
>>> element would receive the click: <div class="wcFrameCenter
>>> wcPanelBackground wcScrollableX wcScrollableY" style="left: 0px; right:
>>> 0px; bottom: 0px;">...</div>
>>>
>>> (Session info: chrome=66.0.3359.181)
>>>
>>> (Driver info: chromedriver=2.38.552518
>>> (183d19265345f54ce39cbb94cf81ba5f15905011),platform=Mac OS X 10.12.6 x86_64)
>>>
>>> 3) One time the test did start, but then I saw this failure:
>>>
>>> ======================================================================
>>>
>>> ERROR: runTest
>>> (pgadmin.feature_tests.pg_utilities_backup_restore_test.PGUtilitiesBackupFeatureTest)
>>>
>>> Test for PG utilities - Backup and Restore
>>>
>>> ----------------------------------------------------------------------
>>>
>>> Traceback (most recent call last):
>>>
>>> File
>>> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/pg_utilities_backup_restore_test.py",
>>> line 93, in runTest
>>>
>>> self.page.fill_input_by_field_name("file", "test_backup_file")
>>>
>>> File
>>> "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgadmin_page.py",
>>> line 211, in fill_input_by_field_name
>>>
>>> self.wait_for_input_field_content(field_name, field_content)
>>>
>>> File
>>> "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgadmin_page.py",
>>> line 251, in wait_for_input_field_content
>>>
>>> "field to contain '" + str(content) + "'", input_field_has_content
>>>
>>> File
>>> "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgadmin_page.py",
>>> line 337, in _wait_for
>>>
>>> "Timed out waiting for " + waiting_for_message
>>>
>>> File
>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/support/wait.py",
>>> line 80, in until
>>>
>>> raise TimeoutException(message, screen, stacktrace)
>>>
>>> TimeoutException: Message: Timed out waiting for field to contain
>>> 'test_backup_file'
>>>
>>>
>>>
>>> (with screenshot attached)
>>>
>>> Thanks.
>>>
>>> I have ran the feature tests with multiple servers many times but didn't
>> get a single failure.
>> I have asked Akshay to run on his machine, let see what he gets.
>>
>>>
>>>
>>>
>>>
>>>>
>>>> Thanks,
>>>> Khushboo
>>>>
>>>>
>>>>
>>>>
>>>> On Wed, Apr 25, 2018 at 9:40 PM, Joao De Almeida Pereira <
>>>> jdealmeidapereira(at)pivotal(dot)io> wrote:
>>>>
>>>>> Hi Khushboo,
>>>>>
>>>>> We reviewed the patch and it is very nice to see some more coverage in
>>>>> this area. Good job :D
>>>>>
>>>>> We passed the tests through our CI the feature tests are not passing,
>>>>> but the linter fails:
>>>>>
>>>>> ./pgadmin/feature_tests/pg_utilities_backup_test.py:37: [E501] line too long (91 > 79 characters)
>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17#L5ad0f3d8:265>
>>>>> ./pgadmin/feature_tests/pg_utilities_backup_test.py:53: [E501] line too long (104 > 79 characters)
>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17#L5ad0f3d8:266>
>>>>> ./pgadmin/feature_tests/pg_utilities_backup_test.py:59: [E501] line too long (85 > 79 characters)
>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17#L5ad0f3d8:267>
>>>>> ./pgadmin/feature_tests/pg_utilities_backup_test.py:62: [E501] line too long (96 > 79 characters)
>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17#L5ad0f3d8:268>
>>>>> ./pgadmin/feature_tests/pg_utilities_backup_test.py:63: [E501] line too long (91 > 79 characters)
>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17#L5ad0f3d8:269>
>>>>> ./pgadmin/feature_tests/pg_utilities_backup_test.py:70: [E501] line too long (118 > 79 characters)
>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17#L5ad0f3d8:270>
>>>>> ./pgadmin/tools/backup/tests/test_backup_message.py:37: [E121] continuation line under-indented for hanging indent
>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17#L5ad0f3d8:271>
>>>>> ./pgadmin/tools/backup/tests/test_backup_message.py:48: [E122] continuation line missing indentation or outdented
>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17#L5ad0f3d8:272>
>>>>> ./pgadmin/tools/backup/tests/test_backup_message.py:49: [E251] unexpected spaces around keyword / parameter equals
>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17#L5ad0f3d8:273>
>>>>> ./pgadmin/tools/backup/tests/test_backup_message.py:49: [E251] unexpected spaces around keyword / parameter equals
>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17#L5ad0f3d8:274>
>>>>> ./pgadmin/tools/backup/tests/test_backup_message.py:51: [E501] line too long (91 > 79 characters)
>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17#L5ad0f3d8:275>
>>>>> ./pgadmin/tools/backup/tests/test_backup_message.py:52: [E501] line too long (94 > 79 characters)
>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17#L5ad0f3d8:276>
>>>>> ./pgadmin/tools/backup/tests/test_backup_message.py:53: [E501] line too long (108 > 79 characters)
>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17#L5ad0f3d8:277>
>>>>> ./pgadmin/tools/backup/tests/test_backup_message.py:81: [E501] line too long (113 > 79 characters)
>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17#L5ad0f3d8:278>
>>>>> ./pgadmin/tools/backup/tests/test_backup_message.py:82: [E501] line too long (94 > 79 characters)
>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17#L5ad0f3d8:279>
>>>>> ./pgadmin/tools/backup/tests/test_backup_message.py:83: [E501] line too long (108 > 79 characters)
>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17#L5ad0f3d8:280>
>>>>> ./pgadmin/tools/backup/tests/test_backup_message.py:111: [E501] line too long (100 > 79 characters)
>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17#L5ad0f3d8:281>
>>>>> ./pgadmin/tools/backup/tests/test_backup_message.py:113: [E501] line too long (94 > 79 characters)
>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17#L5ad0f3d8:282>
>>>>> ./pgadmin/tools/backup/tests/test_backup_message.py:114: [E501] line too long (108 > 79 characters)
>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17#L5ad0f3d8:283>
>>>>> ./pgadmin/tools/backup/tests/test_backup_message.py:147: [E501] line too long (93 > 79 characters)
>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17#L5ad0f3d8:284>
>>>>> ./pgadmin/tools/backup/tests/test_batch_process.py:40: [E121] continuation line under-indented for hanging indent
>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17#L5ad0f3d8:285>
>>>>> ./pgadmin/tools/backup/tests/test_batch_process.py:51: [E122] continuation line missing indentation or outdented
>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17#L5ad0f3d8:286>
>>>>> ./pgadmin/tools/backup/tests/test_batch_process.py:135: [E501] line too long (80 > 79 characters)
>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17#L5ad0f3d8:287>
>>>>> ./pgadmin/tools/backup/tests/test_batch_process.py:137: [E501] line too long (83 > 79 characters)
>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17#L5ad0f3d8:288>
>>>>> ./pgadmin/tools/backup/tests/test_batch_process.py:138: [E122] continuation line missing indentation or outdented
>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17#L5ad0f3d8:289>
>>>>> ./pgadmin/tools/backup/tests/test_batch_process.py:139: [E122] continuation line missing indentation or outdented
>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17#L5ad0f3d8:290>
>>>>> ./pgadmin/tools/backup/tests/test_batch_process.py:140: [E122] continuation line missing indentation or outdented
>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17#L5ad0f3d8:291>
>>>>> ./pgadmin/tools/backup/tests/test_batch_process.py:191: [E501] line too long (81 > 79 characters)
>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17#L5ad0f3d8:292>
>>>>> ./pgadmin/tools/backup/tests/test_batch_process.py:203: [E501] line too long (80 > 79 characters)
>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17#L5ad0f3d8:293>
>>>>> ./pgadmin/tools/backup/tests/test_batch_process.py:204: [E128] continuation line under-indented for visual indent
>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17#L5ad0f3d8:294>
>>>>> ./pgadmin/tools/backup/tests/test_batch_process.py:204: [E501] line too long (94 > 79 characters)
>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17#L5ad0f3d8:295>
>>>>> ./pgadmin/tools/backup/tests/test_batch_process.py:205: [E128] continuation line under-indented for visual indent
>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17#L5ad0f3d8:296>
>>>>> ./pgadmin/tools/backup/tests/test_batch_process.py:205: [E501] line too long (94 > 79 characters)
>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17#L5ad0f3d8:297>
>>>>> ./pgadmin/tools/backup/tests/test_batch_process.py:216: [W391] blank line at end of file
>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17#L5ad0f3d8:298>
>>>>> ./pgadmin/tools/backup/tests/test_create_backup_job.py:296: [E501] line too long (97 > 79 characters)
>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17#L5ad0f3d8:299>
>>>>> ./pgadmin/tools/backup/tests/test_create_backup_job.py:317: [E303] too many blank lines (2)
>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17#L5ad0f3d8:300>
>>>>> ./pgadmin/tools/backup/tests/test_create_backup_job.py:336: [E501] line too long (84 > 79 characters)
>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17#L5ad0f3d8:301>
>>>>> ./pgadmin/tools/backup/tests/test_create_backup_job.py:371: [W391] blank line at end of file
>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17#L5ad0f3d8:302>
>>>>> 2 E121 continuation line under-indented for hanging indent
>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17#L5ad0f3d8:303>
>>>>> 5 E122 continuation line missing indentation or outdented
>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17#L5ad0f3d8:304>
>>>>> 2 E128 continuation line under-indented for visual indent
>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17#L5ad0f3d8:305>
>>>>> 2 E251 unexpected spaces around keyword / parameter equals
>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17#L5ad0f3d8:306>
>>>>> 1 E303 too many blank lines (2)
>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17#L5ad0f3d8:307>
>>>>> 24 E501 line too long (91 > 79 characters)
>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17#L5ad0f3d8:308>
>>>>> 2 W391 blank line at end of file
>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17#L5ad0f3d8:309>
>>>>> 38
>>>>>
>>>>>
>>>>> For the feature tests, we realized we had to update the configuration,
>>>>> and we did that, but we get the following error attached. We spent some
>>>>> time trying to understand the problem but we were not successful.
>>>>>
>>>>>
>>>>> Codewise:
>>>>> - We just found some One Letter Variables in the code...
>>>>> - Looks like there is a bug report in this area of the code and we do
>>>>> not have coverage for it: https://redmine.postgresql.org/issues/3232
>>>>> Looks like in some of the unit tests we only have happy path tests,
>>>>> maybe we should see if there are any sad paths that also need coverage.
>>>>>
>>>>> The configuration change, maybe need to be updated. When we install
>>>>> multiple versions of postgres the binaries live in
>>>>> `/usr/lib/postgresql/{{db_version}}/bin`, which makes us think that this
>>>>> configuration should live near the server configuration, maybe? Also to
>>>>> maintain coherency on the naming maybe we should make it all lower case.
>>>>> Just as an aside, you can add the gpdb configuration as well in you
>>>>> patch.
>>>>>
>>>>> Thanks
>>>>> Victoria & Joao
>>>>>
>>>>> On Wed, Apr 25, 2018 at 5:20 AM Khushboo Vashi <
>>>>> khushboo(dot)vashi(at)enterprisedb(dot)com> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> Please find the attached patch which covers test cases for the backup
>>>>>> module (RM #3206).
>>>>>>
>>>>>> 1. Unit test cases
>>>>>> 2. End to end regression test cases
>>>>>> 3. Feature test cases
>>>>>>
>>>>>> Thanks,
>>>>>> Khushboo
>>>>>>
>>>>>
>>>>
>>>
>>>
>>> --
>>> Dave Page
>>> Blog: http://pgsnake.blogspot.com
>>> Twitter: @pgsnake
>>>
>>> EnterpriseDB UK: http://www.enterprisedb.com
>>> The Enterprise PostgreSQL Company
>>>
>>
>>
>