Re: [pgadmin4][Patch]: Test cases for the backup module

Hi,

Please find the attached updated patch.

On Wed, Jun 13, 2018 at 5:42 PM, Khushboo Vashi <
khushboo(dot)vashi(at)enterprisedb(dot)com> wrote:

> Hi Victoria,
>
>
> On Tue, Jun 12, 2018 at 9:06 PM, Victoria Henry <vhenry(at)pivotal(dot)io> wrote:
>
>> Hi Khushboo
>> can you explain a little bit more this while loop?
>>
>>
>> cnt = 0
>> while 1:
>> if cnt > 1:
>> break
>> # Check the process list
>> response1 = self.tester.get('/misc/bgprocess/?_='.format(
>> random.randint(1, 9999999)))
>> self.assertEquals(response1.status_code, 200)
>> process_list = json.loads(response1.data.decode('utf-8'))
>>
>> if len(process_list) > 0 and 'execution_time' in process_list[0]:
>> break
>> time.sleep(0.5)
>> cnt += 1
>>
>> From what it looks like this will only run twice, maybe a for would be a
>> better solution because we know it will only run twice. Also are we sure we
>> only want it to run twice?
>>
>> The code waits till the background process completes. So, while I
> originally developed, not intended to run only twice.
> But after that I put a kind of break point and that remains there. So, I
> will remove that if condition which is not required.
>
I have made maximum 5 attempts.

> We are using PyCharm to do our developments and we notice there are a big
>> group of unused variables throughout. We should remove them if they are not
>> needed. Not sure if your editor also shows that information or not.
>>
> Do you know if there is a configuration in pycodestyle to enable the check
>> for unused variables? That would help a lot.
>>
>> Removed unused local variables.

> Thanks for sharing the information.
>
>> The code
>>
>> assert 'execution_time' in process_list[0]
>> assert 'stime' in process_list[0]
>> assert 'exit_code' in process_list[0]
>> assert process_list[0]['exit_code'] in self.expected_exit_code
>>
>> in test_Create_restore_job should use self.assertEqual or similar from
>> unittest instead of plain assert. Because when something fails we do not
>> have a way to understand what was wrong.
>>
> Will do.
>
Done

> The tests on the restore are still failing GreenPlum.
>>
> It is failing because, Restore is not working with GreenPlum. Can you
> please look into the Restore functionality for GPDB?
>
>
> Thanks,
> Khushboo
>
>> ​
>>
>>
>> Thanks,
Khushboo

>
>> Thanks
>> Victoria & Joao
>>
>> On Tue, Jun 12, 2018 at 6:44 AM Khushboo Vashi <
>> khushboo(dot)vashi(at)enterprisedb(dot)com> wrote:
>>
>>> Please find the attached updated patch with some code cleanup.
>>>
>>> On Tue, Jun 12, 2018 at 3:54 PM, Khushboo Vashi <
>>> khushboo(dot)vashi(at)enterprisedb(dot)com> wrote:
>>>
>>>> Hi,
>>>>
>>>> Please find the attached patch excluding feature test cases.
>>>> Python test cases are working fine, so we can commit this patch. I am
>>>> working on fixing the feature tests which are failing on the different
>>>> window sizes.
>>>>
>>>> Thanks,
>>>> Khushboo
>>>>
>>>> On Fri, Jun 8, 2018 at 2:38 PM, Dave Page <dpage(at)pgadmin(dot)org> wrote:
>>>>
>>>>> Hi
>>>>>
>>>>> On Fri, Jun 8, 2018 at 6:33 AM, Khushboo Vashi <
>>>>> khushboo(dot)vashi(at)enterprisedb(dot)com> wrote:
>>>>>
>>>>>> Hi Dave,
>>>>>>
>>>>>> As per our discussion I have changed the window size to 1280X800,
>>>>>> before it was 1280X900.
>>>>>> Please find the attached updated patch
>>>>>>
>>>>>
>>>>> I'm not sure that actually made any difference on my system. The
>>>>> window continued to look taller than it is wide, so I wonder if the code to
>>>>> set the size is being ignored, or is at the wrong place?
>>>>>
>>>>> Anyway, I got 10 failures with this patch :-(
>>>>>
>>>>> ======================================================================
>>>>>
>>>>> ERROR: runTest (pgadmin.feature_tests.pg_util
>>>>> ities_backup_restore_test.PGUtilitiesBackupFeatureTest)
>>>>>
>>>>> Test for PG utilities - Backup and Restore
>>>>>
>>>>> ----------------------------------------------------------------------
>>>>>
>>>>> Traceback (most recent call last):
>>>>>
>>>>> File "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/pg_utilities_backup_restore_test.py",
>>>>> line 97, in runTest
>>>>>
>>>>> self.page.find_by_xpath("//div[contains(@class,'wcFloatingFocus')"
>>>>>
>>>>> File "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgadmin_page.py",
>>>>> line 171, in find_by_xpath
>>>>>
>>>>> lambda driver: driver.find_element_by_xpath(xpath)
>>>>>
>>>>> File "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgadmin_page.py",
>>>>> line 263, in wait_for_element
>>>>>
>>>>> return self._wait_for("element to exist", element_if_it_exists)
>>>>>
>>>>> File "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgadmin_page.py",
>>>>> line 337, in _wait_for
>>>>>
>>>>> "Timed out waiting for " + waiting_for_message
>>>>>
>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa
>>>>> ges/selenium/webdriver/support/wait.py", line 80, in until
>>>>>
>>>>> raise TimeoutException(message, screen, stacktrace)
>>>>>
>>>>> TimeoutException: Message: Timed out waiting for element to exist
>>>>>
>>>>>
>>>>>
>>>>> ======================================================================
>>>>>
>>>>> ERROR: runTest (pgadmin.feature_tests.xss_che
>>>>> cks_pgadmin_debugger_test.CheckDebuggerForXssFeatureTest)
>>>>>
>>>>> Tests to check if Debugger is vulnerable to XSS
>>>>>
>>>>> ----------------------------------------------------------------------
>>>>>
>>>>> Traceback (most recent call last):
>>>>>
>>>>> File "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/xss_checks_pgadmin_debugger_test.py",
>>>>> line 42, in runTest
>>>>>
>>>>> self._function_node_expandable()
>>>>>
>>>>> File "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/xss_checks_pgadmin_debugger_test.py",
>>>>> line 57, in _function_node_expandable
>>>>>
>>>>> self.page.select_tree_item("a_test_function()")
>>>>>
>>>>> File "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgadmin_page.py",
>>>>> line 135, in select_tree_item
>>>>>
>>>>> "' and @class='aciTreeItem']").click()
>>>>>
>>>>> File "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgadmin_page.py",
>>>>> line 171, in find_by_xpath
>>>>>
>>>>> lambda driver: driver.find_element_by_xpath(xpath)
>>>>>
>>>>> File "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgadmin_page.py",
>>>>> line 263, in wait_for_element
>>>>>
>>>>> return self._wait_for("element to exist", element_if_it_exists)
>>>>>
>>>>> File "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgadmin_page.py",
>>>>> line 337, in _wait_for
>>>>>
>>>>> "Timed out waiting for " + waiting_for_message
>>>>>
>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa
>>>>> ges/selenium/webdriver/support/wait.py", line 80, in until
>>>>>
>>>>> raise TimeoutException(message, screen, stacktrace)
>>>>>
>>>>> TimeoutException: Message: Timed out waiting for element to exist
>>>>>
>>>>>
>>>>>
>>>>> ======================================================================
>>>>>
>>>>> ERROR: runTest (pgadmin.tools.backup.tests.te
>>>>> st_create_backup_job.BackupJobTest)
>>>>>
>>>>> When backup the object with the default options
>>>>>
>>>>> ----------------------------------------------------------------------
>>>>>
>>>>> Traceback (most recent call last):
>>>>>
>>>>> File "/Users/dpage/git/pgadmin4/web/pgadmin/tools/backup/tests/test_create_backup_job.py",
>>>>> line 58, in runTest
>>>>>
>>>>> self.assertNotIn
>>>>>
>>>>> File "/Users/dpage/git/pgadmin4/web/pgadmin/tools/backup/tests/test_backup_utils.py",
>>>>> line 33, in run_backup_job
>>>>>
>>>>> random.randint(1, 9999999)))
>>>>>
>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/test.py",
>>>>> line 830, in get
>>>>>
>>>>> return self.open(*args, **kw)
>>>>>
>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/testing.py",
>>>>> line 127, in open
>>>>>
>>>>> follow_redirects=follow_redirects)
>>>>>
>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/test.py",
>>>>> line 803, in open
>>>>>
>>>>> response = self.run_wsgi_app(environ, buffered=buffered)
>>>>>
>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/test.py",
>>>>> line 716, in run_wsgi_app
>>>>>
>>>>> rv = run_wsgi_app(self.application, environ, buffered=buffered)
>>>>>
>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/test.py",
>>>>> line 923, in run_wsgi_app
>>>>>
>>>>> app_rv = app(environ, start_response)
>>>>>
>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>>>>> line 1997, in __call__
>>>>>
>>>>> return self.wsgi_app(environ, start_response)
>>>>>
>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>>>>> line 1985, in wsgi_app
>>>>>
>>>>> response = self.handle_exception(e)
>>>>>
>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>>>>> line 1540, in handle_exception
>>>>>
>>>>> reraise(exc_type, exc_value, tb)
>>>>>
>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>>>>> line 1982, in wsgi_app
>>>>>
>>>>> response = self.full_dispatch_request()
>>>>>
>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>>>>> line 1614, in full_dispatch_request
>>>>>
>>>>> rv = self.handle_user_exception(e)
>>>>>
>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>>>>> line 1517, in handle_user_exception
>>>>>
>>>>> reraise(exc_type, exc_value, tb)
>>>>>
>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>>>>> line 1612, in full_dispatch_request
>>>>>
>>>>> rv = self.dispatch_request()
>>>>>
>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>>>>> line 1598, in dispatch_request
>>>>>
>>>>> return self.view_functions[rule.endpoint](**req.view_args)
>>>>>
>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask_login.py",
>>>>> line 792, in decorated_view
>>>>>
>>>>> return func(*args, **kwargs)
>>>>>
>>>>> File "/Users/dpage/git/pgadmin4/web/pgadmin/misc/bgprocess/__init__.py",
>>>>> line 62, in index
>>>>>
>>>>> return make_response(response=BatchProcess.list())
>>>>>
>>>>> File "/Users/dpage/git/pgadmin4/web/pgadmin/misc/bgprocess/processes.py",
>>>>> line 584, in list
>>>>>
>>>>> details = desc.details(p.command, args)
>>>>>
>>>>> File "/Users/dpage/git/pgadmin4/web/pgadmin/tools/backup/__init__.py",
>>>>> line 159, in details
>>>>>
>>>>> name, host, port = self.get_server_details()
>>>>>
>>>>> File "/Users/dpage/git/pgadmin4/web/pgadmin/tools/backup/__init__.py",
>>>>> line 122, in get_server_details
>>>>>
>>>>> host = manager.local_bind_host if manager.use_ssh_tunnel else
>>>>> s.host
>>>>>
>>>>> AttributeError: 'NoneType' object has no attribute 'use_ssh_tunnel'
>>>>>
>>>>>
>>>>> ======================================================================
>>>>>
>>>>> ERROR: runTest (pgadmin.tools.maintenance.tes
>>>>> ts.test_create_maintenance_job.MaintenanceJobTest)
>>>>>
>>>>> When maintenance the object with the default options
>>>>>
>>>>> ----------------------------------------------------------------------
>>>>>
>>>>> Traceback (most recent call last):
>>>>>
>>>>> File "/Users/dpage/git/pgadmin4/web/pgadmin/tools/maintenance/tes
>>>>> ts/test_create_maintenance_job.py", line 71, in runTest
>>>>>
>>>>> random.randint(1, 9999999)))
>>>>>
>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/test.py",
>>>>> line 830, in get
>>>>>
>>>>> return self.open(*args, **kw)
>>>>>
>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/testing.py",
>>>>> line 127, in open
>>>>>
>>>>> follow_redirects=follow_redirects)
>>>>>
>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/test.py",
>>>>> line 803, in open
>>>>>
>>>>> response = self.run_wsgi_app(environ, buffered=buffered)
>>>>>
>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/test.py",
>>>>> line 716, in run_wsgi_app
>>>>>
>>>>> rv = run_wsgi_app(self.application, environ, buffered=buffered)
>>>>>
>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/test.py",
>>>>> line 923, in run_wsgi_app
>>>>>
>>>>> app_rv = app(environ, start_response)
>>>>>
>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>>>>> line 1997, in __call__
>>>>>
>>>>> return self.wsgi_app(environ, start_response)
>>>>>
>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>>>>> line 1985, in wsgi_app
>>>>>
>>>>> response = self.handle_exception(e)
>>>>>
>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>>>>> line 1540, in handle_exception
>>>>>
>>>>> reraise(exc_type, exc_value, tb)
>>>>>
>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>>>>> line 1982, in wsgi_app
>>>>>
>>>>> response = self.full_dispatch_request()
>>>>>
>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>>>>> line 1614, in full_dispatch_request
>>>>>
>>>>> rv = self.handle_user_exception(e)
>>>>>
>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>>>>> line 1517, in handle_user_exception
>>>>>
>>>>> reraise(exc_type, exc_value, tb)
>>>>>
>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>>>>> line 1612, in full_dispatch_request
>>>>>
>>>>> rv = self.dispatch_request()
>>>>>
>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>>>>> line 1598, in dispatch_request
>>>>>
>>>>> return self.view_functions[rule.endpoint](**req.view_args)
>>>>>
>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask_login.py",
>>>>> line 792, in decorated_view
>>>>>
>>>>> return func(*args, **kwargs)
>>>>>
>>>>> File "/Users/dpage/git/pgadmin4/web/pgadmin/misc/bgprocess/__init__.py",
>>>>> line 62, in index
>>>>>
>>>>> return make_response(response=BatchProcess.list())
>>>>>
>>>>> File "/Users/dpage/git/pgadmin4/web/pgadmin/misc/bgprocess/processes.py",
>>>>> line 584, in list
>>>>>
>>>>> details = desc.details(p.command, args)
>>>>>
>>>>> File "/Users/dpage/git/pgadmin4/web/pgadmin/tools/backup/__init__.py",
>>>>> line 159, in details
>>>>>
>>>>> name, host, port = self.get_server_details()
>>>>>
>>>>> File "/Users/dpage/git/pgadmin4/web/pgadmin/tools/backup/__init__.py",
>>>>> line 122, in get_server_details
>>>>>
>>>>> host = manager.local_bind_host if manager.use_ssh_tunnel else
>>>>> s.host
>>>>>
>>>>> AttributeError: 'NoneType' object has no attribute 'use_ssh_tunnel'
>>>>>
>>>>>
>>>>> ======================================================================
>>>>>
>>>>> ERROR: runTest (pgadmin.tools.restore.tests.t
>>>>> est_create_restore_job.RestoreJobTest)
>>>>>
>>>>> When restore the object with the default options
>>>>>
>>>>> ----------------------------------------------------------------------
>>>>>
>>>>> Traceback (most recent call last):
>>>>>
>>>>> File "/Users/dpage/git/pgadmin4/web/pgadmin/tools/restore/tests/test_create_restore_job.py",
>>>>> line 95, in runTest
>>>>>
>>>>> self.create_backup()
>>>>>
>>>>> File "/Users/dpage/git/pgadmin4/web/pgadmin/tools/restore/tests/test_create_restore_job.py",
>>>>> line 86, in create_backup
>>>>>
>>>>> self.assertNotIn
>>>>>
>>>>> File "/Users/dpage/git/pgadmin4/web/pgadmin/tools/backup/tests/test_backup_utils.py",
>>>>> line 33, in run_backup_job
>>>>>
>>>>> random.randint(1, 9999999)))
>>>>>
>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/test.py",
>>>>> line 830, in get
>>>>>
>>>>> return self.open(*args, **kw)
>>>>>
>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/testing.py",
>>>>> line 127, in open
>>>>>
>>>>> follow_redirects=follow_redirects)
>>>>>
>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/test.py",
>>>>> line 803, in open
>>>>>
>>>>> response = self.run_wsgi_app(environ, buffered=buffered)
>>>>>
>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/test.py",
>>>>> line 716, in run_wsgi_app
>>>>>
>>>>> rv = run_wsgi_app(self.application, environ, buffered=buffered)
>>>>>
>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/test.py",
>>>>> line 923, in run_wsgi_app
>>>>>
>>>>> app_rv = app(environ, start_response)
>>>>>
>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>>>>> line 1997, in __call__
>>>>>
>>>>> return self.wsgi_app(environ, start_response)
>>>>>
>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>>>>> line 1985, in wsgi_app
>>>>>
>>>>> response = self.handle_exception(e)
>>>>>
>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>>>>> line 1540, in handle_exception
>>>>>
>>>>> reraise(exc_type, exc_value, tb)
>>>>>
>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>>>>> line 1982, in wsgi_app
>>>>>
>>>>> response = self.full_dispatch_request()
>>>>>
>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>>>>> line 1614, in full_dispatch_request
>>>>>
>>>>> rv = self.handle_user_exception(e)
>>>>>
>>>>>
>