Re: [pgadmin4][Patch]: Test cases for the backup module

Thanks, applied with a few tweaks to remove some unused imports and a
couple of variables.

On Fri, Jun 15, 2018 at 8:13 AM, Khushboo Vashi <
khushboo(dot)vashi(at)enterprisedb(dot)com> wrote:

> Hi,
>
> Please find the attached updated patch.
>
> On Wed, Jun 13, 2018 at 5:42 PM, Khushboo Vashi <
> khushboo(dot)vashi(at)enterprisedb(dot)com> wrote:
>
>> Hi Victoria,
>>
>>
>> On Tue, Jun 12, 2018 at 9:06 PM, Victoria Henry <vhenry(at)pivotal(dot)io>
>> wrote:
>>
>>> Hi Khushboo
>>> can you explain a little bit more this while loop?
>>>
>>>
>>> cnt = 0
>>> while 1:
>>> if cnt > 1:
>>> break
>>> # Check the process list
>>> response1 = self.tester.get('/misc/bgprocess/?_='.format(
>>> random.randint(1, 9999999)))
>>> self.assertEquals(response1.status_code, 200)
>>> process_list = json.loads(response1.data.decode('utf-8'))
>>>
>>> if len(process_list) > 0 and 'execution_time' in process_list[0]:
>>> break
>>> time.sleep(0.5)
>>> cnt += 1
>>>
>>> From what it looks like this will only run twice, maybe a for would be a
>>> better solution because we know it will only run twice. Also are we sure we
>>> only want it to run twice?
>>>
>>> The code waits till the background process completes. So, while I
>> originally developed, not intended to run only twice.
>> But after that I put a kind of break point and that remains there. So, I
>> will remove that if condition which is not required.
>>
> I have made maximum 5 attempts.
>
>> We are using PyCharm to do our developments and we notice there are a big
>>> group of unused variables throughout. We should remove them if they are not
>>> needed. Not sure if your editor also shows that information or not.
>>>
>> Do you know if there is a configuration in pycodestyle to enable the
>>> check for unused variables? That would help a lot.
>>>
>>> Removed unused local variables.
>
>> Thanks for sharing the information.
>>
>>> The code
>>>
>>> assert 'execution_time' in process_list[0]
>>> assert 'stime' in process_list[0]
>>> assert 'exit_code' in process_list[0]
>>> assert process_list[0]['exit_code'] in self.expected_exit_code
>>>
>>> in test_Create_restore_job should use self.assertEqual or similar from
>>> unittest instead of plain assert. Because when something fails we do
>>> not have a way to understand what was wrong.
>>>
>> Will do.
>>
> Done
>
>> The tests on the restore are still failing GreenPlum.
>>>
>> It is failing because, Restore is not working with GreenPlum. Can you
>> please look into the Restore functionality for GPDB?
>>
>>
>> Thanks,
>> Khushboo
>>
>>> ​
>>>
>>>
>>> Thanks,
> Khushboo
>
>>
>>> Thanks
>>> Victoria & Joao
>>>
>>> On Tue, Jun 12, 2018 at 6:44 AM Khushboo Vashi <
>>> khushboo(dot)vashi(at)enterprisedb(dot)com> wrote:
>>>
>>>> Please find the attached updated patch with some code cleanup.
>>>>
>>>> On Tue, Jun 12, 2018 at 3:54 PM, Khushboo Vashi <
>>>> khushboo(dot)vashi(at)enterprisedb(dot)com> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> Please find the attached patch excluding feature test cases.
>>>>> Python test cases are working fine, so we can commit this patch. I am
>>>>> working on fixing the feature tests which are failing on the different
>>>>> window sizes.
>>>>>
>>>>> Thanks,
>>>>> Khushboo
>>>>>
>>>>> On Fri, Jun 8, 2018 at 2:38 PM, Dave Page <dpage(at)pgadmin(dot)org> wrote:
>>>>>
>>>>>> Hi
>>>>>>
>>>>>> On Fri, Jun 8, 2018 at 6:33 AM, Khushboo Vashi <
>>>>>> khushboo(dot)vashi(at)enterprisedb(dot)com> wrote:
>>>>>>
>>>>>>> Hi Dave,
>>>>>>>
>>>>>>> As per our discussion I have changed the window size to 1280X800,
>>>>>>> before it was 1280X900.
>>>>>>> Please find the attached updated patch
>>>>>>>
>>>>>>
>>>>>> I'm not sure that actually made any difference on my system. The
>>>>>> window continued to look taller than it is wide, so I wonder if the code to
>>>>>> set the size is being ignored, or is at the wrong place?
>>>>>>
>>>>>> Anyway, I got 10 failures with this patch :-(
>>>>>>
>>>>>> ============================================================
>>>>>> ==========
>>>>>>
>>>>>> ERROR: runTest (pgadmin.feature_tests.pg_util
>>>>>> ities_backup_restore_test.PGUtilitiesBackupFeatureTest)
>>>>>>
>>>>>> Test for PG utilities - Backup and Restore
>>>>>>
>>>>>> ------------------------------------------------------------
>>>>>> ----------
>>>>>>
>>>>>> Traceback (most recent call last):
>>>>>>
>>>>>> File "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/pg_utilities_backup_restore_test.py",
>>>>>> line 97, in runTest
>>>>>>
>>>>>> self.page.find_by_xpath("//div[contains(@class,'wcFloatingFo
>>>>>> cus')"
>>>>>>
>>>>>> File "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgadmin_page.py",
>>>>>> line 171, in find_by_xpath
>>>>>>
>>>>>> lambda driver: driver.find_element_by_xpath(xpath)
>>>>>>
>>>>>> File "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgadmin_page.py",
>>>>>> line 263, in wait_for_element
>>>>>>
>>>>>> return self._wait_for("element to exist", element_if_it_exists)
>>>>>>
>>>>>> File "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgadmin_page.py",
>>>>>> line 337, in _wait_for
>>>>>>
>>>>>> "Timed out waiting for " + waiting_for_message
>>>>>>
>>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa
>>>>>> ges/selenium/webdriver/support/wait.py", line 80, in until
>>>>>>
>>>>>> raise TimeoutException(message, screen, stacktrace)
>>>>>>
>>>>>> TimeoutException: Message: Timed out waiting for element to exist
>>>>>>
>>>>>>
>>>>>>
>>>>>> ============================================================
>>>>>> ==========
>>>>>>
>>>>>> ERROR: runTest (pgadmin.feature_tests.xss_che
>>>>>> cks_pgadmin_debugger_test.CheckDebuggerForXssFeatureTest)
>>>>>>
>>>>>> Tests to check if Debugger is vulnerable to XSS
>>>>>>
>>>>>> ------------------------------------------------------------
>>>>>> ----------
>>>>>>
>>>>>> Traceback (most recent call last):
>>>>>>
>>>>>> File "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/xss_checks_pgadmin_debugger_test.py",
>>>>>> line 42, in runTest
>>>>>>
>>>>>> self._function_node_expandable()
>>>>>>
>>>>>> File "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/xss_checks_pgadmin_debugger_test.py",
>>>>>> line 57, in _function_node_expandable
>>>>>>
>>>>>> self.page.select_tree_item("a_test_function()")
>>>>>>
>>>>>> File "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgadmin_page.py",
>>>>>> line 135, in select_tree_item
>>>>>>
>>>>>> "' and @class='aciTreeItem']").click()
>>>>>>
>>>>>> File "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgadmin_page.py",
>>>>>> line 171, in find_by_xpath
>>>>>>
>>>>>> lambda driver: driver.find_element_by_xpath(xpath)
>>>>>>
>>>>>> File "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgadmin_page.py",
>>>>>> line 263, in wait_for_element
>>>>>>
>>>>>> return self._wait_for("element to exist", element_if_it_exists)
>>>>>>
>>>>>> File "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgadmin_page.py",
>>>>>> line 337, in _wait_for
>>>>>>
>>>>>> "Timed out waiting for " + waiting_for_message
>>>>>>
>>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa
>>>>>> ges/selenium/webdriver/support/wait.py", line 80, in until
>>>>>>
>>>>>> raise TimeoutException(message, screen, stacktrace)
>>>>>>
>>>>>> TimeoutException: Message: Timed out waiting for element to exist
>>>>>>
>>>>>>
>>>>>>
>>>>>> ============================================================
>>>>>> ==========
>>>>>>
>>>>>> ERROR: runTest (pgadmin.tools.backup.tests.te
>>>>>> st_create_backup_job.BackupJobTest)
>>>>>>
>>>>>> When backup the object with the default options
>>>>>>
>>>>>> ------------------------------------------------------------
>>>>>> ----------
>>>>>>
>>>>>> Traceback (most recent call last):
>>>>>>
>>>>>> File "/Users/dpage/git/pgadmin4/web/pgadmin/tools/backup/tests/test_create_backup_job.py",
>>>>>> line 58, in runTest
>>>>>>
>>>>>> self.assertNotIn
>>>>>>
>>>>>> File "/Users/dpage/git/pgadmin4/web/pgadmin/tools/backup/tests/test_backup_utils.py",
>>>>>> line 33, in run_backup_job
>>>>>>
>>>>>> random.randint(1, 9999999)))
>>>>>>
>>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/test.py",
>>>>>> line 830, in get
>>>>>>
>>>>>> return self.open(*args, **kw)
>>>>>>
>>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/testing.py",
>>>>>> line 127, in open
>>>>>>
>>>>>> follow_redirects=follow_redirects)
>>>>>>
>>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/test.py",
>>>>>> line 803, in open
>>>>>>
>>>>>> response = self.run_wsgi_app(environ, buffered=buffered)
>>>>>>
>>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/test.py",
>>>>>> line 716, in run_wsgi_app
>>>>>>
>>>>>> rv = run_wsgi_app(self.application, environ, buffered=buffered)
>>>>>>
>>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/test.py",
>>>>>> line 923, in run_wsgi_app
>>>>>>
>>>>>> app_rv = app(environ, start_response)
>>>>>>
>>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>>>>>> line 1997, in __call__
>>>>>>
>>>>>> return self.wsgi_app(environ, start_response)
>>>>>>
>>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>>>>>> line 1985, in wsgi_app
>>>>>>
>>>>>> response = self.handle_exception(e)
>>>>>>
>>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>>>>>> line 1540, in handle_exception
>>>>>>
>>>>>> reraise(exc_type, exc_value, tb)
>>>>>>
>>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>>>>>> line 1982, in wsgi_app
>>>>>>
>>>>>> response = self.full_dispatch_request()
>>>>>>
>>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>>>>>> line 1614, in full_dispatch_request
>>>>>>
>>>>>> rv = self.handle_user_exception(e)
>>>>>>
>>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>>>>>> line 1517, in handle_user_exception
>>>>>>
>>>>>> reraise(exc_type, exc_value, tb)
>>>>>>
>>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>>>>>> line 1612, in full_dispatch_request
>>>>>>
>>>>>> rv = self.dispatch_request()
>>>>>>
>>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>>>>>> line 1598, in dispatch_request
>>>>>>
>>>>>> return self.view_functions[rule.endpoint](**req.view_args)
>>>>>>
>>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask_login.py",
>>>>>> line 792, in decorated_view
>>>>>>
>>>>>> return func(*args, **kwargs)
>>>>>>
>>>>>> File "/Users/dpage/git/pgadmin4/web/pgadmin/misc/bgprocess/__init__.py",
>>>>>> line 62, in index
>>>>>>
>>>>>> return make_response(response=BatchProcess.list())
>>>>>>
>>>>>> File "/Users/dpage/git/pgadmin4/web/pgadmin/misc/bgprocess/processes.py",
>>>>>> line 584, in list
>>>>>>
>>>>>> details = desc.details(p.command, args)
>>>>>>
>>>>>> File "/Users/dpage/git/pgadmin4/web/pgadmin/tools/backup/__init__.py",
>>>>>> line 159, in details
>>>>>>
>>>>>> name, host, port = self.get_server_details()
>>>>>>
>>>>>> File "/Users/dpage/git/pgadmin4/web/pgadmin/tools/backup/__init__.py",
>>>>>> line 122, in get_server_details
>>>>>>
>>>>>> host = manager.local_bind_host if manager.use_ssh_tunnel else
>>>>>> s.host
>>>>>>
>>>>>> AttributeError: 'NoneType' object has no attribute 'use_ssh_tunnel'
>>>>>>
>>>>>>
>>>>>> ============================================================
>>>>>> ==========
>>>>>>
>>>>>> ERROR: runTest (pgadmin.tools.maintenance.tes
>>>>>> ts.test_create_maintenance_job.MaintenanceJobTest)
>>>>>>
>>>>>> When maintenance the object with the default options
>>>>>>
>>>>>> ------------------------------------------------------------
>>>>>> ----------
>>>>>>
>>>>>> Traceback (most recent call last):
>>>>>>
>>>>>> File "/Users/dpage/git/pgadmin4/web/pgadmin/tools/maintenance/tes
>>>>>> ts/test_create_maintenance_job.py", line 71, in runTest
>>>>>>
>>>>>> random.randint(1, 9999999)))
>>>>>>
>>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/test.py",
>>>>>> line 830, in get
>>>>>>
>>>>>> return self.open(*args, **kw)
>>>>>>
>>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/testing.py",
>>>>>> line 127, in open
>>>>>>
>>>>>> follow_redirects=follow_redirects)
>>>>>>
>>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/test.py",
>>>>>> line 803, in open
>>>>>>
>>>>>> response = self.run_wsgi_app(environ, buffered=buffered)
>>>>>>
>>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/test.py",
>>>>>> line 716, in run_wsgi_app
>>>>>>
>>>>>> rv = run_wsgi_app(self.application, environ, buffered=buffered)
>>>>>>
>>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/test.py",
>>>>>> line 923, in run_wsgi_app
>>>>>>
>>>>>> app_rv = app(environ, start_response)
>>>>>>
>>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>>>>>> line 1997, in __call__
>>>>>>
>>>>>> return self.wsgi_app(environ, start_response)
>>>>>>
>>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>>>>>> line 1985, in wsgi_app
>>>>>>
>>>>>> response = self.handle_exception(e)
>>>>>>
>>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>>>>>> line 1540, in handle_exception
>>>>>>
>>>>>> reraise(exc_type, exc_value, tb)
>>>>>>
>>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>>>>>> line 1982, in wsgi_app
>>>>>>
>>>>>> response = self.full_dispatch_request()
>>>>>>
>>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>>>>>> line 1614, in full_dispatch_request
>>>>>>
>>>>>> rv = self.handle_user_exception(e)
>>>>>>
>>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>>>>>> line 1517, in handle_user_exception
>>>>>>
>>>>>> reraise(exc_type, exc_value, tb)
>>>>>>
>>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>>>>>> line 1612, in full_dispatch_request
>>>>>>
>>>>>> rv = self.dispatch_request()
>>>>>>
>>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>>>>>> line 1598, in dispatch_request
>>>>>>
>>>>>> return self.view_functions[rule.endpoint](**req.view_args)
>>>>>>
>>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask_login.py",
>>>>>> line 792, in decorated_view
>>>>>>
>>>>>> return func(*args, **kwargs)
>>>>>>
>>>>>> File "/Users/dpage/git/pgadmin4/web/pgadmin/misc/bgprocess/__init__.py",
>>>>>> line 62, in index
>>>>>>
>>>>>> return make_response(response=BatchProcess.list())
>>>>>>
>>>>>> File "/Users/dpage/git/pgadmin4/web/pgadmin/misc/bgprocess/processes.py",
>>>>>> line 584, in list
>>>>>>
>>>>>> details = desc.details(p.command, args)
>>>>>>
>>>>>> File "/Users/dpage/git/pgadmin4/web/pgadmin/tools/backup/__init__.py",
>>>>>> line 159, in details
>>>>>>
>>>>>> name, host, port = self.get_server_details()
>>>>>>
>>>>>> File "/Users/dpage/git/pgadmin4/web/pgadmin/tools/backup/__init__.py",
>>>>>> line 122, in get_server_details
>>>>>>
>>>>>> host = manager.local_bind_host if manager.use_ssh_tunnel else
>>>>>> s.host
>>>>>>
>>>>>> AttributeError: 'NoneType' object has no attribute 'use_ssh_tunnel'
>>>>>>
>>>>>>
>>>>>> ============================================================
>>>>>> ==========
>>>>>>
>>>>>> ERROR: runTest (pgadmin.tools.restore.tests.t
>>>>>> est_create_restore_job.RestoreJobTest)
>>>>>>
>>>>>> When restore the object with the default options
>>>>>>
>>>>>> ------------------------------------------------------------
>>>>>> ----------
>>>>>>
>>>>>> Traceback (most recent call last):
>>>>>>
>>>>>> File "/Users/dpage/git/pgadmin4/web/pgadmin/tools/restore/tests/test_create_restore_job.py",
>>>>>> line 95, in runTest
>>>>>>
>>>>>> self.create_backup()
>>>>>>
>>>>>> File "/Users/dpage/git/pgadmin4/web/pgadmin/tools/restore/tests/test_create_restore_job.py",
>>>>>> line 86, in create_backup
>>>>>>
>>>>>> self.assertNotIn
>>>>>>
>>>>>> File "/Users/dpage/git/pgadmin4/web/pgadmin/tools/backup/tests/test_backup_utils.py",
>>>>>> line 33, in run_backup_job
>>>>>>
>>>>>> random.randint(1, 9999999)))
>>>>>>
>>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/test.py",
>>>>>> line 830, in get
>>>>>>
>>>>>> return self.open(*args, **kw)
>>>>>>
>>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/testing.py",
>>>>>> line 127, in open
>>>>>>
>>>>>> follow_redirects=follow_redirects)
>>>>>>
>>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/test.py",
>>>>>> line 803, in open
>>>>>>
>>>>>> response = self.run_wsgi_app(environ, buffered=buffered)
>>>>>>
>>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/test.py",
>>>>>> line 716, in run_wsgi_app
>>>>>>
>>>>>> rv = run_wsgi_app(self.application, environ, buffered=buffered)
>>>>>>
>>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/test.py",
>>>>>> line 923, in run_wsgi_app
>>>>>>
>>>>>> app_rv = app(environ, start_response)
>>>>>>
>>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>>>>>> line 1997, in __call__
>>>>>>
>>>>>> return self.wsgi_app(environ, start_response)
>>>>>>
>>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>>>>>> line 1985, in wsgi_app
>>>>>>
>>>>>> response = self.handle_exception(e)
>>>>>>
>>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>>>>>> line 1540, in handle_exception
>>>>>>
>>>>>> reraise(exc_type, exc_value, tb)
>>>>>>
>>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>>>>>> line 1982, in wsgi_app
>>>>>>
>>>>>> response = self.full_dispatch_request()
>>>>>>
>>>>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>>>>>> line 1614, in full_dispatch_request
>>>>>>
>>>>>> rv = self.handle_user_exception(e)
>>>>>>
>>>>>>
>>
>

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company