Re: pgAdmin 4 commit: Support EXPLAIN on Greenplum. Fixes #3097

Hello,
Definitely running single tests is something that would be great, specially
if you are TDDing something waiting 30-40 seconds to get feedback is a
little cumbersome when the test you are concerned with take less then a
second.

In the process:
1. Write a test
2. Make the test pass
3. Refactor

in between each step you run the test more then 1 time, and depending on
the refactoring you might need to run it several times. So imagine waiting
30 seconds per run to get results. To run a subset of tests is a pain
because you need to be always changing the way you run the tests.....

I believe we could archive a better granularity and choosing what test to
run if we used a runner like pytest or nose to do it. What was the reason
behind handrolling a test runner script? I am asking this because in a
previous job I decided to handroll a unittest loader script and that was
something that I regretted every time I had to touch it, and eventually was
in the process of changing it to pytest.

I looked into pytest to replace the current the current runtest, and the
major problem I found was the testscenarios integration(See Note 1). It can
be done but we would need to change all the test functions to receive the
scenario variables through arguments on the function. Also didn't dug much
into setting all the variables that we need there and all the environment.
The other issue that I do not like very much about pytest is the fact that
you loose the unittest assertion that is not so bad because there are some
neat libraries like: https://github.com/grappa-py/grappa,
https://github.com/ActivisionGameScience/assertpy,
https://github.com/dgilland/verify. Personally I really like the syntax of
Grapa, but the Veridfy one is pretty similar to Jasmine too.

What are your thoughts?

Note 1: As an example of what our functions would have to look like you can
see:
https://github.com/OriMenashe/pytest-scenario/blob/master/tests/test_parametrize.py
As a example this class:

class ServersWithServiceIDAddTestCase(BaseTestGenerator):
""" This class will add the servers under default server group. """

scenarios = [
# Fetch the default url for server object
(
'Default Server Node url', dict(
url='/browser/server/obj/'
)
)
]

def setUp(self):
pass

def runTest(self):
""" This function will add the server under default server group."""
url = "{0}{1}/".format(self.url, utils.SERVER_GROUP)
# Add service name in the config
self.server['service'] = "TestDB"
response = self.tester.post(
url,
data=json.dumps(self.server),
content_type='html/json'
)
self.assertEquals(response.status_code, 200)
response_data = json.loads(response.data.decode('utf-8'))
self.server_id = response_data['node']['_id']

def tearDown(self):
"""This function delete the server from SQLite """
utils.delete_server_with_api(self.tester, self.server_id)

Would have to look changed to:

class ServersWithServiceIDAddTestCase(object):
""" This class will add the servers under default server group. """

scenarios = [
# Fetch the default url for server object
(
'Default Server Node url', dict(
url='/browser/server/obj/'
)
)
]

def setUp(self):
pass

def runTest(self, url):
""" This function will add the server under default server group."""
url = "{0}{1}/".format(url, utils.SERVER_GROUP)
# Add service name in the config
self.server['service'] = "TestDB"
response = self.tester.post(
url,
data=json.dumps(self.server),
content_type='html/json'
)
self.assertEquals(response.status_code, 200)
response_data = json.loads(response.data.decode('utf-8'))
self.server_id = response_data['node']['_id']

def tearDown(self):
"""This function delete the server from SQLite """
utils.delete_server_with_api(self.tester, self.server_id)

Thanks
Joao

On Fri, Mar 9, 2018 at 8:31 AM Dave Page <dpage(at)pgadmin(dot)org> wrote:

> On Thu, Mar 8, 2018 at 2:22 PM, Joao De Almeida Pereira <
> jdealmeidapereira(at)pivotal(dot)io> wrote:
>
>> Hello Khushboo,
>> Completely forgot about this python "feature".......
>> Attached is the fix.
>>
>
> Thanks, applied.
>
>
>>
>> Just as a side question, does anyone else feel the pain of wanting to run
>> a single test using a IDE or the command line and not being able to?
>>
>
> Not really - the Python and JS tests are so quick I don't really care (and
> with the Python ones, I can execute for a single module for even more
> speed).
>
> What I would *really* like, is the ability to run individual feature
> tests. That would be very valuable and save me a ton of time.
>
>
>
>> We an HandRolled the loader, and that as some implications. Did anyone
>> try to use a different launcher like pytest or nose instead of the current
>> runner?
>> I understand that testscenarios is one of the problems we have if we want
>> to move away from this way of running tests.
>> Any suggestion?
>>
>> Thanks
>> Joao
>>
>> On Wed, Mar 7, 2018 at 11:41 PM Khushboo Vashi <
>> khushboo(dot)vashi(at)enterprisedb(dot)com> wrote:
>>
>>> Hi Joao,
>>>
>>> In the test_start_running_query.py, 2 static methods (is_begin_required_for_sql_query
>>> and is_rollback_statement_required)
>>> of StartRunningQuery class were used directly without @patch. Due to
>>> this, in all the cases, the original value of them doesn't restore.
>>>
>>> To fix this, I have sent the patch in another thread, to restore its
>>> original state, but I wonder if we can use these methods with @patch.
>>>
>>> Thanks,
>>> Khushboo
>>>
>>>
>>> On Fri, Feb 9, 2018 at 5:24 PM, Dave Page <dpage(at)pgadmin(dot)org> wrote:
>>>
>>>> Support EXPLAIN on Greenplum. Fixes #3097
>>>>
>>>> - Extract SQLEditor.execute and SQLEditor._poll into their own files
>>>> and add test around them
>>>> - Extract SQLEditor backend functions that start executing query to
>>>> their own files and add tests around it
>>>> - Move the Explain SQL from the front-end and now pass the Explain
>>>> plan parameters as a JSON object in the start query call.
>>>> - Extract the compile_template_name into a function that can be used
>>>> by the different places that try to select the version of the template and
>>>> the server type
>>>>
>>>> Branch
>>>> ------
>>>> master
>>>>
>>>> Details
>>>> -------
>>>>
>>>> https://git.postgresql.org/gitweb?p=pgadmin4.git;a=commitdiff;h=e16a95275336529a734bf0066889e39cc8ef0662
>>>> Author: Joao Pedro De Almeida Pereira <jdealmeidapereira(at)pivotal(dot)io>
>>>>
>>>> Modified Files
>>>> --------------
>>>> .../databases/schemas/tables/tests/test_utils.py | 0
>>>> web/pgadmin/static/js/sqleditor/execute_query.js | 287 ++++
>>>> .../js/sqleditor/is_new_transaction_required.js | 14 +
>>>> .../static/js/sqleditor/query_tool_actions.js | 33 +-
>>>> web/pgadmin/tools/sqleditor/__init__.py | 396 +----
>>>> web/pgadmin/tools/sqleditor/static/js/sqleditor.js | 227 +--
>>>> .../sqleditor/sql/10_plus/explain_plan.sql | 23 +
>>>> .../sqleditor/sql/9.2_plus/explain_plan.sql | 20 +
>>>> .../sqleditor/sql/default/explain_plan.sql | 17 +
>>>> .../sqleditor/sql/gpdb_5.0_plus/explain_plan.sql | 5 +
>>>> web/pgadmin/tools/sqleditor/tests/__init__.py | 8 +
>>>> .../sqleditor/tests/test_explain_plan_templates.py | 152 ++
>>>> .../test_extract_sql_from_network_parameters.py | 59 +
>>>> .../tools/sqleditor/tests/test_start_query_tool.py | 38 +
>>>> web/pgadmin/tools/sqleditor/utils/__init__.py | 14 +
>>>> .../sqleditor/utils/apply_explain_plan_wrapper.py | 24 +
>>>> .../tools/sqleditor/utils/constant_definition.py | 32 +
>>>> .../tools/sqleditor/utils/is_begin_required.py | 169 ++
>>>> .../tools/sqleditor/utils/start_running_query.py | 172 ++
>>>> .../tools/sqleditor/utils/tests/__init__.py | 8 +
>>>> .../utils/tests/test_apply_explain_plan_wrapper.py | 121 ++
>>>> .../utils/tests/test_start_running_query.py | 445 +++++
>>>> .../utils/update_session_grid_transaction.py | 18 +
>>>> web/pgadmin/utils/compile_template_name.py | 17 +
>>>> .../utils/tests/test_compile_template_name.py | 34 +
>>>> web/pgadmin/utils/versioned_template_loader.py | 2 +-
>>>> web/regression/javascript/fake_endpoints.js | 6 +-
>>>> .../javascript/sqleditor/execute_query_spec.js | 1702
>>>> ++++++++++++++++++++
>>>> .../sqleditor/is_new_transaction_required_spec.js | 65 +
>>>> .../sqleditor/query_tool_actions_spec.js | 141 +-
>>>> 30 files changed, 3670 insertions(+), 579 deletions(-)
>>>>
>>>>
>>>
>
>
> --
> Dave Page
> Blog: http://pgsnake.blogspot.com
> Twitter: @pgsnake
>
> EnterpriseDB UK: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>