Re: [GSoC][New Feature] Editable and Read-only Columns

Hi Dave,

I am now using the same versions of Python, Chrome and chromedriver. The
tests still pass every time. Are you sure you applied the latest patch? I
re-created the patch as a sanity check, please find it attached.

I tried running the specific test a lot of times, passes every time. The
only difference is that I am running Ubuntu, otherwise, everything is the
same.

On Fri, Aug 23, 2019 at 5:00 PM Dave Page <dpage(at)pgadmin(dot)org> wrote:

> Hi
>
> I've run it again 4 more times and it fails consistently for me. I'm
> running on macOS 10.14.5 with Chrome 76.0.3809.100.
>
> On Fri, Aug 23, 2019 at 3:17 PM Yosry Muhammad <yosrym93(at)gmail(dot)com> wrote:
>
>> Hi Dave,
>>
>> The tests pass on my device normally. I am using Python 3.6 on Ubuntu. I
>> have ran all tests using "make check" and they all succeeded. I also ran
>> feature_tests/query_tool_journey_test.py multiple times on its own using
>> "cd web && python regression/runtests.py --pkg feature_tests --modules
>> query_tool_journey_test" and it passed successfully. Could you try again?
>>
>> Generally, I have found that StaleElementReferenceException occurs
>> intermittently on various feature tests.
>>
>> On Fri, Aug 23, 2019 at 3:46 PM Dave Page <dpage(at)pgadmin(dot)org> wrote:
>>
>>> Thanks. Unfortunately this is failing the tests:
>>>
>>> ======================================================================
>>> ERROR: runTest
>>> (pgadmin.feature_tests.query_tool_journey_test.QueryToolJourneyTest)
>>> Tests the path through the query tool
>>> ----------------------------------------------------------------------
>>> Traceback (most recent call last):
>>> File
>>> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_journey_test.py",
>>> line 85, in runTest
>>> self._test_updatable_resultset()
>>> File
>>> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_journey_test.py",
>>> line 254, in _test_updatable_resultset
>>> discard_changes_modal=True)
>>> File
>>> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_journey_test.py",
>>> line 417, in _check_query_results_editable
>>> is_editable = self._check_cell_editable(column_index)
>>> File
>>> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_journey_test.py",
>>> line 430, in _check_cell_editable
>>> cell_value = int(cell_el.text)
>>> File
>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python3.7/site-packages/selenium/webdriver/remote/webelement.py",
>>> line 76, in text
>>> return self._execute(Command.GET_ELEMENT_TEXT)['value']
>>> File
>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python3.7/site-packages/selenium/webdriver/remote/webelement.py",
>>> line 628, in _execute
>>> return self._parent.execute(command, params)
>>> File
>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py",
>>> line 312, in execute
>>> self.error_handler.check_response(response)
>>> File
>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python3.7/site-packages/selenium/webdriver/remote/errorhandler.py",
>>> line 242, in check_response
>>> raise exception_class(message, screen, stacktrace)
>>> selenium.common.exceptions.StaleElementReferenceException: Message:
>>> stale element reference: element is not attached to the page document
>>> (Session info: chrome=76.0.3809.100)
>>> (Driver info: chromedriver=76.0.3809.126
>>> (d80a294506b4c9d18015e755cee48f953ddc3f2f-refs/branch-heads/3809(at){#1024}),platform=Mac
>>> OS X 10.14.5 x86_64)
>>>
>>> Without the patch, the tests pass.
>>>
>>>
>>> On Fri, Aug 23, 2019 at 1:10 PM Yosry Muhammad <yosrym93(at)gmail(dot)com>
>>> wrote:
>>>
>>>> Please find an updated patch attached.
>>>>
>>>> On Fri, Aug 23, 2019 at 1:22 PM Dave Page <dpage(at)pgadmin(dot)org> wrote:
>>>>
>>>>> Except it now doesn't apply as the async connection/rendering patch
>>>>> was in the queue right ahead of it :-(.
>>>>>
>>>>> Can I get a rebased patch please?
>>>>>
>>>>> On Fri, Aug 23, 2019 at 11:57 AM Aditya Toshniwal <
>>>>> aditya(dot)toshniwal(at)enterprisedb(dot)com> wrote:
>>>>>
>>>>>> Excellent !! Looks good to me.
>>>>>>
>>>>>> On Fri, Aug 23, 2019 at 2:23 PM Yosry Muhammad <yosrym93(at)gmail(dot)com>
>>>>>> wrote:
>>>>>>
>>>>>>> I updated the patch for backwards compatibility anyway.
>>>>>>>
>>>>>>> On Fri, Aug 23, 2019 at 10:45 AM Dave Page <dpage(at)pgadmin(dot)org>
>>>>>>> wrote:
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Fri, Aug 23, 2019 at 9:43 AM Yosry Muhammad <yosrym93(at)gmail(dot)com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi Aditya,
>>>>>>>>>
>>>>>>>>> On Fri, Aug 23, 2019 at 7:33 AM Aditya Toshniwal <
>>>>>>>>> aditya(dot)toshniwal(at)enterprisedb(dot)com> wrote:
>>>>>>>>>
>>>>>>>>>> Hi Yosry,
>>>>>>>>>>
>>>>>>>>>> The previously mentioned issues seems to be fixed. Below are few
>>>>>>>>>> issues:
>>>>>>>>>> 1) On Python 2.7, I get below error when opening query tool:
>>>>>>>>>> 2019-08-23 10:49:09,329: ERROR flask.app: Object of type buffer
>>>>>>>>>> is not JSON serializable
>>>>>>>>>> Traceback (most recent call last):
>>>>>>>>>> File
>>>>>>>>>> "/Users/adityatoshniwal/venv/pypg27/lib/python2.7/site-packages/flask/app.py",
>>>>>>>>>> line 1813, in full_dispatch_request
>>>>>>>>>> rv = self.dispatch_request()
>>>>>>>>>> File
>>>>>>>>>> "/Users/adityatoshniwal/venv/pypg27/lib/python2.7/site-packages/flask/app.py",
>>>>>>>>>> line 1799, in dispatch_request
>>>>>>>>>> return self.view_functions[rule.endpoint](**req.view_args)
>>>>>>>>>> File
>>>>>>>>>> "/Users/adityatoshniwal/venv/pypg27/lib/python2.7/site-packages/flask_login/utils.py",
>>>>>>>>>> line 261, in decorated_view
>>>>>>>>>> return func(*args, **kwargs)
>>>>>>>>>> File
>>>>>>>>>> "/Users/adityatoshniwal/projects/pgadmin4/web/pgadmin/tools/sqleditor/__init__.py",
>>>>>>>>>> line 1544, in get_query_history
>>>>>>>>>> return QueryHistory.get(current_user.id, trans_obj.sid,
>>>>>>>>>> conn.db)
>>>>>>>>>> File
>>>>>>>>>> "/Users/adityatoshniwal/projects/pgadmin4/web/pgadmin/tools/sqleditor/utils/query_history.py",
>>>>>>>>>> line 21, in get
>>>>>>>>>> 'result': [rec.query_info for rec in result]
>>>>>>>>>> File
>>>>>>>>>> "/Users/adityatoshniwal/projects/pgadmin4/web/pgadmin/utils/ajax.py", line
>>>>>>>>>> 75, in make_json_response
>>>>>>>>>> separators=(',', ':'), encoding=encoding),
>>>>>>>>>> File
>>>>>>>>>> "/Users/adityatoshniwal/venv/pypg27/lib/python2.7/site-packages/simplejson/__init__.py",
>>>>>>>>>> line 399, in dumps
>>>>>>>>>> **kw).encode(obj)
>>>>>>>>>> File
>>>>>>>>>> "/Users/adityatoshniwal/venv/pypg27/lib/python2.7/site-packages/simplejson/encoder.py",
>>>>>>>>>> line 296, in encode
>>>>>>>>>> chunks = self.iterencode(o, _one_shot=True)
>>>>>>>>>> File
>>>>>>>>>> "/Users/adityatoshniwal/venv/pypg27/lib/python2.7/site-packages/simplejson/encoder.py",
>>>>>>>>>> line 378, in iterencode
>>>>>>>>>> return _iterencode(o, 0)
>>>>>>>>>> File
>>>>>>>>>> "/Users/adityatoshniwal/projects/pgadmin4/web/pgadmin/utils/ajax.py", line
>>>>>>>>>> 30, in default
>>>>>>>>>> return json.JSONEncoder.default(self, obj)
>>>>>>>>>> File
>>>>>>>>>> "/Users/adityatoshniwal/venv/pypg27/lib/python2.7/site-packages/simplejson/encoder.py",
>>>>>>>>>> line 273, in default
>>>>>>>>>> o.__class__.__name__)
>>>>>>>>>> TypeError: Object of type buffer is not JSON serializable
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> This error exists on the master branch on Python 2.7, it is not
>>>>>>>>> caused by the patch. It is also not a regression of my previous Query
>>>>>>>>> History Enhancement patch, I checked.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> 2) The patch is not compatible with older pyscopg2 (2.7.7). I get
>>>>>>>>>> below error in View/Edit data.
>>>>>>>>>> [image: Screenshot 2019-08-23 at 10.53.30.png]
>>>>>>>>>> 2019-08-23 10:53:12,020: ERROR flask.app: 'table_column'
>>>>>>>>>> Traceback (most recent call last):
>>>>>>>>>> File
>>>>>>>>>> "/Users/adityatoshniwal/venv/pypg35/lib/python3.5/site-packages/flask/app.py",
>>>>>>>>>> line 1813, in full_dispatch_request
>>>>>>>>>> rv = self.dispatch_request()
>>>>>>>>>> File
>>>>>>>>>> "/Users/adityatoshniwal/venv/pypg35/lib/python3.5/site-packages/flask/app.py",
>>>>>>>>>> line 1799, in dispatch_request
>>>>>>>>>> return self.view_functions[rule.endpoint](**req.view_args)
>>>>>>>>>> File
>>>>>>>>>> "/Users/adityatoshniwal/venv/pypg35/lib/python3.5/site-packages/flask_login/utils.py",
>>>>>>>>>> line 261, in decorated_view
>>>>>>>>>> return func(*args, **kwargs)
>>>>>>>>>> File
>>>>>>>>>> "/Users/adityatoshniwal/projects/pgadmin4/web/pgadmin/tools/sqleditor/__init__.py",
>>>>>>>>>> line 435, in poll
>>>>>>>>>> columns = trans_obj.get_columns_types(conn)
>>>>>>>>>> File
>>>>>>>>>> "/Users/adityatoshniwal/projects/pgadmin4/web/pgadmin/tools/sqleditor/command.py",
>>>>>>>>>> line 688, in get_columns_types
>>>>>>>>>> table_oid=table_oid)
>>>>>>>>>> File
>>>>>>>>>> "/Users/adityatoshniwal/projects/pgadmin4/web/pgadmin/tools/sqleditor/utils/get_column_types.py",
>>>>>>>>>> line 38, in get_columns_types
>>>>>>>>>> if row['oid'] == col['table_column']:
>>>>>>>>>> KeyError: 'table_column'
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> I recall Dave mentioning that it is okay to use psycopg 2.8+
>>>>>>>>> features and that it was a requirement of pgAdmin 4 now (clearly mentioned
>>>>>>>>> in requirements.txt too).
>>>>>>>>>
>>>>>>>>
>>>>>>>> I did - then we had the whole table_oid issue on Debian/Ubuntu :-(
>>>>>>>>
>>>>>>>>
>>>>>>>>> However, I updated the patch for backwards compatibility. Please
>>>>>>>>> find an updated patch attached.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> *Yosry Muhammad Yosry*
>>>>>>>>>
>>>>>>>>> Computer Engineering student,
>>>>>>>>> The Faculty of Engineering,
>>>>>>>>> Cairo University (2021).
>>>>>>>>> Class representative of CMP 2021.
>>>>>>>>> https://www.linkedin.com/in/yosrym93/
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Dave Page
>>>>>>>> Blog: http://pgsnake.blogspot.com
>>>>>>>> Twitter: @pgsnake
>>>>>>>>
>>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com
>>>>>>>> The Enterprise PostgreSQL Company
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> *Yosry Muhammad Yosry*
>>>>>>>
>>>>>>> Computer Engineering student,
>>>>>>> The Faculty of Engineering,
>>>>>>> Cairo University (2021).
>>>>>>> Class representative of CMP 2021.
>>>>>>> https://www.linkedin.com/in/yosrym93/
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Thanks and Regards,
>>>>>> Aditya Toshniwal
>>>>>> Software Engineer | EnterpriseDB India | Pune
>>>>>> "Don't Complain about Heat, Plant a TREE"
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Dave Page
>>>>> Blog: http://pgsnake.blogspot.com
>>>>> Twitter: @pgsnake
>>>>>
>>>>> EnterpriseDB UK: http://www.enterprisedb.com
>>>>> The Enterprise PostgreSQL Company
>>>>>
>>>>
>>>>
>>>> --
>>>> *Yosry Muhammad Yosry*
>>>>
>>>> Computer Engineering student,
>>>> The Faculty of Engineering,
>>>> Cairo University (2021).
>>>> Class representative of CMP 2021.
>>>> https://www.linkedin.com/in/yosrym93/
>>>>
>>>
>>>
>>> --
>>> Dave Page
>>> Blog: http://pgsnake.blogspot.com
>>> Twitter: @pgsnake
>>>
>>> EnterpriseDB UK: http://www.enterprisedb.com
>>> The Enterprise PostgreSQL Company
>>>
>>
>>
>> --
>> *Yosry Muhammad Yosry*
>>
>> Computer Engineering student,
>> The Faculty of Engineering,
>> Cairo University (2021).
>> Class representative of CMP 2021.
>> https://www.linkedin.com/in/yosrym93/
>>
>
>
> --
> Dave Page
> Blog: http://pgsnake.blogspot.com
> Twitter: @pgsnake
>
> EnterpriseDB UK: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>

--
*Yosry Muhammad Yosry*

Computer Engineering student,
The Faculty of Engineering,
Cairo University (2021).
Class representative of CMP 2021.
https://www.linkedin.com/in/yosrym93/