Re: [GSoC][New Feature] Editable and Read-only Columns

Hi Aditya,

Thanks a lot.

Do you have any idea why this problem never occurred on my machine by the
way?

On Sat, Aug 24, 2019, 8:18 AM Aditya Toshniwal <
aditya(dot)toshniwal(at)enterprisedb(dot)com> wrote:

> Hi Yosry,
>
> It's failing for me as well. I debugged and found that, sometimes the grid
> render is delayed (in microseconds) after query execution. Because of
> which, the cell you had selected using xpath is actually the older one. And
> when you're trying to set the value the cell no longer exists as the new
> cell is rendered now. That's why stale element exception. Adding below code
> solved the problem for me. Attached is the patch.
>
> import time
> time.sleep(0.5)
> for column_index, should_be_editable in enumerated_should_be_editable:
>
>
> On Sat, Aug 24, 2019 at 4:08 AM Yosry Muhammad <yosrym93(at)gmail(dot)com> wrote:
>
>> Hi Dave,
>>
>> I am now using the same versions of Python, Chrome and chromedriver. The
>> tests still pass every time. Are you sure you applied the latest patch? I
>> re-created the patch as a sanity check, please find it attached.
>>
>> I tried running the specific test a lot of times, passes every time. The
>> only difference is that I am running Ubuntu, otherwise, everything is the
>> same.
>>
>>
>> On Fri, Aug 23, 2019 at 5:00 PM Dave Page <dpage(at)pgadmin(dot)org> wrote:
>>
>>> Hi
>>>
>>> I've run it again 4 more times and it fails consistently for me. I'm
>>> running on macOS 10.14.5 with Chrome 76.0.3809.100.
>>>
>>> On Fri, Aug 23, 2019 at 3:17 PM Yosry Muhammad <yosrym93(at)gmail(dot)com>
>>> wrote:
>>>
>>>> Hi Dave,
>>>>
>>>> The tests pass on my device normally. I am using Python 3.6 on Ubuntu.
>>>> I have ran all tests using "make check" and they all succeeded. I also ran
>>>> feature_tests/query_tool_journey_test.py multiple times on its own using
>>>> "cd web && python regression/runtests.py --pkg feature_tests --modules
>>>> query_tool_journey_test" and it passed successfully. Could you try again?
>>>>
>>>> Generally, I have found that StaleElementReferenceException occurs
>>>> intermittently on various feature tests.
>>>>
>>>> On Fri, Aug 23, 2019 at 3:46 PM Dave Page <dpage(at)pgadmin(dot)org> wrote:
>>>>
>>>>> Thanks. Unfortunately this is failing the tests:
>>>>>
>>>>> ======================================================================
>>>>> ERROR: runTest
>>>>> (pgadmin.feature_tests.query_tool_journey_test.QueryToolJourneyTest)
>>>>> Tests the path through the query tool
>>>>> ----------------------------------------------------------------------
>>>>> Traceback (most recent call last):
>>>>> File
>>>>> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_journey_test.py",
>>>>> line 85, in runTest
>>>>> self._test_updatable_resultset()
>>>>> File
>>>>> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_journey_test.py",
>>>>> line 254, in _test_updatable_resultset
>>>>> discard_changes_modal=True)
>>>>> File
>>>>> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_journey_test.py",
>>>>> line 417, in _check_query_results_editable
>>>>> is_editable = self._check_cell_editable(column_index)
>>>>> File
>>>>> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_journey_test.py",
>>>>> line 430, in _check_cell_editable
>>>>> cell_value = int(cell_el.text)
>>>>> File
>>>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python3.7/site-packages/selenium/webdriver/remote/webelement.py",
>>>>> line 76, in text
>>>>> return self._execute(Command.GET_ELEMENT_TEXT)['value']
>>>>> File
>>>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python3.7/site-packages/selenium/webdriver/remote/webelement.py",
>>>>> line 628, in _execute
>>>>> return self._parent.execute(command, params)
>>>>> File
>>>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py",
>>>>> line 312, in execute
>>>>> self.error_handler.check_response(response)
>>>>> File
>>>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python3.7/site-packages/selenium/webdriver/remote/errorhandler.py",
>>>>> line 242, in check_response
>>>>> raise exception_class(message, screen, stacktrace)
>>>>> selenium.common.exceptions.StaleElementReferenceException: Message:
>>>>> stale element reference: element is not attached to the page document
>>>>> (Session info: chrome=76.0.3809.100)
>>>>> (Driver info: chromedriver=76.0.3809.126
>>>>> (d80a294506b4c9d18015e755cee48f953ddc3f2f-refs/branch-heads/3809(at){#1024}),platform=Mac
>>>>> OS X 10.14.5 x86_64)
>>>>>
>>>>> Without the patch, the tests pass.
>>>>>
>>>>>
>>>>> On Fri, Aug 23, 2019 at 1:10 PM Yosry Muhammad <yosrym93(at)gmail(dot)com>
>>>>> wrote:
>>>>>
>>>>>> Please find an updated patch attached.
>>>>>>
>>>>>> On Fri, Aug 23, 2019 at 1:22 PM Dave Page <dpage(at)pgadmin(dot)org> wrote:
>>>>>>
>>>>>>> Except it now doesn't apply as the async connection/rendering patch
>>>>>>> was in the queue right ahead of it :-(.
>>>>>>>
>>>>>>> Can I get a rebased patch please?
>>>>>>>
>>>>>>> On Fri, Aug 23, 2019 at 11:57 AM Aditya Toshniwal <
>>>>>>> aditya(dot)toshniwal(at)enterprisedb(dot)com> wrote:
>>>>>>>
>>>>>>>> Excellent !! Looks good to me.
>>>>>>>>
>>>>>>>> On Fri, Aug 23, 2019 at 2:23 PM Yosry Muhammad <yosrym93(at)gmail(dot)com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> I updated the patch for backwards compatibility anyway.
>>>>>>>>>
>>>>>>>>> On Fri, Aug 23, 2019 at 10:45 AM Dave Page <dpage(at)pgadmin(dot)org>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Fri, Aug 23, 2019 at 9:43 AM Yosry Muhammad <
>>>>>>>>>> yosrym93(at)gmail(dot)com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi Aditya,
>>>>>>>>>>>
>>>>>>>>>>> On Fri, Aug 23, 2019 at 7:33 AM Aditya Toshniwal <
>>>>>>>>>>> aditya(dot)toshniwal(at)enterprisedb(dot)com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi Yosry,
>>>>>>>>>>>>
>>>>>>>>>>>> The previously mentioned issues seems to be fixed. Below are
>>>>>>>>>>>> few issues:
>>>>>>>>>>>> 1) On Python 2.7, I get below error when opening query tool:
>>>>>>>>>>>> 2019-08-23 10:49:09,329: ERROR flask.app: Object of type buffer
>>>>>>>>>>>> is not JSON serializable
>>>>>>>>>>>> Traceback (most recent call last):
>>>>>>>>>>>> File
>>>>>>>>>>>> "/Users/adityatoshniwal/venv/pypg27/lib/python2.7/site-packages/flask/app.py",
>>>>>>>>>>>> line 1813, in full_dispatch_request
>>>>>>>>>>>> rv = self.dispatch_request()
>>>>>>>>>>>> File
>>>>>>>>>>>> "/Users/adityatoshniwal/venv/pypg27/lib/python2.7/site-packages/flask/app.py",
>>>>>>>>>>>> line 1799, in dispatch_request
>>>>>>>>>>>> return self.view_functions[rule.endpoint](**req.view_args)
>>>>>>>>>>>> File
>>>>>>>>>>>> "/Users/adityatoshniwal/venv/pypg27/lib/python2.7/site-packages/flask_login/utils.py",
>>>>>>>>>>>> line 261, in decorated_view
>>>>>>>>>>>> return func(*args, **kwargs)
>>>>>>>>>>>> File
>>>>>>>>>>>> "/Users/adityatoshniwal/projects/pgadmin4/web/pgadmin/tools/sqleditor/__init__.py",
>>>>>>>>>>>> line 1544, in get_query_history
>>>>>>>>>>>> return QueryHistory.get(current_user.id, trans_obj.sid,
>>>>>>>>>>>> conn.db)
>>>>>>>>>>>> File
>>>>>>>>>>>> "/Users/adityatoshniwal/projects/pgadmin4/web/pgadmin/tools/sqleditor/utils/query_history.py",
>>>>>>>>>>>> line 21, in get
>>>>>>>>>>>> 'result': [rec.query_info for rec in result]
>>>>>>>>>>>> File
>>>>>>>>>>>> "/Users/adityatoshniwal/projects/pgadmin4/web/pgadmin/utils/ajax.py", line
>>>>>>>>>>>> 75, in make_json_response
>>>>>>>>>>>> separators=(',', ':'), encoding=encoding),
>>>>>>>>>>>> File
>>>>>>>>>>>> "/Users/adityatoshniwal/venv/pypg27/lib/python2.7/site-packages/simplejson/__init__.py",
>>>>>>>>>>>> line 399, in dumps
>>>>>>>>>>>> **kw).encode(obj)
>>>>>>>>>>>> File
>>>>>>>>>>>> "/Users/adityatoshniwal/venv/pypg27/lib/python2.7/site-packages/simplejson/encoder.py",
>>>>>>>>>>>> line 296, in encode
>>>>>>>>>>>> chunks = self.iterencode(o, _one_shot=True)
>>>>>>>>>>>> File
>>>>>>>>>>>> "/Users/adityatoshniwal/venv/pypg27/lib/python2.7/site-packages/simplejson/encoder.py",
>>>>>>>>>>>> line 378, in iterencode
>>>>>>>>>>>> return _iterencode(o, 0)
>>>>>>>>>>>> File
>>>>>>>>>>>> "/Users/adityatoshniwal/projects/pgadmin4/web/pgadmin/utils/ajax.py", line
>>>>>>>>>>>> 30, in default
>>>>>>>>>>>> return json.JSONEncoder.default(self, obj)
>>>>>>>>>>>> File
>>>>>>>>>>>> "/Users/adityatoshniwal/venv/pypg27/lib/python2.7/site-packages/simplejson/encoder.py",
>>>>>>>>>>>> line 273, in default
>>>>>>>>>>>> o.__class__.__name__)
>>>>>>>>>>>> TypeError: Object of type buffer is not JSON serializable
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> This error exists on the master branch on Python 2.7, it is not
>>>>>>>>>>> caused by the patch. It is also not a regression of my previous Query
>>>>>>>>>>> History Enhancement patch, I checked.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> 2) The patch is not compatible with older pyscopg2 (2.7.7). I
>>>>>>>>>>>> get below error in View/Edit data.
>>>>>>>>>>>> [image: Screenshot 2019-08-23 at 10.53.30.png]
>>>>>>>>>>>> 2019-08-23 10:53:12,020: ERROR flask.app: 'table_column'
>>>>>>>>>>>> Traceback (most recent call last):
>>>>>>>>>>>> File
>>>>>>>>>>>> "/Users/adityatoshniwal/venv/pypg35/lib/python3.5/site-packages/flask/app.py",
>>>>>>>>>>>> line 1813, in full_dispatch_request
>>>>>>>>>>>> rv = self.dispatch_request()
>>>>>>>>>>>> File
>>>>>>>>>>>> "/Users/adityatoshniwal/venv/pypg35/lib/python3.5/site-packages/flask/app.py",
>>>>>>>>>>>> line 1799, in dispatch_request
>>>>>>>>>>>> return self.view_functions[rule.endpoint](**req.view_args)
>>>>>>>>>>>> File
>>>>>>>>>>>> "/Users/adityatoshniwal/venv/pypg35/lib/python3.5/site-packages/flask_login/utils.py",
>>>>>>>>>>>> line 261, in decorated_view
>>>>>>>>>>>> return func(*args, **kwargs)
>>>>>>>>>>>> File
>>>>>>>>>>>> "/Users/adityatoshniwal/projects/pgadmin4/web/pgadmin/tools/sqleditor/__init__.py",
>>>>>>>>>>>> line 435, in poll
>>>>>>>>>>>> columns = trans_obj.get_columns_types(conn)
>>>>>>>>>>>> File
>>>>>>>>>>>> "/Users/adityatoshniwal/projects/pgadmin4/web/pgadmin/tools/sqleditor/command.py",
>>>>>>>>>>>> line 688, in get_columns_types
>>>>>>>>>>>> table_oid=table_oid)
>>>>>>>>>>>> File
>>>>>>>>>>>> "/Users/adityatoshniwal/projects/pgadmin4/web/pgadmin/tools/sqleditor/utils/get_column_types.py",
>>>>>>>>>>>> line 38, in get_columns_types
>>>>>>>>>>>> if row['oid'] == col['table_column']:
>>>>>>>>>>>> KeyError: 'table_column'
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> I recall Dave mentioning that it is okay to use psycopg 2.8+
>>>>>>>>>>> features and that it was a requirement of pgAdmin 4 now (clearly mentioned
>>>>>>>>>>> in requirements.txt too).
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I did - then we had the whole table_oid issue on Debian/Ubuntu :-(
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> However, I updated the patch for backwards compatibility. Please
>>>>>>>>>>> find an updated patch attached.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> *Yosry Muhammad Yosry*
>>>>>>>>>>>
>>>>>>>>>>> Computer Engineering student,
>>>>>>>>>>> The Faculty of Engineering,
>>>>>>>>>>> Cairo University (2021).
>>>>>>>>>>> Class representative of CMP 2021.
>>>>>>>>>>> https://www.linkedin.com/in/yosrym93/
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Dave Page
>>>>>>>>>> Blog: http://pgsnake.blogspot.com
>>>>>>>>>> Twitter: @pgsnake
>>>>>>>>>>
>>>>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com
>>>>>>>>>> The Enterprise PostgreSQL Company
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> *Yosry Muhammad Yosry*
>>>>>>>>>
>>>>>>>>> Computer Engineering student,
>>>>>>>>> The Faculty of Engineering,
>>>>>>>>> Cairo University (2021).
>>>>>>>>> Class representative of CMP 2021.
>>>>>>>>> https://www.linkedin.com/in/yosrym93/
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Thanks and Regards,
>>>>>>>> Aditya Toshniwal
>>>>>>>> Software Engineer | EnterpriseDB India | Pune
>>>>>>>> "Don't Complain about Heat, Plant a TREE"
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Dave Page
>>>>>>> Blog: http://pgsnake.blogspot.com
>>>>>>> Twitter: @pgsnake
>>>>>>>
>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com
>>>>>>> The Enterprise PostgreSQL Company
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> *Yosry Muhammad Yosry*
>>>>>>
>>>>>> Computer Engineering student,
>>>>>> The Faculty of Engineering,
>>>>>> Cairo University (2021).
>>>>>> Class representative of CMP 2021.
>>>>>> https://www.linkedin.com/in/yosrym93/
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Dave Page
>>>>> Blog: http://pgsnake.blogspot.com
>>>>> Twitter: @pgsnake
>>>>>
>>>>> EnterpriseDB UK: http://www.enterprisedb.com
>>>>> The Enterprise PostgreSQL Company
>>>>>
>>>>
>>>>
>>>> --
>>>> *Yosry Muhammad Yosry*
>>>>
>>>> Computer Engineering student,
>>>> The Faculty of Engineering,
>>>> Cairo University (2021).
>>>> Class representative of CMP 2021.
>>>> https://www.linkedin.com/in/yosrym93/
>>>>
>>>
>>>
>>> --
>>> Dave Page
>>> Blog: http://pgsnake.blogspot.com
>>> Twitter: @pgsnake
>>>
>>> EnterpriseDB UK: http://www.enterprisedb.com
>>> The Enterprise PostgreSQL Company
>>>
>>
>>
>> --
>> *Yosry Muhammad Yosry*
>>
>> Computer Engineering student,
>> The Faculty of Engineering,
>> Cairo University (2021).
>> Class representative of CMP 2021.
>> https://www.linkedin.com/in/yosrym93/
>>
>
>
> --
> Thanks and Regards,
> Aditya Toshniwal
> Software Engineer | EnterpriseDB India | Pune
> "Don't Complain about Heat, Plant a TREE"
>