Re: [GSoC][New Feature] Editable and Read-only Columns

Hi Yosry,

My guess is because selenium is running faster on our machine. Depends on
machine.

On Sat, Aug 24, 2019, 12:06 Yosry Muhammad <yosrym93(at)gmail(dot)com> wrote:

> Hi Aditya,
>
> Thanks a lot.
>
> Do you have any idea why this problem never occurred on my machine by the
> way?
>
> On Sat, Aug 24, 2019, 8:18 AM Aditya Toshniwal <
> aditya(dot)toshniwal(at)enterprisedb(dot)com> wrote:
>
>> Hi Yosry,
>>
>> It's failing for me as well. I debugged and found that, sometimes the
>> grid render is delayed (in microseconds) after query execution. Because of
>> which, the cell you had selected using xpath is actually the older one. And
>> when you're trying to set the value the cell no longer exists as the new
>> cell is rendered now. That's why stale element exception. Adding below code
>> solved the problem for me. Attached is the patch.
>>
>> import time
>> time.sleep(0.5)
>> for column_index, should_be_editable in enumerated_should_be_editable:
>>
>>
>> On Sat, Aug 24, 2019 at 4:08 AM Yosry Muhammad <yosrym93(at)gmail(dot)com>
>> wrote:
>>
>>> Hi Dave,
>>>
>>> I am now using the same versions of Python, Chrome and chromedriver. The
>>> tests still pass every time. Are you sure you applied the latest patch? I
>>> re-created the patch as a sanity check, please find it attached.
>>>
>>> I tried running the specific test a lot of times, passes every time. The
>>> only difference is that I am running Ubuntu, otherwise, everything is the
>>> same.
>>>
>>>
>>> On Fri, Aug 23, 2019 at 5:00 PM Dave Page <dpage(at)pgadmin(dot)org> wrote:
>>>
>>>> Hi
>>>>
>>>> I've run it again 4 more times and it fails consistently for me. I'm
>>>> running on macOS 10.14.5 with Chrome 76.0.3809.100.
>>>>
>>>> On Fri, Aug 23, 2019 at 3:17 PM Yosry Muhammad <yosrym93(at)gmail(dot)com>
>>>> wrote:
>>>>
>>>>> Hi Dave,
>>>>>
>>>>> The tests pass on my device normally. I am using Python 3.6 on Ubuntu.
>>>>> I have ran all tests using "make check" and they all succeeded. I also ran
>>>>> feature_tests/query_tool_journey_test.py multiple times on its own using
>>>>> "cd web && python regression/runtests.py --pkg feature_tests --modules
>>>>> query_tool_journey_test" and it passed successfully. Could you try again?
>>>>>
>>>>> Generally, I have found that StaleElementReferenceException occurs
>>>>> intermittently on various feature tests.
>>>>>
>>>>> On Fri, Aug 23, 2019 at 3:46 PM Dave Page <dpage(at)pgadmin(dot)org> wrote:
>>>>>
>>>>>> Thanks. Unfortunately this is failing the tests:
>>>>>>
>>>>>> ======================================================================
>>>>>> ERROR: runTest
>>>>>> (pgadmin.feature_tests.query_tool_journey_test.QueryToolJourneyTest)
>>>>>> Tests the path through the query tool
>>>>>> ----------------------------------------------------------------------
>>>>>> Traceback (most recent call last):
>>>>>> File
>>>>>> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_journey_test.py",
>>>>>> line 85, in runTest
>>>>>> self._test_updatable_resultset()
>>>>>> File
>>>>>> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_journey_test.py",
>>>>>> line 254, in _test_updatable_resultset
>>>>>> discard_changes_modal=True)
>>>>>> File
>>>>>> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_journey_test.py",
>>>>>> line 417, in _check_query_results_editable
>>>>>> is_editable = self._check_cell_editable(column_index)
>>>>>> File
>>>>>> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_journey_test.py",
>>>>>> line 430, in _check_cell_editable
>>>>>> cell_value = int(cell_el.text)
>>>>>> File
>>>>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python3.7/site-packages/selenium/webdriver/remote/webelement.py",
>>>>>> line 76, in text
>>>>>> return self._execute(Command.GET_ELEMENT_TEXT)['value']
>>>>>> File
>>>>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python3.7/site-packages/selenium/webdriver/remote/webelement.py",
>>>>>> line 628, in _execute
>>>>>> return self._parent.execute(command, params)
>>>>>> File
>>>>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py",
>>>>>> line 312, in execute
>>>>>> self.error_handler.check_response(response)
>>>>>> File
>>>>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python3.7/site-packages/selenium/webdriver/remote/errorhandler.py",
>>>>>> line 242, in check_response
>>>>>> raise exception_class(message, screen, stacktrace)
>>>>>> selenium.common.exceptions.StaleElementReferenceException: Message:
>>>>>> stale element reference: element is not attached to the page document
>>>>>> (Session info: chrome=76.0.3809.100)
>>>>>> (Driver info: chromedriver=76.0.3809.126
>>>>>> (d80a294506b4c9d18015e755cee48f953ddc3f2f-refs/branch-heads/3809(at){#1024}),platform=Mac
>>>>>> OS X 10.14.5 x86_64)
>>>>>>
>>>>>> Without the patch, the tests pass.
>>>>>>
>>>>>>
>>>>>> On Fri, Aug 23, 2019 at 1:10 PM Yosry Muhammad <yosrym93(at)gmail(dot)com>
>>>>>> wrote:
>>>>>>
>>>>>>> Please find an updated patch attached.
>>>>>>>
>>>>>>> On Fri, Aug 23, 2019 at 1:22 PM Dave Page <dpage(at)pgadmin(dot)org> wrote:
>>>>>>>
>>>>>>>> Except it now doesn't apply as the async connection/rendering patch
>>>>>>>> was in the queue right ahead of it :-(.
>>>>>>>>
>>>>>>>> Can I get a rebased patch please?
>>>>>>>>
>>>>>>>> On Fri, Aug 23, 2019 at 11:57 AM Aditya Toshniwal <
>>>>>>>> aditya(dot)toshniwal(at)enterprisedb(dot)com> wrote:
>>>>>>>>
>>>>>>>>> Excellent !! Looks good to me.
>>>>>>>>>
>>>>>>>>> On Fri, Aug 23, 2019 at 2:23 PM Yosry Muhammad <yosrym93(at)gmail(dot)com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> I updated the patch for backwards compatibility anyway.
>>>>>>>>>>
>>>>>>>>>> On Fri, Aug 23, 2019 at 10:45 AM Dave Page <dpage(at)pgadmin(dot)org>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Fri, Aug 23, 2019 at 9:43 AM Yosry Muhammad <
>>>>>>>>>>> yosrym93(at)gmail(dot)com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi Aditya,
>>>>>>>>>>>>
>>>>>>>>>>>> On Fri, Aug 23, 2019 at 7:33 AM Aditya Toshniwal <
>>>>>>>>>>>> aditya(dot)toshniwal(at)enterprisedb(dot)com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Yosry,
>>>>>>>>>>>>>
>>>>>>>>>>>>> The previously mentioned issues seems to be fixed. Below are
>>>>>>>>>>>>> few issues:
>>>>>>>>>>>>> 1) On Python 2.7, I get below error when opening query tool:
>>>>>>>>>>>>> 2019-08-23 10:49:09,329: ERROR flask.app: Object of type
>>>>>>>>>>>>> buffer is not JSON serializable
>>>>>>>>>>>>> Traceback (most recent call last):
>>>>>>>>>>>>> File
>>>>>>>>>>>>> "/Users/adityatoshniwal/venv/pypg27/lib/python2.7/site-packages/flask/app.py",
>>>>>>>>>>>>> line 1813, in full_dispatch_request
>>>>>>>>>>>>> rv = self.dispatch_request()
>>>>>>>>>>>>> File
>>>>>>>>>>>>> "/Users/adityatoshniwal/venv/pypg27/lib/python2.7/site-packages/flask/app.py",
>>>>>>>>>>>>> line 1799, in dispatch_request
>>>>>>>>>>>>> return self.view_functions[rule.endpoint](**req.view_args)
>>>>>>>>>>>>> File
>>>>>>>>>>>>> "/Users/adityatoshniwal/venv/pypg27/lib/python2.7/site-packages/flask_login/utils.py",
>>>>>>>>>>>>> line 261, in decorated_view
>>>>>>>>>>>>> return func(*args, **kwargs)
>>>>>>>>>>>>> File
>>>>>>>>>>>>> "/Users/adityatoshniwal/projects/pgadmin4/web/pgadmin/tools/sqleditor/__init__.py",
>>>>>>>>>>>>> line 1544, in get_query_history
>>>>>>>>>>>>> return QueryHistory.get(current_user.id, trans_obj.sid,
>>>>>>>>>>>>> conn.db)
>>>>>>>>>>>>> File
>>>>>>>>>>>>> "/Users/adityatoshniwal/projects/pgadmin4/web/pgadmin/tools/sqleditor/utils/query_history.py",
>>>>>>>>>>>>> line 21, in get
>>>>>>>>>>>>> 'result': [rec.query_info for rec in result]
>>>>>>>>>>>>> File
>>>>>>>>>>>>> "/Users/adityatoshniwal/projects/pgadmin4/web/pgadmin/utils/ajax.py", line
>>>>>>>>>>>>> 75, in make_json_response
>>>>>>>>>>>>> separators=(',', ':'), encoding=encoding),
>>>>>>>>>>>>> File
>>>>>>>>>>>>> "/Users/adityatoshniwal/venv/pypg27/lib/python2.7/site-packages/simplejson/__init__.py",
>>>>>>>>>>>>> line 399, in dumps
>>>>>>>>>>>>> **kw).encode(obj)
>>>>>>>>>>>>> File
>>>>>>>>>>>>> "/Users/adityatoshniwal/venv/pypg27/lib/python2.7/site-packages/simplejson/encoder.py",
>>>>>>>>>>>>> line 296, in encode
>>>>>>>>>>>>> chunks = self.iterencode(o, _one_shot=True)
>>>>>>>>>>>>> File
>>>>>>>>>>>>> "/Users/adityatoshniwal/venv/pypg27/lib/python2.7/site-packages/simplejson/encoder.py",
>>>>>>>>>>>>> line 378, in iterencode
>>>>>>>>>>>>> return _iterencode(o, 0)
>>>>>>>>>>>>> File
>>>>>>>>>>>>> "/Users/adityatoshniwal/projects/pgadmin4/web/pgadmin/utils/ajax.py", line
>>>>>>>>>>>>> 30, in default
>>>>>>>>>>>>> return json.JSONEncoder.default(self, obj)
>>>>>>>>>>>>> File
>>>>>>>>>>>>> "/Users/adityatoshniwal/venv/pypg27/lib/python2.7/site-packages/simplejson/encoder.py",
>>>>>>>>>>>>> line 273, in default
>>>>>>>>>>>>> o.__class__.__name__)
>>>>>>>>>>>>> TypeError: Object of type buffer is not JSON serializable
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> This error exists on the master branch on Python 2.7, it is not
>>>>>>>>>>>> caused by the patch. It is also not a regression of my previous Query
>>>>>>>>>>>> History Enhancement patch, I checked.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> 2) The patch is not compatible with older pyscopg2 (2.7.7). I
>>>>>>>>>>>>> get below error in View/Edit data.
>>>>>>>>>>>>> [image: Screenshot 2019-08-23 at 10.53.30.png]
>>>>>>>>>>>>> 2019-08-23 10:53:12,020: ERROR flask.app: 'table_column'
>>>>>>>>>>>>> Traceback (most recent call last):
>>>>>>>>>>>>> File
>>>>>>>>>>>>> "/Users/adityatoshniwal/venv/pypg35/lib/python3.5/site-packages/flask/app.py",
>>>>>>>>>>>>> line 1813, in full_dispatch_request
>>>>>>>>>>>>> rv = self.dispatch_request()
>>>>>>>>>>>>> File
>>>>>>>>>>>>> "/Users/adityatoshniwal/venv/pypg35/lib/python3.5/site-packages/flask/app.py",
>>>>>>>>>>>>> line 1799, in dispatch_request
>>>>>>>>>>>>> return self.view_functions[rule.endpoint](**req.view_args)
>>>>>>>>>>>>> File
>>>>>>>>>>>>> "/Users/adityatoshniwal/venv/pypg35/lib/python3.5/site-packages/flask_login/utils.py",
>>>>>>>>>>>>> line 261, in decorated_view
>>>>>>>>>>>>> return func(*args, **kwargs)
>>>>>>>>>>>>> File
>>>>>>>>>>>>> "/Users/adityatoshniwal/projects/pgadmin4/web/pgadmin/tools/sqleditor/__init__.py",
>>>>>>>>>>>>> line 435, in poll
>>>>>>>>>>>>> columns = trans_obj.get_columns_types(conn)
>>>>>>>>>>>>> File
>>>>>>>>>>>>> "/Users/adityatoshniwal/projects/pgadmin4/web/pgadmin/tools/sqleditor/command.py",
>>>>>>>>>>>>> line 688, in get_columns_types
>>>>>>>>>>>>> table_oid=table_oid)
>>>>>>>>>>>>> File
>>>>>>>>>>>>> "/Users/adityatoshniwal/projects/pgadmin4/web/pgadmin/tools/sqleditor/utils/get_column_types.py",
>>>>>>>>>>>>> line 38, in get_columns_types
>>>>>>>>>>>>> if row['oid'] == col['table_column']:
>>>>>>>>>>>>> KeyError: 'table_column'
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> I recall Dave mentioning that it is okay to use psycopg 2.8+
>>>>>>>>>>>> features and that it was a requirement of pgAdmin 4 now (clearly mentioned
>>>>>>>>>>>> in requirements.txt too).
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> I did - then we had the whole table_oid issue on Debian/Ubuntu
>>>>>>>>>>> :-(
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> However, I updated the patch for backwards compatibility.
>>>>>>>>>>>> Please find an updated patch attached.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> *Yosry Muhammad Yosry*
>>>>>>>>>>>>
>>>>>>>>>>>> Computer Engineering student,
>>>>>>>>>>>> The Faculty of Engineering,
>>>>>>>>>>>> Cairo University (2021).
>>>>>>>>>>>> Class representative of CMP 2021.
>>>>>>>>>>>> https://www.linkedin.com/in/yosrym93/
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Dave Page
>>>>>>>>>>> Blog: http://pgsnake.blogspot.com
>>>>>>>>>>> Twitter: @pgsnake
>>>>>>>>>>>
>>>>>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com
>>>>>>>>>>> The Enterprise PostgreSQL Company
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> *Yosry Muhammad Yosry*
>>>>>>>>>>
>>>>>>>>>> Computer Engineering student,
>>>>>>>>>> The Faculty of Engineering,
>>>>>>>>>> Cairo University (2021).
>>>>>>>>>> Class representative of CMP 2021.
>>>>>>>>>> https://www.linkedin.com/in/yosrym93/
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Thanks and Regards,
>>>>>>>>> Aditya Toshniwal
>>>>>>>>> Software Engineer | EnterpriseDB India | Pune
>>>>>>>>> "Don't Complain about Heat, Plant a TREE"
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Dave Page
>>>>>>>> Blog: http://pgsnake.blogspot.com
>>>>>>>> Twitter: @pgsnake
>>>>>>>>
>>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com
>>>>>>>> The Enterprise PostgreSQL Company
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> *Yosry Muhammad Yosry*
>>>>>>>
>>>>>>> Computer Engineering student,
>>>>>>> The Faculty of Engineering,
>>>>>>> Cairo University (2021).
>>>>>>> Class representative of CMP 2021.
>>>>>>> https://www.linkedin.com/in/yosrym93/
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Dave Page
>>>>>> Blog: http://pgsnake.blogspot.com
>>>>>> Twitter: @pgsnake
>>>>>>
>>>>>> EnterpriseDB UK: http://www.enterprisedb.com
>>>>>> The Enterprise PostgreSQL Company
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> *Yosry Muhammad Yosry*
>>>>>
>>>>> Computer Engineering student,
>>>>> The Faculty of Engineering,
>>>>> Cairo University (2021).
>>>>> Class representative of CMP 2021.
>>>>> https://www.linkedin.com/in/yosrym93/
>>>>>
>>>>
>>>>
>>>> --
>>>> Dave Page
>>>> Blog: http://pgsnake.blogspot.com
>>>> Twitter: @pgsnake
>>>>
>>>> EnterpriseDB UK: http://www.enterprisedb.com
>>>> The Enterprise PostgreSQL Company
>>>>
>>>
>>>
>>> --
>>> *Yosry Muhammad Yosry*
>>>
>>> Computer Engineering student,
>>> The Faculty of Engineering,
>>> Cairo University (2021).
>>> Class representative of CMP 2021.
>>> https://www.linkedin.com/in/yosrym93/
>>>
>>
>>
>> --
>> Thanks and Regards,
>> Aditya Toshniwal
>> Software Engineer | EnterpriseDB India | Pune
>> "Don't Complain about Heat, Plant a TREE"
>>
>