Re: [GSoC][New Feature] Editable and Read-only Columns

Makes sense. Thanks a lot for your help :)

On Sat, Aug 24, 2019, 8:40 AM Aditya Toshniwal <
aditya(dot)toshniwal(at)enterprisedb(dot)com> wrote:

> Hi Yosry,
>
> My guess is because selenium is running faster on our machine. Depends on
> machine.
>
> On Sat, Aug 24, 2019, 12:06 Yosry Muhammad <yosrym93(at)gmail(dot)com> wrote:
>
>> Hi Aditya,
>>
>> Thanks a lot.
>>
>> Do you have any idea why this problem never occurred on my machine by the
>> way?
>>
>> On Sat, Aug 24, 2019, 8:18 AM Aditya Toshniwal <
>> aditya(dot)toshniwal(at)enterprisedb(dot)com> wrote:
>>
>>> Hi Yosry,
>>>
>>> It's failing for me as well. I debugged and found that, sometimes the
>>> grid render is delayed (in microseconds) after query execution. Because of
>>> which, the cell you had selected using xpath is actually the older one. And
>>> when you're trying to set the value the cell no longer exists as the new
>>> cell is rendered now. That's why stale element exception. Adding below code
>>> solved the problem for me. Attached is the patch.
>>>
>>> import time
>>> time.sleep(0.5)
>>> for column_index, should_be_editable in enumerated_should_be_editable:
>>>
>>>
>>> On Sat, Aug 24, 2019 at 4:08 AM Yosry Muhammad <yosrym93(at)gmail(dot)com>
>>> wrote:
>>>
>>>> Hi Dave,
>>>>
>>>> I am now using the same versions of Python, Chrome and chromedriver.
>>>> The tests still pass every time. Are you sure you applied the latest patch?
>>>> I re-created the patch as a sanity check, please find it attached.
>>>>
>>>> I tried running the specific test a lot of times, passes every time.
>>>> The only difference is that I am running Ubuntu, otherwise, everything is
>>>> the same.
>>>>
>>>>
>>>> On Fri, Aug 23, 2019 at 5:00 PM Dave Page <dpage(at)pgadmin(dot)org> wrote:
>>>>
>>>>> Hi
>>>>>
>>>>> I've run it again 4 more times and it fails consistently for me. I'm
>>>>> running on macOS 10.14.5 with Chrome 76.0.3809.100.
>>>>>
>>>>> On Fri, Aug 23, 2019 at 3:17 PM Yosry Muhammad <yosrym93(at)gmail(dot)com>
>>>>> wrote:
>>>>>
>>>>>> Hi Dave,
>>>>>>
>>>>>> The tests pass on my device normally. I am using Python 3.6 on
>>>>>> Ubuntu. I have ran all tests using "make check" and they all succeeded. I
>>>>>> also ran feature_tests/query_tool_journey_test.py multiple times on its own
>>>>>> using "cd web && python regression/runtests.py --pkg feature_tests
>>>>>> --modules query_tool_journey_test" and it passed successfully. Could you
>>>>>> try again?
>>>>>>
>>>>>> Generally, I have found that StaleElementReferenceException occurs
>>>>>> intermittently on various feature tests.
>>>>>>
>>>>>> On Fri, Aug 23, 2019 at 3:46 PM Dave Page <dpage(at)pgadmin(dot)org> wrote:
>>>>>>
>>>>>>> Thanks. Unfortunately this is failing the tests:
>>>>>>>
>>>>>>>
>>>>>>> ======================================================================
>>>>>>> ERROR: runTest
>>>>>>> (pgadmin.feature_tests.query_tool_journey_test.QueryToolJourneyTest)
>>>>>>> Tests the path through the query tool
>>>>>>>
>>>>>>> ----------------------------------------------------------------------
>>>>>>> Traceback (most recent call last):
>>>>>>> File
>>>>>>> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_journey_test.py",
>>>>>>> line 85, in runTest
>>>>>>> self._test_updatable_resultset()
>>>>>>> File
>>>>>>> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_journey_test.py",
>>>>>>> line 254, in _test_updatable_resultset
>>>>>>> discard_changes_modal=True)
>>>>>>> File
>>>>>>> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_journey_test.py",
>>>>>>> line 417, in _check_query_results_editable
>>>>>>> is_editable = self._check_cell_editable(column_index)
>>>>>>> File
>>>>>>> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_journey_test.py",
>>>>>>> line 430, in _check_cell_editable
>>>>>>> cell_value = int(cell_el.text)
>>>>>>> File
>>>>>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python3.7/site-packages/selenium/webdriver/remote/webelement.py",
>>>>>>> line 76, in text
>>>>>>> return self._execute(Command.GET_ELEMENT_TEXT)['value']
>>>>>>> File
>>>>>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python3.7/site-packages/selenium/webdriver/remote/webelement.py",
>>>>>>> line 628, in _execute
>>>>>>> return self._parent.execute(command, params)
>>>>>>> File
>>>>>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py",
>>>>>>> line 312, in execute
>>>>>>> self.error_handler.check_response(response)
>>>>>>> File
>>>>>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python3.7/site-packages/selenium/webdriver/remote/errorhandler.py",
>>>>>>> line 242, in check_response
>>>>>>> raise exception_class(message, screen, stacktrace)
>>>>>>> selenium.common.exceptions.StaleElementReferenceException: Message:
>>>>>>> stale element reference: element is not attached to the page document
>>>>>>> (Session info: chrome=76.0.3809.100)
>>>>>>> (Driver info: chromedriver=76.0.3809.126
>>>>>>> (d80a294506b4c9d18015e755cee48f953ddc3f2f-refs/branch-heads/3809(at){#1024}),platform=Mac
>>>>>>> OS X 10.14.5 x86_64)
>>>>>>>
>>>>>>> Without the patch, the tests pass.
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Aug 23, 2019 at 1:10 PM Yosry Muhammad <yosrym93(at)gmail(dot)com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Please find an updated patch attached.
>>>>>>>>
>>>>>>>> On Fri, Aug 23, 2019 at 1:22 PM Dave Page <dpage(at)pgadmin(dot)org>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Except it now doesn't apply as the async connection/rendering
>>>>>>>>> patch was in the queue right ahead of it :-(.
>>>>>>>>>
>>>>>>>>> Can I get a rebased patch please?
>>>>>>>>>
>>>>>>>>> On Fri, Aug 23, 2019 at 11:57 AM Aditya Toshniwal <
>>>>>>>>> aditya(dot)toshniwal(at)enterprisedb(dot)com> wrote:
>>>>>>>>>
>>>>>>>>>> Excellent !! Looks good to me.
>>>>>>>>>>
>>>>>>>>>> On Fri, Aug 23, 2019 at 2:23 PM Yosry Muhammad <
>>>>>>>>>> yosrym93(at)gmail(dot)com> wrote:
>>>>>>>>>>
>>>>>>>>>>> I updated the patch for backwards compatibility anyway.
>>>>>>>>>>>
>>>>>>>>>>> On Fri, Aug 23, 2019 at 10:45 AM Dave Page <dpage(at)pgadmin(dot)org>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Fri, Aug 23, 2019 at 9:43 AM Yosry Muhammad <
>>>>>>>>>>>> yosrym93(at)gmail(dot)com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Aditya,
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Fri, Aug 23, 2019 at 7:33 AM Aditya Toshniwal <
>>>>>>>>>>>>> aditya(dot)toshniwal(at)enterprisedb(dot)com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi Yosry,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> The previously mentioned issues seems to be fixed. Below are
>>>>>>>>>>>>>> few issues:
>>>>>>>>>>>>>> 1) On Python 2.7, I get below error when opening query tool:
>>>>>>>>>>>>>> 2019-08-23 10:49:09,329: ERROR flask.app: Object of type
>>>>>>>>>>>>>> buffer is not JSON serializable
>>>>>>>>>>>>>> Traceback (most recent call last):
>>>>>>>>>>>>>> File
>>>>>>>>>>>>>> "/Users/adityatoshniwal/venv/pypg27/lib/python2.7/site-packages/flask/app.py",
>>>>>>>>>>>>>> line 1813, in full_dispatch_request
>>>>>>>>>>>>>> rv = self.dispatch_request()
>>>>>>>>>>>>>> File
>>>>>>>>>>>>>> "/Users/adityatoshniwal/venv/pypg27/lib/python2.7/site-packages/flask/app.py",
>>>>>>>>>>>>>> line 1799, in dispatch_request
>>>>>>>>>>>>>> return self.view_functions[rule.endpoint](**req.view_args)
>>>>>>>>>>>>>> File
>>>>>>>>>>>>>> "/Users/adityatoshniwal/venv/pypg27/lib/python2.7/site-packages/flask_login/utils.py",
>>>>>>>>>>>>>> line 261, in decorated_view
>>>>>>>>>>>>>> return func(*args, **kwargs)
>>>>>>>>>>>>>> File
>>>>>>>>>>>>>> "/Users/adityatoshniwal/projects/pgadmin4/web/pgadmin/tools/sqleditor/__init__.py",
>>>>>>>>>>>>>> line 1544, in get_query_history
>>>>>>>>>>>>>> return QueryHistory.get(current_user.id, trans_obj.sid,
>>>>>>>>>>>>>> conn.db)
>>>>>>>>>>>>>> File
>>>>>>>>>>>>>> "/Users/adityatoshniwal/projects/pgadmin4/web/pgadmin/tools/sqleditor/utils/query_history.py",
>>>>>>>>>>>>>> line 21, in get
>>>>>>>>>>>>>> 'result': [rec.query_info for rec in result]
>>>>>>>>>>>>>> File
>>>>>>>>>>>>>> "/Users/adityatoshniwal/projects/pgadmin4/web/pgadmin/utils/ajax.py", line
>>>>>>>>>>>>>> 75, in make_json_response
>>>>>>>>>>>>>> separators=(',', ':'), encoding=encoding),
>>>>>>>>>>>>>> File
>>>>>>>>>>>>>> "/Users/adityatoshniwal/venv/pypg27/lib/python2.7/site-packages/simplejson/__init__.py",
>>>>>>>>>>>>>> line 399, in dumps
>>>>>>>>>>>>>> **kw).encode(obj)
>>>>>>>>>>>>>> File
>>>>>>>>>>>>>> "/Users/adityatoshniwal/venv/pypg27/lib/python2.7/site-packages/simplejson/encoder.py",
>>>>>>>>>>>>>> line 296, in encode
>>>>>>>>>>>>>> chunks = self.iterencode(o, _one_shot=True)
>>>>>>>>>>>>>> File
>>>>>>>>>>>>>> "/Users/adityatoshniwal/venv/pypg27/lib/python2.7/site-packages/simplejson/encoder.py",
>>>>>>>>>>>>>> line 378, in iterencode
>>>>>>>>>>>>>> return _iterencode(o, 0)
>>>>>>>>>>>>>> File
>>>>>>>>>>>>>> "/Users/adityatoshniwal/projects/pgadmin4/web/pgadmin/utils/ajax.py", line
>>>>>>>>>>>>>> 30, in default
>>>>>>>>>>>>>> return json.JSONEncoder.default(self, obj)
>>>>>>>>>>>>>> File
>>>>>>>>>>>>>> "/Users/adityatoshniwal/venv/pypg27/lib/python2.7/site-packages/simplejson/encoder.py",
>>>>>>>>>>>>>> line 273, in default
>>>>>>>>>>>>>> o.__class__.__name__)
>>>>>>>>>>>>>> TypeError: Object of type buffer is not JSON serializable
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> This error exists on the master branch on Python 2.7, it is
>>>>>>>>>>>>> not caused by the patch. It is also not a regression of my previous Query
>>>>>>>>>>>>> History Enhancement patch, I checked.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 2) The patch is not compatible with older pyscopg2 (2.7.7). I
>>>>>>>>>>>>>> get below error in View/Edit data.
>>>>>>>>>>>>>> [image: Screenshot 2019-08-23 at 10.53.30.png]
>>>>>>>>>>>>>> 2019-08-23 10:53:12,020: ERROR flask.app: 'table_column'
>>>>>>>>>>>>>> Traceback (most recent call last):
>>>>>>>>>>>>>> File
>>>>>>>>>>>>>> "/Users/adityatoshniwal/venv/pypg35/lib/python3.5/site-packages/flask/app.py",
>>>>>>>>>>>>>> line 1813, in full_dispatch_request
>>>>>>>>>>>>>> rv = self.dispatch_request()
>>>>>>>>>>>>>> File
>>>>>>>>>>>>>> "/Users/adityatoshniwal/venv/pypg35/lib/python3.5/site-packages/flask/app.py",
>>>>>>>>>>>>>> line 1799, in dispatch_request
>>>>>>>>>>>>>> return self.view_functions[rule.endpoint](**req.view_args)
>>>>>>>>>>>>>> File
>>>>>>>>>>>>>> "/Users/adityatoshniwal/venv/pypg35/lib/python3.5/site-packages/flask_login/utils.py",
>>>>>>>>>>>>>> line 261, in decorated_view
>>>>>>>>>>>>>> return func(*args, **kwargs)
>>>>>>>>>>>>>> File
>>>>>>>>>>>>>> "/Users/adityatoshniwal/projects/pgadmin4/web/pgadmin/tools/sqleditor/__init__.py",
>>>>>>>>>>>>>> line 435, in poll
>>>>>>>>>>>>>> columns = trans_obj.get_columns_types(conn)
>>>>>>>>>>>>>> File
>>>>>>>>>>>>>> "/Users/adityatoshniwal/projects/pgadmin4/web/pgadmin/tools/sqleditor/command.py",
>>>>>>>>>>>>>> line 688, in get_columns_types
>>>>>>>>>>>>>> table_oid=table_oid)
>>>>>>>>>>>>>> File
>>>>>>>>>>>>>> "/Users/adityatoshniwal/projects/pgadmin4/web/pgadmin/tools/sqleditor/utils/get_column_types.py",
>>>>>>>>>>>>>> line 38, in get_columns_types
>>>>>>>>>>>>>> if row['oid'] == col['table_column']:
>>>>>>>>>>>>>> KeyError: 'table_column'
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> I recall Dave mentioning that it is okay to use psycopg 2.8+
>>>>>>>>>>>>> features and that it was a requirement of pgAdmin 4 now (clearly mentioned
>>>>>>>>>>>>> in requirements.txt too).
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> I did - then we had the whole table_oid issue on Debian/Ubuntu
>>>>>>>>>>>> :-(
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>> However, I updated the patch for backwards compatibility.
>>>>>>>>>>>>> Please find an updated patch attached.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> *Yosry Muhammad Yosry*
>>>>>>>>>>>>>
>>>>>>>>>>>>> Computer Engineering student,
>>>>>>>>>>>>> The Faculty of Engineering,
>>>>>>>>>>>>> Cairo University (2021).
>>>>>>>>>>>>> Class representative of CMP 2021.
>>>>>>>>>>>>> https://www.linkedin.com/in/yosrym93/
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> Dave Page
>>>>>>>>>>>> Blog: http://pgsnake.blogspot.com
>>>>>>>>>>>> Twitter: @pgsnake
>>>>>>>>>>>>
>>>>>>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com
>>>>>>>>>>>> The Enterprise PostgreSQL Company
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> *Yosry Muhammad Yosry*
>>>>>>>>>>>
>>>>>>>>>>> Computer Engineering student,
>>>>>>>>>>> The Faculty of Engineering,
>>>>>>>>>>> Cairo University (2021).
>>>>>>>>>>> Class representative of CMP 2021.
>>>>>>>>>>> https://www.linkedin.com/in/yosrym93/
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Thanks and Regards,
>>>>>>>>>> Aditya Toshniwal
>>>>>>>>>> Software Engineer | EnterpriseDB India | Pune
>>>>>>>>>> "Don't Complain about Heat, Plant a TREE"
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Dave Page
>>>>>>>>> Blog: http://pgsnake.blogspot.com
>>>>>>>>> Twitter: @pgsnake
>>>>>>>>>
>>>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com
>>>>>>>>> The Enterprise PostgreSQL Company
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> *Yosry Muhammad Yosry*
>>>>>>>>
>>>>>>>> Computer Engineering student,
>>>>>>>> The Faculty of Engineering,
>>>>>>>> Cairo University (2021).
>>>>>>>> Class representative of CMP 2021.
>>>>>>>> https://www.linkedin.com/in/yosrym93/
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Dave Page
>>>>>>> Blog: http://pgsnake.blogspot.com
>>>>>>> Twitter: @pgsnake
>>>>>>>
>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com
>>>>>>> The Enterprise PostgreSQL Company
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> *Yosry Muhammad Yosry*
>>>>>>
>>>>>> Computer Engineering student,
>>>>>> The Faculty of Engineering,
>>>>>> Cairo University (2021).
>>>>>> Class representative of CMP 2021.
>>>>>> https://www.linkedin.com/in/yosrym93/
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Dave Page
>>>>> Blog: http://pgsnake.blogspot.com
>>>>> Twitter: @pgsnake
>>>>>
>>>>> EnterpriseDB UK: http://www.enterprisedb.com
>>>>> The Enterprise PostgreSQL Company
>>>>>
>>>>
>>>>
>>>> --
>>>> *Yosry Muhammad Yosry*
>>>>
>>>> Computer Engineering student,
>>>> The Faculty of Engineering,
>>>> Cairo University (2021).
>>>> Class representative of CMP 2021.
>>>> https://www.linkedin.com/in/yosrym93/
>>>>
>>>
>>>
>>> --
>>> Thanks and Regards,
>>> Aditya Toshniwal
>>> Software Engineer | EnterpriseDB India | Pune
>>> "Don't Complain about Heat, Plant a TREE"
>>>
>>