Re: [GSoC][New Feature] Editable and Read-only Columns

From: Aditya Toshniwal <aditya(dot)toshniwal(at)enterprisedb(dot)com>
To: Yosry Muhammad <yosrym93(at)gmail(dot)com>
Cc: Dave Page <dpage(at)pgadmin(dot)org>, pgadmin-hackers <pgadmin-hackers(at)postgresql(dot)org>
Subject: Re: [GSoC][New Feature] Editable and Read-only Columns
Date: 2019-08-24 06:18:08
Message-ID: CAM9w-_nRuqpZikHQ4dxtfgnN49HygTWwx_6tjPmMVp8mzejbbw@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgadmin-hackers

Hi Yosry,

It's failing for me as well. I debugged and found that, sometimes the grid
render is delayed (in microseconds) after query execution. Because of
which, the cell you had selected using xpath is actually the older one. And
when you're trying to set the value the cell no longer exists as the new
cell is rendered now. That's why stale element exception. Adding below code
solved the problem for me. Attached is the patch.

import time
time.sleep(0.5)
for column_index, should_be_editable in enumerated_should_be_editable:

On Sat, Aug 24, 2019 at 4:08 AM Yosry Muhammad <yosrym93(at)gmail(dot)com> wrote:

> Hi Dave,
>
> I am now using the same versions of Python, Chrome and chromedriver. The
> tests still pass every time. Are you sure you applied the latest patch? I
> re-created the patch as a sanity check, please find it attached.
>
> I tried running the specific test a lot of times, passes every time. The
> only difference is that I am running Ubuntu, otherwise, everything is the
> same.
>
>
> On Fri, Aug 23, 2019 at 5:00 PM Dave Page <dpage(at)pgadmin(dot)org> wrote:
>
>> Hi
>>
>> I've run it again 4 more times and it fails consistently for me. I'm
>> running on macOS 10.14.5 with Chrome 76.0.3809.100.
>>
>> On Fri, Aug 23, 2019 at 3:17 PM Yosry Muhammad <yosrym93(at)gmail(dot)com>
>> wrote:
>>
>>> Hi Dave,
>>>
>>> The tests pass on my device normally. I am using Python 3.6 on Ubuntu. I
>>> have ran all tests using "make check" and they all succeeded. I also ran
>>> feature_tests/query_tool_journey_test.py multiple times on its own using
>>> "cd web && python regression/runtests.py --pkg feature_tests --modules
>>> query_tool_journey_test" and it passed successfully. Could you try again?
>>>
>>> Generally, I have found that StaleElementReferenceException occurs
>>> intermittently on various feature tests.
>>>
>>> On Fri, Aug 23, 2019 at 3:46 PM Dave Page <dpage(at)pgadmin(dot)org> wrote:
>>>
>>>> Thanks. Unfortunately this is failing the tests:
>>>>
>>>> ======================================================================
>>>> ERROR: runTest
>>>> (pgadmin.feature_tests.query_tool_journey_test.QueryToolJourneyTest)
>>>> Tests the path through the query tool
>>>> ----------------------------------------------------------------------
>>>> Traceback (most recent call last):
>>>> File
>>>> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_journey_test.py",
>>>> line 85, in runTest
>>>> self._test_updatable_resultset()
>>>> File
>>>> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_journey_test.py",
>>>> line 254, in _test_updatable_resultset
>>>> discard_changes_modal=True)
>>>> File
>>>> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_journey_test.py",
>>>> line 417, in _check_query_results_editable
>>>> is_editable = self._check_cell_editable(column_index)
>>>> File
>>>> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_journey_test.py",
>>>> line 430, in _check_cell_editable
>>>> cell_value = int(cell_el.text)
>>>> File
>>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python3.7/site-packages/selenium/webdriver/remote/webelement.py",
>>>> line 76, in text
>>>> return self._execute(Command.GET_ELEMENT_TEXT)['value']
>>>> File
>>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python3.7/site-packages/selenium/webdriver/remote/webelement.py",
>>>> line 628, in _execute
>>>> return self._parent.execute(command, params)
>>>> File
>>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py",
>>>> line 312, in execute
>>>> self.error_handler.check_response(response)
>>>> File
>>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python3.7/site-packages/selenium/webdriver/remote/errorhandler.py",
>>>> line 242, in check_response
>>>> raise exception_class(message, screen, stacktrace)
>>>> selenium.common.exceptions.StaleElementReferenceException: Message:
>>>> stale element reference: element is not attached to the page document
>>>> (Session info: chrome=76.0.3809.100)
>>>> (Driver info: chromedriver=76.0.3809.126
>>>> (d80a294506b4c9d18015e755cee48f953ddc3f2f-refs/branch-heads/3809(at){#1024}),platform=Mac
>>>> OS X 10.14.5 x86_64)
>>>>
>>>> Without the patch, the tests pass.
>>>>
>>>>
>>>> On Fri, Aug 23, 2019 at 1:10 PM Yosry Muhammad <yosrym93(at)gmail(dot)com>
>>>> wrote:
>>>>
>>>>> Please find an updated patch attached.
>>>>>
>>>>> On Fri, Aug 23, 2019 at 1:22 PM Dave Page <dpage(at)pgadmin(dot)org> wrote:
>>>>>
>>>>>> Except it now doesn't apply as the async connection/rendering patch
>>>>>> was in the queue right ahead of it :-(.
>>>>>>
>>>>>> Can I get a rebased patch please?
>>>>>>
>>>>>> On Fri, Aug 23, 2019 at 11:57 AM Aditya Toshniwal <
>>>>>> aditya(dot)toshniwal(at)enterprisedb(dot)com> wrote:
>>>>>>
>>>>>>> Excellent !! Looks good to me.
>>>>>>>
>>>>>>> On Fri, Aug 23, 2019 at 2:23 PM Yosry Muhammad <yosrym93(at)gmail(dot)com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> I updated the patch for backwards compatibility anyway.
>>>>>>>>
>>>>>>>> On Fri, Aug 23, 2019 at 10:45 AM Dave Page <dpage(at)pgadmin(dot)org>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Fri, Aug 23, 2019 at 9:43 AM Yosry Muhammad <yosrym93(at)gmail(dot)com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hi Aditya,
>>>>>>>>>>
>>>>>>>>>> On Fri, Aug 23, 2019 at 7:33 AM Aditya Toshniwal <
>>>>>>>>>> aditya(dot)toshniwal(at)enterprisedb(dot)com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi Yosry,
>>>>>>>>>>>
>>>>>>>>>>> The previously mentioned issues seems to be fixed. Below are few
>>>>>>>>>>> issues:
>>>>>>>>>>> 1) On Python 2.7, I get below error when opening query tool:
>>>>>>>>>>> 2019-08-23 10:49:09,329: ERROR flask.app: Object of type buffer
>>>>>>>>>>> is not JSON serializable
>>>>>>>>>>> Traceback (most recent call last):
>>>>>>>>>>> File
>>>>>>>>>>> "/Users/adityatoshniwal/venv/pypg27/lib/python2.7/site-packages/flask/app.py",
>>>>>>>>>>> line 1813, in full_dispatch_request
>>>>>>>>>>> rv = self.dispatch_request()
>>>>>>>>>>> File
>>>>>>>>>>> "/Users/adityatoshniwal/venv/pypg27/lib/python2.7/site-packages/flask/app.py",
>>>>>>>>>>> line 1799, in dispatch_request
>>>>>>>>>>> return self.view_functions[rule.endpoint](**req.view_args)
>>>>>>>>>>> File
>>>>>>>>>>> "/Users/adityatoshniwal/venv/pypg27/lib/python2.7/site-packages/flask_login/utils.py",
>>>>>>>>>>> line 261, in decorated_view
>>>>>>>>>>> return func(*args, **kwargs)
>>>>>>>>>>> File
>>>>>>>>>>> "/Users/adityatoshniwal/projects/pgadmin4/web/pgadmin/tools/sqleditor/__init__.py",
>>>>>>>>>>> line 1544, in get_query_history
>>>>>>>>>>> return QueryHistory.get(current_user.id, trans_obj.sid,
>>>>>>>>>>> conn.db)
>>>>>>>>>>> File
>>>>>>>>>>> "/Users/adityatoshniwal/projects/pgadmin4/web/pgadmin/tools/sqleditor/utils/query_history.py",
>>>>>>>>>>> line 21, in get
>>>>>>>>>>> 'result': [rec.query_info for rec in result]
>>>>>>>>>>> File
>>>>>>>>>>> "/Users/adityatoshniwal/projects/pgadmin4/web/pgadmin/utils/ajax.py", line
>>>>>>>>>>> 75, in make_json_response
>>>>>>>>>>> separators=(',', ':'), encoding=encoding),
>>>>>>>>>>> File
>>>>>>>>>>> "/Users/adityatoshniwal/venv/pypg27/lib/python2.7/site-packages/simplejson/__init__.py",
>>>>>>>>>>> line 399, in dumps
>>>>>>>>>>> **kw).encode(obj)
>>>>>>>>>>> File
>>>>>>>>>>> "/Users/adityatoshniwal/venv/pypg27/lib/python2.7/site-packages/simplejson/encoder.py",
>>>>>>>>>>> line 296, in encode
>>>>>>>>>>> chunks = self.iterencode(o, _one_shot=True)
>>>>>>>>>>> File
>>>>>>>>>>> "/Users/adityatoshniwal/venv/pypg27/lib/python2.7/site-packages/simplejson/encoder.py",
>>>>>>>>>>> line 378, in iterencode
>>>>>>>>>>> return _iterencode(o, 0)
>>>>>>>>>>> File
>>>>>>>>>>> "/Users/adityatoshniwal/projects/pgadmin4/web/pgadmin/utils/ajax.py", line
>>>>>>>>>>> 30, in default
>>>>>>>>>>> return json.JSONEncoder.default(self, obj)
>>>>>>>>>>> File
>>>>>>>>>>> "/Users/adityatoshniwal/venv/pypg27/lib/python2.7/site-packages/simplejson/encoder.py",
>>>>>>>>>>> line 273, in default
>>>>>>>>>>> o.__class__.__name__)
>>>>>>>>>>> TypeError: Object of type buffer is not JSON serializable
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> This error exists on the master branch on Python 2.7, it is not
>>>>>>>>>> caused by the patch. It is also not a regression of my previous Query
>>>>>>>>>> History Enhancement patch, I checked.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> 2) The patch is not compatible with older pyscopg2 (2.7.7). I
>>>>>>>>>>> get below error in View/Edit data.
>>>>>>>>>>> [image: Screenshot 2019-08-23 at 10.53.30.png]
>>>>>>>>>>> 2019-08-23 10:53:12,020: ERROR flask.app: 'table_column'
>>>>>>>>>>> Traceback (most recent call last):
>>>>>>>>>>> File
>>>>>>>>>>> "/Users/adityatoshniwal/venv/pypg35/lib/python3.5/site-packages/flask/app.py",
>>>>>>>>>>> line 1813, in full_dispatch_request
>>>>>>>>>>> rv = self.dispatch_request()
>>>>>>>>>>> File
>>>>>>>>>>> "/Users/adityatoshniwal/venv/pypg35/lib/python3.5/site-packages/flask/app.py",
>>>>>>>>>>> line 1799, in dispatch_request
>>>>>>>>>>> return self.view_functions[rule.endpoint](**req.view_args)
>>>>>>>>>>> File
>>>>>>>>>>> "/Users/adityatoshniwal/venv/pypg35/lib/python3.5/site-packages/flask_login/utils.py",
>>>>>>>>>>> line 261, in decorated_view
>>>>>>>>>>> return func(*args, **kwargs)
>>>>>>>>>>> File
>>>>>>>>>>> "/Users/adityatoshniwal/projects/pgadmin4/web/pgadmin/tools/sqleditor/__init__.py",
>>>>>>>>>>> line 435, in poll
>>>>>>>>>>> columns = trans_obj.get_columns_types(conn)
>>>>>>>>>>> File
>>>>>>>>>>> "/Users/adityatoshniwal/projects/pgadmin4/web/pgadmin/tools/sqleditor/command.py",
>>>>>>>>>>> line 688, in get_columns_types
>>>>>>>>>>> table_oid=table_oid)
>>>>>>>>>>> File
>>>>>>>>>>> "/Users/adityatoshniwal/projects/pgadmin4/web/pgadmin/tools/sqleditor/utils/get_column_types.py",
>>>>>>>>>>> line 38, in get_columns_types
>>>>>>>>>>> if row['oid'] == col['table_column']:
>>>>>>>>>>> KeyError: 'table_column'
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I recall Dave mentioning that it is okay to use psycopg 2.8+
>>>>>>>>>> features and that it was a requirement of pgAdmin 4 now (clearly mentioned
>>>>>>>>>> in requirements.txt too).
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> I did - then we had the whole table_oid issue on Debian/Ubuntu :-(
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> However, I updated the patch for backwards compatibility. Please
>>>>>>>>>> find an updated patch attached.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> *Yosry Muhammad Yosry*
>>>>>>>>>>
>>>>>>>>>> Computer Engineering student,
>>>>>>>>>> The Faculty of Engineering,
>>>>>>>>>> Cairo University (2021).
>>>>>>>>>> Class representative of CMP 2021.
>>>>>>>>>> https://www.linkedin.com/in/yosrym93/
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Dave Page
>>>>>>>>> Blog: http://pgsnake.blogspot.com
>>>>>>>>> Twitter: @pgsnake
>>>>>>>>>
>>>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com
>>>>>>>>> The Enterprise PostgreSQL Company
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> *Yosry Muhammad Yosry*
>>>>>>>>
>>>>>>>> Computer Engineering student,
>>>>>>>> The Faculty of Engineering,
>>>>>>>> Cairo University (2021).
>>>>>>>> Class representative of CMP 2021.
>>>>>>>> https://www.linkedin.com/in/yosrym93/
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Thanks and Regards,
>>>>>>> Aditya Toshniwal
>>>>>>> Software Engineer | EnterpriseDB India | Pune
>>>>>>> "Don't Complain about Heat, Plant a TREE"
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Dave Page
>>>>>> Blog: http://pgsnake.blogspot.com
>>>>>> Twitter: @pgsnake
>>>>>>
>>>>>> EnterpriseDB UK: http://www.enterprisedb.com
>>>>>> The Enterprise PostgreSQL Company
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> *Yosry Muhammad Yosry*
>>>>>
>>>>> Computer Engineering student,
>>>>> The Faculty of Engineering,
>>>>> Cairo University (2021).
>>>>> Class representative of CMP 2021.
>>>>> https://www.linkedin.com/in/yosrym93/
>>>>>
>>>>
>>>>
>>>> --
>>>> Dave Page
>>>> Blog: http://pgsnake.blogspot.com
>>>> Twitter: @pgsnake
>>>>
>>>> EnterpriseDB UK: http://www.enterprisedb.com
>>>> The Enterprise PostgreSQL Company
>>>>
>>>
>>>
>>> --
>>> *Yosry Muhammad Yosry*
>>>
>>> Computer Engineering student,
>>> The Faculty of Engineering,
>>> Cairo University (2021).
>>> Class representative of CMP 2021.
>>> https://www.linkedin.com/in/yosrym93/
>>>
>>
>>
>> --
>> Dave Page
>> Blog: http://pgsnake.blogspot.com
>> Twitter: @pgsnake
>>
>> EnterpriseDB UK: http://www.enterprisedb.com
>> The Enterprise PostgreSQL Company
>>
>
>
> --
> *Yosry Muhammad Yosry*
>
> Computer Engineering student,
> The Faculty of Engineering,
> Cairo University (2021).
> Class representative of CMP 2021.
> https://www.linkedin.com/in/yosrym93/
>

--
Thanks and Regards,
Aditya Toshniwal
Software Engineer | EnterpriseDB India | Pune
"Don't Complain about Heat, Plant a TREE"

Attachment Content-Type Size
editable_readonly_columns_v6.patch application/octet-stream 76.2 KB

In response to

Responses

Browse pgadmin-hackers by date

  From Date Subject
Next Message Yosry Muhammad 2019-08-24 06:36:13 Re: [GSoC][New Feature] Editable and Read-only Columns
Previous Message Yosry Muhammad 2019-08-23 22:38:45 Re: [GSoC][New Feature] Editable and Read-only Columns