Re: [GSoC][New Feature] Editable and Read-only Columns

Thanks patch applied.

On Sat, Aug 24, 2019 at 12:13 PM Yosry Muhammad <yosrym93(at)gmail(dot)com> wrote:

> Makes sense. Thanks a lot for your help :)
>
> On Sat, Aug 24, 2019, 8:40 AM Aditya Toshniwal <
> aditya(dot)toshniwal(at)enterprisedb(dot)com> wrote:
>
>> Hi Yosry,
>>
>> My guess is because selenium is running faster on our machine. Depends on
>> machine.
>>
>> On Sat, Aug 24, 2019, 12:06 Yosry Muhammad <yosrym93(at)gmail(dot)com> wrote:
>>
>>> Hi Aditya,
>>>
>>> Thanks a lot.
>>>
>>> Do you have any idea why this problem never occurred on my machine by
>>> the way?
>>>
>>> On Sat, Aug 24, 2019, 8:18 AM Aditya Toshniwal <
>>> aditya(dot)toshniwal(at)enterprisedb(dot)com> wrote:
>>>
>>>> Hi Yosry,
>>>>
>>>> It's failing for me as well. I debugged and found that, sometimes the
>>>> grid render is delayed (in microseconds) after query execution. Because of
>>>> which, the cell you had selected using xpath is actually the older one. And
>>>> when you're trying to set the value the cell no longer exists as the new
>>>> cell is rendered now. That's why stale element exception. Adding below code
>>>> solved the problem for me. Attached is the patch.
>>>>
>>>> import time
>>>> time.sleep(0.5)
>>>> for column_index, should_be_editable in enumerated_should_be_editable:
>>>>
>>>>
>>>> On Sat, Aug 24, 2019 at 4:08 AM Yosry Muhammad <yosrym93(at)gmail(dot)com>
>>>> wrote:
>>>>
>>>>> Hi Dave,
>>>>>
>>>>> I am now using the same versions of Python, Chrome and chromedriver.
>>>>> The tests still pass every time. Are you sure you applied the latest patch?
>>>>> I re-created the patch as a sanity check, please find it attached.
>>>>>
>>>>> I tried running the specific test a lot of times, passes every time.
>>>>> The only difference is that I am running Ubuntu, otherwise, everything is
>>>>> the same.
>>>>>
>>>>>
>>>>> On Fri, Aug 23, 2019 at 5:00 PM Dave Page <dpage(at)pgadmin(dot)org> wrote:
>>>>>
>>>>>> Hi
>>>>>>
>>>>>> I've run it again 4 more times and it fails consistently for me. I'm
>>>>>> running on macOS 10.14.5 with Chrome 76.0.3809.100.
>>>>>>
>>>>>> On Fri, Aug 23, 2019 at 3:17 PM Yosry Muhammad <yosrym93(at)gmail(dot)com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Dave,
>>>>>>>
>>>>>>> The tests pass on my device normally. I am using Python 3.6 on
>>>>>>> Ubuntu. I have ran all tests using "make check" and they all succeeded. I
>>>>>>> also ran feature_tests/query_tool_journey_test.py multiple times on its own
>>>>>>> using "cd web && python regression/runtests.py --pkg feature_tests
>>>>>>> --modules query_tool_journey_test" and it passed successfully. Could you
>>>>>>> try again?
>>>>>>>
>>>>>>> Generally, I have found that StaleElementReferenceException occurs
>>>>>>> intermittently on various feature tests.
>>>>>>>
>>>>>>> On Fri, Aug 23, 2019 at 3:46 PM Dave Page <dpage(at)pgadmin(dot)org> wrote:
>>>>>>>
>>>>>>>> Thanks. Unfortunately this is failing the tests:
>>>>>>>>
>>>>>>>>
>>>>>>>> ======================================================================
>>>>>>>> ERROR: runTest
>>>>>>>> (pgadmin.feature_tests.query_tool_journey_test.QueryToolJourneyTest)
>>>>>>>> Tests the path through the query tool
>>>>>>>>
>>>>>>>> ----------------------------------------------------------------------
>>>>>>>> Traceback (most recent call last):
>>>>>>>> File
>>>>>>>> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_journey_test.py",
>>>>>>>> line 85, in runTest
>>>>>>>> self._test_updatable_resultset()
>>>>>>>> File
>>>>>>>> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_journey_test.py",
>>>>>>>> line 254, in _test_updatable_resultset
>>>>>>>> discard_changes_modal=True)
>>>>>>>> File
>>>>>>>> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_journey_test.py",
>>>>>>>> line 417, in _check_query_results_editable
>>>>>>>> is_editable = self._check_cell_editable(column_index)
>>>>>>>> File
>>>>>>>> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_journey_test.py",
>>>>>>>> line 430, in _check_cell_editable
>>>>>>>> cell_value = int(cell_el.text)
>>>>>>>> File
>>>>>>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python3.7/site-packages/selenium/webdriver/remote/webelement.py",
>>>>>>>> line 76, in text
>>>>>>>> return self._execute(Command.GET_ELEMENT_TEXT)['value']
>>>>>>>> File
>>>>>>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python3.7/site-packages/selenium/webdriver/remote/webelement.py",
>>>>>>>> line 628, in _execute
>>>>>>>> return self._parent.execute(command, params)
>>>>>>>> File
>>>>>>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py",
>>>>>>>> line 312, in execute
>>>>>>>> self.error_handler.check_response(response)
>>>>>>>> File
>>>>>>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python3.7/site-packages/selenium/webdriver/remote/errorhandler.py",
>>>>>>>> line 242, in check_response
>>>>>>>> raise exception_class(message, screen, stacktrace)
>>>>>>>> selenium.common.exceptions.StaleElementReferenceException: Message:
>>>>>>>> stale element reference: element is not attached to the page document
>>>>>>>> (Session info: chrome=76.0.3809.100)
>>>>>>>> (Driver info: chromedriver=76.0.3809.126
>>>>>>>> (d80a294506b4c9d18015e755cee48f953ddc3f2f-refs/branch-heads/3809(at){#1024}),platform=Mac
>>>>>>>> OS X 10.14.5 x86_64)
>>>>>>>>
>>>>>>>> Without the patch, the tests pass.
>>>>>>>>
>>>>>>>>
>>>>>>>> On Fri, Aug 23, 2019 at 1:10 PM Yosry Muhammad <yosrym93(at)gmail(dot)com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Please find an updated patch attached.
>>>>>>>>>
>>>>>>>>> On Fri, Aug 23, 2019 at 1:22 PM Dave Page <dpage(at)pgadmin(dot)org>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Except it now doesn't apply as the async connection/rendering
>>>>>>>>>> patch was in the queue right ahead of it :-(.
>>>>>>>>>>
>>>>>>>>>> Can I get a rebased patch please?
>>>>>>>>>>
>>>>>>>>>> On Fri, Aug 23, 2019 at 11:57 AM Aditya Toshniwal <
>>>>>>>>>> aditya(dot)toshniwal(at)enterprisedb(dot)com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Excellent !! Looks good to me.
>>>>>>>>>>>
>>>>>>>>>>> On Fri, Aug 23, 2019 at 2:23 PM Yosry Muhammad <
>>>>>>>>>>> yosrym93(at)gmail(dot)com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> I updated the patch for backwards compatibility anyway.
>>>>>>>>>>>>
>>>>>>>>>>>> On Fri, Aug 23, 2019 at 10:45 AM Dave Page <dpage(at)pgadmin(dot)org>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Fri, Aug 23, 2019 at 9:43 AM Yosry Muhammad <
>>>>>>>>>>>>> yosrym93(at)gmail(dot)com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi Aditya,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Fri, Aug 23, 2019 at 7:33 AM Aditya Toshniwal <
>>>>>>>>>>>>>> aditya(dot)toshniwal(at)enterprisedb(dot)com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi Yosry,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> The previously mentioned issues seems to be fixed. Below are
>>>>>>>>>>>>>>> few issues:
>>>>>>>>>>>>>>> 1) On Python 2.7, I get below error when opening query tool:
>>>>>>>>>>>>>>> 2019-08-23 10:49:09,329: ERROR flask.app: Object of type
>>>>>>>>>>>>>>> buffer is not JSON serializable
>>>>>>>>>>>>>>> Traceback (most recent call last):
>>>>>>>>>>>>>>> File
>>>>>>>>>>>>>>> "/Users/adityatoshniwal/venv/pypg27/lib/python2.7/site-packages/flask/app.py",
>>>>>>>>>>>>>>> line 1813, in full_dispatch_request
>>>>>>>>>>>>>>> rv = self.dispatch_request()
>>>>>>>>>>>>>>> File
>>>>>>>>>>>>>>> "/Users/adityatoshniwal/venv/pypg27/lib/python2.7/site-packages/flask/app.py",
>>>>>>>>>>>>>>> line 1799, in dispatch_request
>>>>>>>>>>>>>>> return
>>>>>>>>>>>>>>> self.view_functions[rule.endpoint](**req.view_args)
>>>>>>>>>>>>>>> File
>>>>>>>>>>>>>>> "/Users/adityatoshniwal/venv/pypg27/lib/python2.7/site-packages/flask_login/utils.py",
>>>>>>>>>>>>>>> line 261, in decorated_view
>>>>>>>>>>>>>>> return func(*args, **kwargs)
>>>>>>>>>>>>>>> File
>>>>>>>>>>>>>>> "/Users/adityatoshniwal/projects/pgadmin4/web/pgadmin/tools/sqleditor/__init__.py",
>>>>>>>>>>>>>>> line 1544, in get_query_history
>>>>>>>>>>>>>>> return QueryHistory.get(current_user.id, trans_obj.sid,
>>>>>>>>>>>>>>> conn.db)
>>>>>>>>>>>>>>> File
>>>>>>>>>>>>>>> "/Users/adityatoshniwal/projects/pgadmin4/web/pgadmin/tools/sqleditor/utils/query_history.py",
>>>>>>>>>>>>>>> line 21, in get
>>>>>>>>>>>>>>> 'result': [rec.query_info for rec in result]
>>>>>>>>>>>>>>> File
>>>>>>>>>>>>>>> "/Users/adityatoshniwal/projects/pgadmin4/web/pgadmin/utils/ajax.py", line
>>>>>>>>>>>>>>> 75, in make_json_response
>>>>>>>>>>>>>>> separators=(',', ':'), encoding=encoding),
>>>>>>>>>>>>>>> File
>>>>>>>>>>>>>>> "/Users/adityatoshniwal/venv/pypg27/lib/python2.7/site-packages/simplejson/__init__.py",
>>>>>>>>>>>>>>> line 399, in dumps
>>>>>>>>>>>>>>> **kw).encode(obj)
>>>>>>>>>>>>>>> File
>>>>>>>>>>>>>>> "/Users/adityatoshniwal/venv/pypg27/lib/python2.7/site-packages/simplejson/encoder.py",
>>>>>>>>>>>>>>> line 296, in encode
>>>>>>>>>>>>>>> chunks = self.iterencode(o, _one_shot=True)
>>>>>>>>>>>>>>> File
>>>>>>>>>>>>>>> "/Users/adityatoshniwal/venv/pypg27/lib/python2.7/site-packages/simplejson/encoder.py",
>>>>>>>>>>>>>>> line 378, in iterencode
>>>>>>>>>>>>>>> return _iterencode(o, 0)
>>>>>>>>>>>>>>> File
>>>>>>>>>>>>>>> "/Users/adityatoshniwal/projects/pgadmin4/web/pgadmin/utils/ajax.py", line
>>>>>>>>>>>>>>> 30, in default
>>>>>>>>>>>>>>> return json.JSONEncoder.default(self, obj)
>>>>>>>>>>>>>>> File
>>>>>>>>>>>>>>> "/Users/adityatoshniwal/venv/pypg27/lib/python2.7/site-packages/simplejson/encoder.py",
>>>>>>>>>>>>>>> line 273, in default
>>>>>>>>>>>>>>> o.__class__.__name__)
>>>>>>>>>>>>>>> TypeError: Object of type buffer is not JSON serializable
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> This error exists on the master branch on Python 2.7, it is
>>>>>>>>>>>>>> not caused by the patch. It is also not a regression of my previous Query
>>>>>>>>>>>>>> History Enhancement patch, I checked.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 2) The patch is not compatible with older pyscopg2 (2.7.7).
>>>>>>>>>>>>>>> I get below error in View/Edit data.
>>>>>>>>>>>>>>> [image: Screenshot 2019-08-23 at 10.53.30.png]
>>>>>>>>>>>>>>> 2019-08-23 10:53:12,020: ERROR flask.app: 'table_column'
>>>>>>>>>>>>>>> Traceback (most recent call last):
>>>>>>>>>>>>>>> File
>>>>>>>>>>>>>>> "/Users/adityatoshniwal/venv/pypg35/lib/python3.5/site-packages/flask/app.py",
>>>>>>>>>>>>>>> line 1813, in full_dispatch_request
>>>>>>>>>>>>>>> rv = self.dispatch_request()
>>>>>>>>>>>>>>> File
>>>>>>>>>>>>>>> "/Users/adityatoshniwal/venv/pypg35/lib/python3.5/site-packages/flask/app.py",
>>>>>>>>>>>>>>> line 1799, in dispatch_request
>>>>>>>>>>>>>>> return
>>>>>>>>>>>>>>> self.view_functions[rule.endpoint](**req.view_args)
>>>>>>>>>>>>>>> File
>>>>>>>>>>>>>>> "/Users/adityatoshniwal/venv/pypg35/lib/python3.5/site-packages/flask_login/utils.py",
>>>>>>>>>>>>>>> line 261, in decorated_view
>>>>>>>>>>>>>>> return func(*args, **kwargs)
>>>>>>>>>>>>>>> File
>>>>>>>>>>>>>>> "/Users/adityatoshniwal/projects/pgadmin4/web/pgadmin/tools/sqleditor/__init__.py",
>>>>>>>>>>>>>>> line 435, in poll
>>>>>>>>>>>>>>> columns = trans_obj.get_columns_types(conn)
>>>>>>>>>>>>>>> File
>>>>>>>>>>>>>>> "/Users/adityatoshniwal/projects/pgadmin4/web/pgadmin/tools/sqleditor/command.py",
>>>>>>>>>>>>>>> line 688, in get_columns_types
>>>>>>>>>>>>>>> table_oid=table_oid)
>>>>>>>>>>>>>>> File
>>>>>>>>>>>>>>> "/Users/adityatoshniwal/projects/pgadmin4/web/pgadmin/tools/sqleditor/utils/get_column_types.py",
>>>>>>>>>>>>>>> line 38, in get_columns_types
>>>>>>>>>>>>>>> if row['oid'] == col['table_column']:
>>>>>>>>>>>>>>> KeyError: 'table_column'
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I recall Dave mentioning that it is okay to use psycopg 2.8+
>>>>>>>>>>>>>> features and that it was a requirement of pgAdmin 4 now (clearly mentioned
>>>>>>>>>>>>>> in requirements.txt too).
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> I did - then we had the whole table_oid issue on Debian/Ubuntu
>>>>>>>>>>>>> :-(
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>> However, I updated the patch for backwards compatibility.
>>>>>>>>>>>>>> Please find an updated patch attached.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> *Yosry Muhammad Yosry*
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Computer Engineering student,
>>>>>>>>>>>>>> The Faculty of Engineering,
>>>>>>>>>>>>>> Cairo University (2021).
>>>>>>>>>>>>>> Class representative of CMP 2021.
>>>>>>>>>>>>>> https://www.linkedin.com/in/yosrym93/
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> Dave Page
>>>>>>>>>>>>> Blog: http://pgsnake.blogspot.com
>>>>>>>>>>>>> Twitter: @pgsnake
>>>>>>>>>>>>>
>>>>>>>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com
>>>>>>>>>>>>> The Enterprise PostgreSQL Company
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> *Yosry Muhammad Yosry*
>>>>>>>>>>>>
>>>>>>>>>>>> Computer Engineering student,
>>>>>>>>>>>> The Faculty of Engineering,
>>>>>>>>>>>> Cairo University (2021).
>>>>>>>>>>>> Class representative of CMP 2021.
>>>>>>>>>>>> https://www.linkedin.com/in/yosrym93/
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Thanks and Regards,
>>>>>>>>>>> Aditya Toshniwal
>>>>>>>>>>> Software Engineer | EnterpriseDB India | Pune
>>>>>>>>>>> "Don't Complain about Heat, Plant a TREE"
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Dave Page
>>>>>>>>>> Blog: http://pgsnake.blogspot.com
>>>>>>>>>> Twitter: @pgsnake
>>>>>>>>>>
>>>>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com
>>>>>>>>>> The Enterprise PostgreSQL Company
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> *Yosry Muhammad Yosry*
>>>>>>>>>
>>>>>>>>> Computer Engineering student,
>>>>>>>>> The Faculty of Engineering,
>>>>>>>>> Cairo University (2021).
>>>>>>>>> Class representative of CMP 2021.
>>>>>>>>> https://www.linkedin.com/in/yosrym93/
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Dave Page
>>>>>>>> Blog: http://pgsnake.blogspot.com
>>>>>>>> Twitter: @pgsnake
>>>>>>>>
>>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com
>>>>>>>> The Enterprise PostgreSQL Company
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> *Yosry Muhammad Yosry*
>>>>>>>
>>>>>>> Computer Engineering student,
>>>>>>> The Faculty of Engineering,
>>>>>>> Cairo University (2021).
>>>>>>> Class representative of CMP 2021.
>>>>>>> https://www.linkedin.com/in/yosrym93/
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Dave Page
>>>>>> Blog: http://pgsnake.blogspot.com
>>>>>> Twitter: @pgsnake
>>>>>>
>>>>>> EnterpriseDB UK: http://www.enterprisedb.com
>>>>>> The Enterprise PostgreSQL Company
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> *Yosry Muhammad Yosry*
>>>>>
>>>>> Computer Engineering student,
>>>>> The Faculty of Engineering,
>>>>> Cairo University (2021).
>>>>> Class representative of CMP 2021.
>>>>> https://www.linkedin.com/in/yosrym93/
>>>>>
>>>>
>>>>
>>>> --
>>>> Thanks and Regards,
>>>> Aditya Toshniwal
>>>> Software Engineer | EnterpriseDB India | Pune
>>>> "Don't Complain about Heat, Plant a TREE"
>>>>
>>>

--
*Thanks & Regards*
*Akshay Joshi*

*Sr. Software Architect*
*EnterpriseDB Software India Private Limited*
*Mobile: +91 976-788-8246*