Re: [pgAdmin4][RM#3289] Can't query SQL_ASCII database.

Hello Aditya,

It looks ok and it passes CI.

We have some recommendations:
- These look like 2 different changes so they should be in separated commits
- Do we have test coverage for the bug that you are talking about? If not
we should, to ensure this problem does not happen again in a future change.

Thanks
Victoria && Joao

On Wed, May 30, 2018 at 3:06 AM Aditya Toshniwal <
aditya(dot)toshniwal(at)enterprisedb(dot)com> wrote:

> Hi Hackers,
>
> PFA updated patch after all the permutations, combinations for encoding
> for SQL_ASCII database. Also fixed a small glitch for sql editor
> connection status check.
>
> Please note, ERROR: invalid byte sequence for encoding "UTF8": 0xe9 0x73
> is a Postgres DB error and not pgAdmin4 error.
>
>
>
> You need to change client_encoding to the appropriate. After changing
> client_encoding using command - set client_encoding='XYZ', it will give not
> give error.
>
>
>
>
>
>
>
>
> Thanks and Regards,
> Aditya Toshniwal
> Software Engineer | EnterpriseDB Software Solutions | Pune
> "Don't Complain about Heat, Plant a tree"
>
> On Wed, May 23, 2018 at 10:13 AM, Aditya Toshniwal <
> aditya(dot)toshniwal(at)enterprisedb(dot)com> wrote:
>
>> Thank you Victoria, Anthony.
>>
>> Thanks and Regards,
>> Aditya Toshniwal
>> Software Engineer | EnterpriseDB Software Solutions | Pune
>> "Don't Complain about Heat, Plant a tree"
>>
>> On Tue, May 22, 2018 at 7:15 PM, Victoria Henry <vhenry(at)pivotal(dot)io>
>> wrote:
>>
>>> Hi Aditya,
>>>
>>> We made a minor change to make the patch so the python linter can pass.
>>> Attached is the change we made.
>>> Everything else looks good.
>>>
>>> Sincerely,
>>>
>>> Victoria & Anthony
>>>
>>> On Tue, May 22, 2018 at 4:46 AM Aditya Toshniwal <
>>> aditya(dot)toshniwal(at)enterprisedb(dot)com> wrote:
>>>
>>>> Hi,
>>>>
>>>> PFA updated patch. Linter issues are fixed ( we dont have any linter
>>>> setup for python :-( )
>>>> Regarding test cases, they run successfully on my system and the reason
>>>> it failed for pivotal is timeout exception. I am sorry I can't help with
>>>> that.
>>>>
>>>> Traceback (most recent call last):
>>>> File
>>>> "/tmp/build/a453582b/pgadmin-repo/web/pgadmin/feature_tests/keyboard_shortcut_test.py",
>>>> line 52, in runTest
>>>> self._check_shortcuts()
>>>> File
>>>> "/tmp/build/a453582b/pgadmin-repo/web/pgadmin/feature_tests/keyboard_shortcut_test.py",
>>>> line 77, in _check_shortcuts
>>>> ") and contains(@class, 'open')]")
>>>> File
>>>> "/root/.pyenv/versions/pgadmin36/lib/python3.6/site-packages/selenium/webdriver/support/wait.py",
>>>> line 80, in until
>>>> raise TimeoutException(message, screen, stacktrace)
>>>> selenium.common.exceptions.TimeoutException: Message:
>>>>
>>>> Thanks and Regards,
>>>> Aditya Toshniwal
>>>> Software Engineer | EnterpriseDB Software Solutions | Pune
>>>> "Don't Complain about Heat, Plant a tree"
>>>>
>>>> On Tue, May 22, 2018 at 1:37 PM, Dave Page <dpage(at)pgadmin(dot)org> wrote:
>>>>
>>>>> Hi
>>>>>
>>>>> Pivotal's buildbot is showing problems with this patch:
>>>>>
>>>>>
>>>>> https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/66
>>>>> (linter failed)
>>>>>
>>>>> https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-tests/builds/84
>>>>> (tests failed)
>>>>>
>>>>>
>>>>> On Tue, May 22, 2018 at 7:05 AM, Aditya Toshniwal <
>>>>> aditya(dot)toshniwal(at)enterprisedb(dot)com> wrote:
>>>>>
>>>>>> Hi Hackers,
>>>>>>
>>>>>> PFA patch for RM#3289 where decode error was thrown on querying a
>>>>>> SQL_ASCII database table. Please note, this problem occurs only on windows.
>>>>>> Sample insert - insert into test_tab values ('é');
>>>>>>
>>>>>> psycopg2 has a encodings dictionary where Postgres Database Encodings
>>>>>> are mapped to python equivalent. It uses 'ascii' decoder of python to
>>>>>> decode for SQL_ASCII encoding. If data has characters beyond the limit of
>>>>>> ascii then it failed. The solution would be to use utf_8 decoder instead of
>>>>>> ascii. I tried setting the client_encoding using
>>>>>> set_client_encoding('UTF8') method of a psycopg2 connection but no luck
>>>>>> (also its not allowed for async connection). I also tried executing "SET
>>>>>> CLIENT_ENCODING='UTF8'" but it didn't work too.
>>>>>> So, as in the patch, I had to set encodings dict value directly to
>>>>>> 'utf_8' and it seems to be working. Please note, the same is added to
>>>>>> psycopg3 milestones
>>>>>> https://github.com/psycopg/psycopg2/milestone/4
>>>>>>
>>>>>> Also fixed a small glitch for sql editor connection status check.
>>>>>>
>>>>>> Kindly review.
>>>>>>
>>>>>> Thanks and Regards,
>>>>>> Aditya Toshniwal
>>>>>> Software Engineer | EnterpriseDB Software Solutions | Pune
>>>>>> "Don't Complain about Heat, Plant a tree"
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Dave Page
>>>>> Blog: http://pgsnake.blogspot.com
>>>>> Twitter: @pgsnake
>>>>>
>>>>> EnterpriseDB UK: http://www.enterprisedb.com
>>>>> The Enterprise PostgreSQL Company
>>>>>
>>>>
>>>>
>>
>