Re: Re: [pgAdmin4][Patch]: Fixes #1986 - Properly handle non-ascii characters while loading & saving file

Hi

On Thu, Nov 24, 2016 at 10:11 AM, Surinder Kumar
<surinder(dot)kumar(at)enterprisedb(dot)com> wrote:
> Please ignore above patch. because it breaks when we save a file with name
> containing ascii chars in py35
> It is fixed. Please find attached patch and review.
>
> On Thu, Nov 24, 2016 at 2:56 PM, Surinder Kumar
> <surinder(dot)kumar(at)enterprisedb(dot)com> wrote:
>>
>> Hi
>>
>> Issue:
>> The content will not write to disk if it contains ascii characters.
>>
>> Solution:
>> Encode/Decode the content before writing/reading the file.
>>
>> By the way, the file/folder name should also encode/decode, a similar
>> issue is already logged
>> #1940. I will work on it.
>>
>> Please find attached path and review.

I'm running into all sorts of problems testing this. I'm on Mac, using
Apple's Python 2.7.10 build.

1) Sometimes when opening the file manager to open a file, I get the
following exception. It appears (but I'm not certain) to be when I've
already opened a file with non-ASCII chars in the name and I've been
prompted to discard the changes.

2016-11-25 09:42:20,087: INFO werkzeug: 127.0.0.1 - - [25/Nov/2016
09:42:20] "GET /file_manager/ HTTP/1.1" 500 -
Traceback (most recent call last):
File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 2000, in __call__
return self.wsgi_app(environ, start_response)
File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1991, in wsgi_app
response = self.make_response(self.handle_exception(e))
File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1567, in handle_exception
reraise(exc_type, exc_value, tb)
File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1988, in wsgi_app
response = self.full_dispatch_request()
File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1643, in full_dispatch_request
response = self.process_response(response)
File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1864, in process_response
self.save_session(ctx.session, response)
File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 926, in save_session
return self.session_interface.save_session(self, session, response)
File "/Users/dpage/git/pgadmin4/web/pgadmin/utils/session.py", line
267, in save_session
self.manager.put(session)
File "/Users/dpage/git/pgadmin4/web/pgadmin/utils/session.py", line
144, in put
self.parent.put(session)
File "/Users/dpage/git/pgadmin4/web/pgadmin/utils/session.py", line
220, in put
f
RuntimeError: dictionary changed size during iteration

2) If I use "Save As" and select an existing file with non-ASCII chars
in the name, the filename box shows an encoded filename. See
screenshot 1.

3) If I open an existing file with non-ASCII chars in the name, the
title of the Query Tool tab is shown encoded. See screenshot 2. Note
that if I save a new file, the name is displayed correctly.

4) Just selecting a file to open (without then clicking open) in the
file manager gives the following exception:

2016-11-25 09:51:37,585: INFO werkzeug: 127.0.0.1 - - [25/Nov/2016
09:51:37] "POST /file_manager/filemanager/2265288/ HTTP/1.1" 500 -
Traceback (most recent call last):
File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 2000, in __call__
return self.wsgi_app(environ, start_response)
File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1991, in wsgi_app
response = self.make_response(self.handle_exception(e))
File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1567, in handle_exception
reraise(exc_type, exc_value, tb)
File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1988, in wsgi_app
response = self.full_dispatch_request()
File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1641, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1544, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1639, in full_dispatch_request
rv = self.dispatch_request()
File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1625, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask_login.py",
line 792, in decorated_view
return func(*args, **kwargs)
File "/Users/dpage/git/pgadmin4/web/pgadmin/misc/file_manager/__init__.py",
line 820, in file_manager
return getattr(myFilemanager, mode)(**kwargs)
File "/Users/dpage/git/pgadmin4/web/pgadmin/misc/file_manager/__init__.py",
line 518, in getinfo
orig_path = "{0}{1}".format(self.dir, path)
UnicodeEncodeError: 'ascii' codec can't encode character u'\u0303' in
position 18: ordinal not in range(128)

Please test thoroughly in both Python 2 and 3 before re-submitting.

Thanks.

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company