Re: Unified server/desktop config

Hi

This patch includes the fix where this patch breaks when feature tests are
run.

Set builtins.SERVER_MODE if it is present in globals()['SERVER_MODE'] else
set to None.

Please find updated patch.
Thanks,
Surinder
​

On Wed, Aug 9, 2017 at 11:57 AM, Surinder Kumar <
surinder(dot)kumar(at)enterprisedb(dot)com> wrote:

> Sure, I will update.
>
> On Wed, Aug 9, 2017 at 11:17 AM, Dave Page <dpage(at)pgadmin(dot)org> wrote:
>
>> Please update the patch :-)
>>
>> --
>> Dave Page
>> Blog: http://pgsnake.blogspot.com
>> Twitter: @pgsnake
>>
>> EnterpriseDB UK:http://www.enterprisedb.com
>> The Enterprise PostgreSQL Company
>>
>> On 9 Aug 2017, at 05:53, Surinder Kumar <surinder(dot)kumar(at)enterprisedb(dot)com>
>> wrote:
>>
>> Hi,
>>
>> I noticed that test cases don’t run and I got an error:
>>
>> (pgAdmin_27)Laptop195:regression surinder$ python runtests.py --pkg feature_tests
>> <module '__builtin__' (built-in)>
>> Traceback (most recent call last):
>> File "runtests.py", line 45, in <module>
>> import config
>> File "/Users/surinder/Documents/Projects/pgadmin4/web/config.py", line 121, in <module>
>> if builtins.SERVER_MODE is None:
>> AttributeError: 'module' object has no attribute'
>> SERVER_MODE
>> '
>>
>> I think this is because we are setting​ ​first​ ​builtins.SERVER_MODE​ in​ ​pgAdmin4.py
>> but in test cases pgAdmin4.py is not being called.​
>>
>>
>> On Tue, Aug 8, 2017 at 4:03 PM, Dave Page <dpage(at)pgadmin(dot)org> wrote:
>>
>>>
>>>
>>> On Tue, Aug 8, 2017 at 11:27 AM, Surinder Kumar <
>>> surinder(dot)kumar(at)enterprisedb(dot)com> wrote:
>>>
>>>> On Tue, Aug 8, 2017 at 1:47 PM, Dave Page <dpage(at)pgadmin(dot)org> wrote:
>>>>
>>>>
>>>>>
>>>>> On Tue, Aug 8, 2017 at 7:18 AM, Surinder Kumar <
>>>>> surinder(dot)kumar(at)enterprisedb(dot)com> wrote:
>>>>>
>>>>>> Hi
>>>>>>
>>>>>> On Ubuntu-14.04, I got error Application Server couldn't be contacted
>>>>>> :
>>>>>>
>>>>>> *Steps performed:*
>>>>>>
>>>>>> - I have already installed pgAdmin4-1.4 which come with
>>>>>> PostgreSQL-9.6 installer.
>>>>>> then I run root(at)ubuntu:/opt/PostgreSQL/9.6/pgAdmin 4/bin#
>>>>>> ./pgAdmin4./.
>>>>>> - Now took latest git pull from HEAD
>>>>>> - Apply unified_config.diff patch.
>>>>>> - Then compiled pgAdmin4 in runtime and then run ./pgAdmin.
>>>>>> - Got error Application Server couldn't be contacted.
>>>>>>
>>>>>> But when I ran ./pgAdmin4 for the second time. pgAdmin4 runs without
>>>>>> any issue.
>>>>>> I didn’t get any error on the terminal and log file.
>>>>>> ​ I couldn't find why it gives this error.​
>>>>>>
>>>>>
>>>>> I know Fahar has run into this with existing releases on Ubuntu. If
>>>>> you enable debugging, can you see any clues? I assume it's going round the
>>>>> retry loop before aborting?
>>>>>
>>>>>> *Another issue related to Alembic:*
>>>>>>
>>>>>> If I am running pgAdmin4 already installed on my machine, then I
>>>>>> upgrade pgAdmin4 using Python wheel:
>>>>>>
>>>>>> (test_p27) surinder(at)ubuntu:~/virtualenvs/test_p27$ python
>>>>>> ~/virtualenvs/test_p27/lib/python2.7/site-packages/pgadmin4/
>>>>>> pgAdmin4.py
>>>>>>
>>>>>> *I am getting error:*
>>>>>>
>>>>>> alembic.util.exc.CommandError: Can't locate revision identified by
>>>>>> 'd85a62333272'
>>>>>>
>>>>>> To fix this, I have to delete existing pgadmin4.db file. I don’t
>>>>>> know if it is a valid case or should I log an RM if it is?
>>>>>>
>>>>> That's an annoying side-effect of the move to Alembic. The previous DB
>>>>> code would quite happily (and intentionally) run with a newer version of
>>>>> the database, however, the new code does not. You'll see this issue
>>>>> whenever you've run a newer version of pgAdmin and then go back to an older
>>>>> version that uses an older schema version.
>>>>>
>>>>> It would be nice to change this so it doesn't complain - but we'd have
>>>>> to be cautious to only break compatibility on major releases.
>>>>>
>>>>>
>>>>>
>>>>>> Apart for this, I didn’t see any functionality break. It works!!
>>>>>> ​
>>>>>>
>>>>>> I liked the approach to set SEVER_MODE in runtime using built-ins.
>>>>>>
>>>>> Thanks. Do you think it warrants a 2.0 version number, given the
>>>>> potential for breaking existing installations (I do, but would like other
>>>>> feedback)? If we do that - and thus allow a parallel installation of 1.x
>>>>> and 2.x), how would we resolve the Alembic issue you noted such that both
>>>>> versions could be run against the same DB?
>>>>>
>>>> To resolve this issue while upgrading to newer version of pgAdmin4, we
>>>> can add a parameter version_table: 'alembic_version-1.6 in
>>>> web/migrations/env.py of upcoming pgAdmin4 source code(reference link
>>>> <https://issues.asterisk.org/jira/secure/attachment/51610/ASTERISK-24311-set-version-table.diff>
>>>> )
>>>> It will create a new version table in pgadmin4.db database and it will
>>>> instruct the Alembic to run migrations for the version stored in
>>>> table(alembic_version-1.6) when new pgAdmin4 will be installed.
>>>>
>>>> I tried to check if it works or not. But somehow I couldn’t install
>>>> pgAdmin4-1.5 from wheel.
>>>>
>>>> Alternatively I will try to generate new schema using in pgAdmin4
>>>> source code(Git HEAD). It will generate a new migration version and then i
>>>> will run pgAdmin4-1.6 from wheel having different migration number. It will
>>>> create pgadmin4.db for 1.5.
>>>> Then
>>>> I will run pgAdmin4-1.6 from source code, so migration will take place
>>>> and hopefully
>>>> ​the ​
>>>> sa
>>>> me
>>>> ​ ​
>>>> pgadmin4.db will work for newer pgAdmin4 version.
>>>>
>>>> Reference link <https://issues.asterisk.org/jira/browse/ASTERISK-24311>
>>>> where the same issue has been discussed.
>>>>
>>>
>>> Sounds good - please investigate further.
>>>
>> I spent some time, understands how Alembic works and I thought it is
>> easy using the reference link but not, It needs more R&D. I will look how
>> can we fix it.
>>
>>>
>>> Thanks!
>>>
>>>
>>>> Thanks,
>>>> Surinder
>>>>
>>>>
>>>>>
>>>>>
>>>>>> Thanks,
>>>>>> Surinder
>>>>>> ​
>>>>>>
>>>>>> On Mon, Aug 7, 2017 at 7:01 PM, Dave Page <dpage(at)pgadmin(dot)org> wrote:
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Aug 7, 2017 at 1:43 PM, Surinder Kumar <
>>>>>>> surinder(dot)kumar(at)enterprisedb(dot)com> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> The patch seems to work in Runtime mode, but fails in Server mode
>>>>>>>> with error:
>>>>>>>>
>>>>>>>> (pgAdmin_27)Laptop195:pgadmin4 surinder$ python web/pgAdmin4.py
>>>>>>>> Traceback (most recent call last):
>>>>>>>> File "web/pgAdmin4.py", line 55, in <module>
>>>>>>>> exec(open(file_quote(setupfile), 'r').read())
>>>>>>>> File "<string>", line 35, in <module>
>>>>>>>> File "/Users/surinder/Documents/Projects/pgadmin4/web/pgadmin/setup/data_directory.py", line 23, in create_app_data_directory
>>>>>>>> _create_directory_if_not_exists(os.path.dirname(config.SQLITE_PATH))
>>>>>>>> File "/Users/surinder/Documents/Projects/pgadmin4/web/pgadmin/setup/data_directory.py", line 15, in _create_directory_if_not_exists
>>>>>>>> os.mkdir(_path)
>>>>>>>> OSError: [Errno 13] Permission denied: '/var/lib/pgadmin'
>>>>>>>> (pgAdmin_27)Laptop195:pgadmin4 surinder$
>>>>>>>>
>>>>>>>> This is because the directory /var/lib/ has root only access and I
>>>>>>>> am running pgAdmin4 with the non-root user.
>>>>>>>>
>>>>>>>> Also pgadmin directory is not created.
>>>>>>>>
>>>>>>>> (pgAdmin_35)Laptop195:pgadmin4 surinder$ ls /var/lib/pgadmin
>>>>>>>> ls: /var/lib/pgadmin: No such file or directory
>>>>>>>>
>>>>>>>> I got same error with MacOSX and Ubuntu-14.04 machines irrespective
>>>>>>>> of Python version.
>>>>>>>>
>>>>>>>> Meanwhile, I am testing patch with other test cases.
>>>>>>>>
>>>>>>> That's fully expected. In the case of Linux, the packages will be
>>>>>>> responsible for creating those directories with the appropriate ownership.
>>>>>>> In other cases, the user would.
>>>>>>>
>>>>>> ​ok, I got it.​
>>>>>>
>>>>>>>
>>>>>>> There's not really much we can do about that - and it's exactly what
>>>>>>> would happen if you try to run many other packages yourself when standard
>>>>>>> *nix paths are used.
>>>>>>>
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Surinder
>>>>>>>> ​
>>>>>>>>
>>>>>>>> On Mon, Aug 7, 2017 at 4:33 PM, Surinder Kumar <
>>>>>>>> surinder(dot)kumar(at)enterprisedb(dot)com> wrote:
>>>>>>>>
>>>>>>>>> On Mon, Aug 7, 2017 at 4:11 PM, Ashesh Vashi <
>>>>>>>>> ashesh(dot)vashi(at)enterprisedb(dot)com> wrote:
>>>>>>>>>
>>>>>>>>>> On Mon, Aug 7, 2017 at 3:59 PM, Dave Page <dpage(at)pgadmin(dot)org>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Anyone?
>>>>>>>>>>>
>>>>>>>>>> Surinder - please give this one priority.
>>>>>>>>>>
>>>>>>>>> Sure
>>>>>>>>> ​.​
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>>
>>>>>>>>>> Thanks & Regards,
>>>>>>>>>>
>>>>>>>>>> Ashesh Vashi
>>>>>>>>>> EnterpriseDB INDIA: Enterprise PostgreSQL Company
>>>>>>>>>> <http://www.enterprisedb.com/>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> *http://www.linkedin.com/in/asheshvashi*
>>>>>>>>>> <http://www.linkedin.com/in/asheshvashi>
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Jul 20, 2017 at 5:03 PM, Dave Page <dpage(at)pgadmin(dot)org>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> All,
>>>>>>>>>>>>
>>>>>>>>>>>> Attached is a patch that aims to allow us to have a
>>>>>>>>>>>> standardised config that will work out of the box for both web and desktop
>>>>>>>>>>>> modes. It does this by doing two things:
>>>>>>>>>>>>
>>>>>>>>>>>> 1) The runtime sets SERVER_MODE in the Python environment
>>>>>>>>>>>> before starting the app. If this value is set, then it overrides the
>>>>>>>>>>>> default value of SERVER_MODE in the config.
>>>>>>>>>>>>
>>>>>>>>>>>> 2) The config file then offers default values for the various
>>>>>>>>>>>> file locations for both server and desktop mode, setting them appropriately
>>>>>>>>>>>> based on the derived SERVER_MODE value.
>>>>>>>>>>>>
>>>>>>>>>>>> The only downsides I can see from this are:
>>>>>>>>>>>>
>>>>>>>>>>>> - You cannot run in server mode in the runtime without manually
>>>>>>>>>>>> reconfiguring SERVER_MODE and likely a bunch of paths in config_local.py
>>>>>>>>>>>>
>>>>>>>>>>>> - If you want to override SERVER_MODE, you'll probably also
>>>>>>>>>>>> need to redefine the various paths in config_local.py.
>>>>>>>>>>>>
>>>>>>>>>>>> I don't see either being something 99.9% of users would need
>>>>>>>>>>>> though.
>>>>>>>>>>>>
>>>>>>>>>>>> Can anyone see if the patch breaks anything, or if I missed any
>>>>>>>>>>>> side effects?
>>>>>>>>>>>>
>>>>>>>>>>>> Is it likely to break things during upgrades? I suspect so...
>>>>>>>>>>>> so maybe this should prompt v2.0?
>>>>>>>>>>>>
>>>>>>>>>>>> I'd appreciate multiple reviews of this, as it could break
>>>>>>>>>>>> things. Note that I haven't yet updated the docs.
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> Dave Page
>>>>>>>>>>>> Blog: http://pgsnake.blogspot.com
>>>>>>>>>>>> Twitter: @pgsnake
>>>>>>>>>>>>
>>>>>>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com
>>>>>>>>>>>> The Enterprise PostgreSQL Company
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Dave Page
>>>>>>>>>>> Blog: http://pgsnake.blogspot.com
>>>>>>>>>>> Twitter: @pgsnake
>>>>>>>>>>>
>>>>>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com
>>>>>>>>>>> The Enterprise PostgreSQL Company
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Dave Page
>>>>>>> Blog: http://pgsnake.blogspot.com
>>>>>>> Twitter: @pgsnake
>>>>>>>
>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com
>>>>>>> The Enterprise PostgreSQL Company
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Dave Page
>>>>> Blog: http://pgsnake.blogspot.com
>>>>> Twitter: @pgsnake
>>>>>
>>>>> EnterpriseDB UK: http://www.enterprisedb.com
>>>>> The Enterprise PostgreSQL Company
>>>>>
>>>> ​
>>>>
>>>
>>>
>>>
>>> --
>>> Dave Page
>>> Blog: http://pgsnake.blogspot.com
>>> Twitter: @pgsnake
>>>
>>> EnterpriseDB UK: http://www.enterprisedb.com
>>> The Enterprise PostgreSQL Company
>>>
>>
>>
>