Re: Unified server/desktop config

Hi, Dave
Our previous config_local.py is:
===================================================================
from config import *
DEBUG = True
# Enable the test module
MODULE_BLACKLIST.remove('test')
# Log
CONSOLE_LOG_LEVEL = DEBUG
FILE_LOG_LEVEL = DEBUG
DEFAULT_SERVER = '127.0.0.1'
UPGRADE_CHECK_ENABLED = True
SERVER_MODE = False
# Use a different config DB for each server mode.
if SERVER_MODE == False:
SQLITE_PATH = os.path.join(
DATA_DIR,
'pgadmin4-desktop.db'
)
else:
SQLITE_PATH = os.path.join(
DATA_DIR,
) ==============================================================
Now we need config
DATA_DIR, LOG_FILE and all the configurations related to DATA_DIR

Thanks,
Violet & Bing

On Thu, Aug 17, 2017 at 4:23 PM, Dave Page <dpage(at)pgadmin(dot)org> wrote:

> Thanks. Out of interest, what changes were required?
>
> All, this reinforces to me that this should be a v2.0 change. Other
> thoughts?
>
> On Mon, Aug 14, 2017 at 4:59 AM, Bing Xu <bxu(at)pivotal(dot)io> wrote:
>
>> Hi hackers,
>>
>> We've reviewed this patch and it need us changing our own config_local.py
>> file. It's ok and we'll do the change in our configuration.
>>
>> Thanks,
>> Bing & Violet
>>
>> On Wed, Aug 9, 2017 at 8:49 PM, Surinder Kumar <
>> surinder(dot)kumar(at)enterprisedb(dot)com> wrote:
>>
>>> Hi
>>>
>>> This patch includes the fix where this patch breaks when feature tests
>>> are run.
>>>
>>> Set builtins.SERVER_MODE if it is present in globals()['SERVER_MODE']
>>> else set to None.
>>>
>>> Please find updated patch.
>>> Thanks,
>>> Surinder
>>> ​
>>>
>>> On Wed, Aug 9, 2017 at 11:57 AM, Surinder Kumar <
>>> surinder(dot)kumar(at)enterprisedb(dot)com> wrote:
>>>
>>>> Sure, I will update.
>>>>
>>>> On Wed, Aug 9, 2017 at 11:17 AM, Dave Page <dpage(at)pgadmin(dot)org> wrote:
>>>>
>>>>> Please update the patch :-)
>>>>>
>>>>> --
>>>>> Dave Page
>>>>> Blog: http://pgsnake.blogspot.com
>>>>> Twitter: @pgsnake
>>>>>
>>>>> EnterpriseDB UK:http://www.enterprisedb.com
>>>>> The Enterprise PostgreSQL Company
>>>>>
>>>>> On 9 Aug 2017, at 05:53, Surinder Kumar <surinder(dot)kumar(at)enterprisedb(dot)c
>>>>> om> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> I noticed that test cases don’t run and I got an error:
>>>>>
>>>>> (pgAdmin_27)Laptop195:regression surinder$ python runtests.py --pkg feature_tests
>>>>> <module '__builtin__' (built-in)>
>>>>> Traceback (most recent call last):
>>>>> File "runtests.py", line 45, in <module>
>>>>> import config
>>>>> File "/Users/surinder/Documents/Projects/pgadmin4/web/config.py", line 121, in <module>
>>>>> if builtins.SERVER_MODE is None:
>>>>> AttributeError: 'module' object has no attribute'
>>>>> SERVER_MODE
>>>>> '
>>>>>
>>>>> I think this is because we are setting​ ​first​ ​builtins.SERVER_MODE​ in​ ​pgAdmin4.py
>>>>> but in test cases pgAdmin4.py is not being called.​
>>>>>
>>>>>
>>>>> On Tue, Aug 8, 2017 at 4:03 PM, Dave Page <dpage(at)pgadmin(dot)org> wrote:
>>>>>
>>>>>>
>>>>>>
>>>>>> On Tue, Aug 8, 2017 at 11:27 AM, Surinder Kumar <
>>>>>> surinder(dot)kumar(at)enterprisedb(dot)com> wrote:
>>>>>>
>>>>>>> On Tue, Aug 8, 2017 at 1:47 PM, Dave Page <dpage(at)pgadmin(dot)org> wrote:
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Aug 8, 2017 at 7:18 AM, Surinder Kumar <
>>>>>>>> surinder(dot)kumar(at)enterprisedb(dot)com> wrote:
>>>>>>>>
>>>>>>>>> Hi
>>>>>>>>>
>>>>>>>>> On Ubuntu-14.04, I got error Application Server couldn't be
>>>>>>>>> contacted:
>>>>>>>>>
>>>>>>>>> *Steps performed:*
>>>>>>>>>
>>>>>>>>> - I have already installed pgAdmin4-1.4 which come with
>>>>>>>>> PostgreSQL-9.6 installer.
>>>>>>>>> then I run root(at)ubuntu:/opt/PostgreSQL/9.6/pgAdmin 4/bin#
>>>>>>>>> ./pgAdmin4./.
>>>>>>>>> - Now took latest git pull from HEAD
>>>>>>>>> - Apply unified_config.diff patch.
>>>>>>>>> - Then compiled pgAdmin4 in runtime and then run ./pgAdmin.
>>>>>>>>> - Got error Application Server couldn't be contacted.
>>>>>>>>>
>>>>>>>>> But when I ran ./pgAdmin4 for the second time. pgAdmin4 runs
>>>>>>>>> without any issue.
>>>>>>>>> I didn’t get any error on the terminal and log file.
>>>>>>>>> ​ I couldn't find why it gives this error.​
>>>>>>>>>
>>>>>>>>
>>>>>>>> I know Fahar has run into this with existing releases on Ubuntu. If
>>>>>>>> you enable debugging, can you see any clues? I assume it's going round the
>>>>>>>> retry loop before aborting?
>>>>>>>>
>>>>>>>>> *Another issue related to Alembic:*
>>>>>>>>>
>>>>>>>>> If I am running pgAdmin4 already installed on my machine, then I
>>>>>>>>> upgrade pgAdmin4 using Python wheel:
>>>>>>>>>
>>>>>>>>> (test_p27) surinder(at)ubuntu:~/virtualenvs/test_p27$ python
>>>>>>>>> ~/virtualenvs/test_p27/lib/python2.7/site-packages/pgadmin4/
>>>>>>>>> pgAdmin4.py
>>>>>>>>>
>>>>>>>>> *I am getting error:*
>>>>>>>>>
>>>>>>>>> alembic.util.exc.CommandError: Can't locate revision identified by
>>>>>>>>> 'd85a62333272'
>>>>>>>>>
>>>>>>>>> To fix this, I have to delete existing pgadmin4.db file. I don’t
>>>>>>>>> know if it is a valid case or should I log an RM if it is?
>>>>>>>>>
>>>>>>>> That's an annoying side-effect of the move to Alembic. The previous
>>>>>>>> DB code would quite happily (and intentionally) run with a newer version of
>>>>>>>> the database, however, the new code does not. You'll see this issue
>>>>>>>> whenever you've run a newer version of pgAdmin and then go back to an older
>>>>>>>> version that uses an older schema version.
>>>>>>>>
>>>>>>>> It would be nice to change this so it doesn't complain - but we'd
>>>>>>>> have to be cautious to only break compatibility on major releases.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> Apart for this, I didn’t see any functionality break. It works!!
>>>>>>>>> ​
>>>>>>>>>
>>>>>>>>> I liked the approach to set SEVER_MODE in runtime using built-ins.
>>>>>>>>>
>>>>>>>> Thanks. Do you think it warrants a 2.0 version number, given the
>>>>>>>> potential for breaking existing installations (I do, but would like other
>>>>>>>> feedback)? If we do that - and thus allow a parallel installation of 1.x
>>>>>>>> and 2.x), how would we resolve the Alembic issue you noted such that both
>>>>>>>> versions could be run against the same DB?
>>>>>>>>
>>>>>>> To resolve this issue while upgrading to newer version of pgAdmin4,
>>>>>>> we can add a parameter version_table: 'alembic_version-1.6 in
>>>>>>> web/migrations/env.py of upcoming pgAdmin4 source code(reference
>>>>>>> link
>>>>>>> <https://issues.asterisk.org/jira/secure/attachment/51610/ASTERISK-24311-set-version-table.diff>
>>>>>>> )
>>>>>>> It will create a new version table in pgadmin4.db database and it
>>>>>>> will instruct the Alembic to run migrations for the version stored in
>>>>>>> table(alembic_version-1.6) when new pgAdmin4 will be installed.
>>>>>>>
>>>>>>> I tried to check if it works or not. But somehow I couldn’t install
>>>>>>> pgAdmin4-1.5 from wheel.
>>>>>>>
>>>>>>> Alternatively I will try to generate new schema using in pgAdmin4
>>>>>>> source code(Git HEAD). It will generate a new migration version and then i
>>>>>>> will run pgAdmin4-1.6 from wheel having different migration number. It will
>>>>>>> create pgadmin4.db for 1.5.
>>>>>>> Then
>>>>>>> I will run pgAdmin4-1.6 from source code, so migration will take
>>>>>>> place and hopefully
>>>>>>> ​the ​
>>>>>>> sa
>>>>>>> me
>>>>>>> ​ ​
>>>>>>> pgadmin4.db will work for newer pgAdmin4 version.
>>>>>>>
>>>>>>> Reference link
>>>>>>> <https://issues.asterisk.org/jira/browse/ASTERISK-24311> where the
>>>>>>> same issue has been discussed.
>>>>>>>
>>>>>>
>>>>>> Sounds good - please investigate further.
>>>>>>
>>>>> I spent some time, understands how Alembic works and I thought it is
>>>>> easy using the reference link but not, It needs more R&D. I will look how
>>>>> can we fix it.
>>>>>
>>>>>>
>>>>>> Thanks!
>>>>>>
>>>>>>
>>>>>>> Thanks,
>>>>>>> Surinder
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Surinder
>>>>>>>>> ​
>>>>>>>>>
>>>>>>>>> On Mon, Aug 7, 2017 at 7:01 PM, Dave Page <dpage(at)pgadmin(dot)org>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Mon, Aug 7, 2017 at 1:43 PM, Surinder Kumar <
>>>>>>>>>> surinder(dot)kumar(at)enterprisedb(dot)com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> The patch seems to work in Runtime mode, but fails in Server
>>>>>>>>>>> mode with error:
>>>>>>>>>>>
>>>>>>>>>>> (pgAdmin_27)Laptop195:pgadmin4 surinder$ python web/pgAdmin4.py
>>>>>>>>>>> Traceback (most recent call last):
>>>>>>>>>>> File "web/pgAdmin4.py", line 55, in <module>
>>>>>>>>>>> exec(open(file_quote(setupfile), 'r').read())
>>>>>>>>>>> File "<string>", line 35, in <module>
>>>>>>>>>>> File "/Users/surinder/Documents/Projects/pgadmin4/web/pgadmin/setup/data_directory.py", line 23, in create_app_data_directory
>>>>>>>>>>> _create_directory_if_not_exists(os.path.dirname(config.SQLITE_PATH))
>>>>>>>>>>> File "/Users/surinder/Documents/Projects/pgadmin4/web/pgadmin/setup/data_directory.py", line 15, in _create_directory_if_not_exists
>>>>>>>>>>> os.mkdir(_path)
>>>>>>>>>>> OSError: [Errno 13] Permission denied: '/var/lib/pgadmin'
>>>>>>>>>>> (pgAdmin_27)Laptop195:pgadmin4 surinder$
>>>>>>>>>>>
>>>>>>>>>>> This is because the directory /var/lib/ has root only access
>>>>>>>>>>> and I am running pgAdmin4 with the non-root user.
>>>>>>>>>>>
>>>>>>>>>>> Also pgadmin directory is not created.
>>>>>>>>>>>
>>>>>>>>>>> (pgAdmin_35)Laptop195:pgadmin4 surinder$ ls /var/lib/pgadmin
>>>>>>>>>>> ls: /var/lib/pgadmin: No such file or directory
>>>>>>>>>>>
>>>>>>>>>>> I got same error with MacOSX and Ubuntu-14.04 machines
>>>>>>>>>>> irrespective of Python version.
>>>>>>>>>>>
>>>>>>>>>>> Meanwhile, I am testing patch with other test cases.
>>>>>>>>>>>
>>>>>>>>>> That's fully expected. In the case of Linux, the packages will be
>>>>>>>>>> responsible for creating those directories with the appropriate ownership.
>>>>>>>>>> In other cases, the user would.
>>>>>>>>>>
>>>>>>>>> ​ok, I got it.​
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> There's not really much we can do about that - and it's exactly
>>>>>>>>>> what would happen if you try to run many other packages yourself when
>>>>>>>>>> standard *nix paths are used.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Surinder
>>>>>>>>>>> ​
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Aug 7, 2017 at 4:33 PM, Surinder Kumar <
>>>>>>>>>>> surinder(dot)kumar(at)enterprisedb(dot)com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Aug 7, 2017 at 4:11 PM, Ashesh Vashi <
>>>>>>>>>>>> ashesh(dot)vashi(at)enterprisedb(dot)com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> On Mon, Aug 7, 2017 at 3:59 PM, Dave Page <dpage(at)pgadmin(dot)org>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Anyone?
>>>>>>>>>>>>>>
>>>>>>>>>>>>> Surinder - please give this one priority.
>>>>>>>>>>>>>
>>>>>>>>>>>> Sure
>>>>>>>>>>>> ​.​
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks & Regards,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Ashesh Vashi
>>>>>>>>>>>>> EnterpriseDB INDIA: Enterprise PostgreSQL Company
>>>>>>>>>>>>> <http://www.enterprisedb.com/>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> *http://www.linkedin.com/in/asheshvashi*
>>>>>>>>>>>>> <http://www.linkedin.com/in/asheshvashi>
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Thu, Jul 20, 2017 at 5:03 PM, Dave Page <dpage(at)pgadmin(dot)org
>>>>>>>>>>>>>> > wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> All,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Attached is a patch that aims to allow us to have a
>>>>>>>>>>>>>>> standardised config that will work out of the box for both web and desktop
>>>>>>>>>>>>>>> modes. It does this by doing two things:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 1) The runtime sets SERVER_MODE in the Python environment
>>>>>>>>>>>>>>> before starting the app. If this value is set, then it overrides the
>>>>>>>>>>>>>>> default value of SERVER_MODE in the config.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 2) The config file then offers default values for the
>>>>>>>>>>>>>>> various file locations for both server and desktop mode, setting them
>>>>>>>>>>>>>>> appropriately based on the derived SERVER_MODE value.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> The only downsides I can see from this are:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> - You cannot run in server mode in the runtime without
>>>>>>>>>>>>>>> manually reconfiguring SERVER_MODE and likely a bunch of paths in
>>>>>>>>>>>>>>> config_local.py
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> - If you want to override SERVER_MODE, you'll probably also
>>>>>>>>>>>>>>> need to redefine the various paths in config_local.py.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I don't see either being something 99.9% of users would need
>>>>>>>>>>>>>>> though.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Can anyone see if the patch breaks anything, or if I missed
>>>>>>>>>>>>>>> any side effects?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Is it likely to break things during upgrades? I suspect
>>>>>>>>>>>>>>> so... so maybe this should prompt v2.0?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I'd appreciate multiple reviews of this, as it could break
>>>>>>>>>>>>>>> things. Note that I haven't yet updated the docs.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> Dave Page
>>>>>>>>>>>>>>> Blog: http://pgsnake.blogspot.com
>>>>>>>>>>>>>>> Twitter: @pgsnake
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com
>>>>>>>>>>>>>>> The Enterprise PostgreSQL Company
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> Dave Page
>>>>>>>>>>>>>> Blog: http://pgsnake.blogspot.com
>>>>>>>>>>>>>> Twitter: @pgsnake
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com
>>>>>>>>>>>>>> The Enterprise PostgreSQL Company
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Dave Page
>>>>>>>>>> Blog: http://pgsnake.blogspot.com
>>>>>>>>>> Twitter: @pgsnake
>>>>>>>>>>
>>>>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com
>>>>>>>>>> The Enterprise PostgreSQL Company
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Dave Page
>>>>>>>> Blog: http://pgsnake.blogspot.com
>>>>>>>> Twitter: @pgsnake
>>>>>>>>
>>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com
>>>>>>>> The Enterprise PostgreSQL Company
>>>>>>>>
>>>>>>> ​
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Dave Page
>>>>>> Blog: http://pgsnake.blogspot.com
>>>>>> Twitter: @pgsnake
>>>>>>
>>>>>> EnterpriseDB UK: http://www.enterprisedb.com
>>>>>> The Enterprise PostgreSQL Company
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>
>
> --
> Dave Page
> Blog: http://pgsnake.blogspot.com
> Twitter: @pgsnake
>
> EnterpriseDB UK: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>