Re: Unified server/desktop config

Hi,

I noticed that test cases don’t run and I got an error:

(pgAdmin_27)Laptop195:regression surinder$ python runtests.py --pkg
feature_tests
<module '__builtin__' (built-in)>
Traceback (most recent call last):
File "runtests.py", line 45, in <module>
import config
File "/Users/surinder/Documents/Projects/pgadmin4/web/config.py",
line 121, in <module>
if builtins.SERVER_MODE is None:
AttributeError: 'module' object has no attribute'
SERVER_MODE
'

I think this is because we are setting​ ​first​ ​builtins.SERVER_MODE​
in​ ​pgAdmin4.py
but in test cases pgAdmin4.py is not being called.​

On Tue, Aug 8, 2017 at 4:03 PM, Dave Page <dpage(at)pgadmin(dot)org> wrote:

>
>
> On Tue, Aug 8, 2017 at 11:27 AM, Surinder Kumar <
> surinder(dot)kumar(at)enterprisedb(dot)com> wrote:
>
>> On Tue, Aug 8, 2017 at 1:47 PM, Dave Page <dpage(at)pgadmin(dot)org> wrote:
>>
>>
>>>
>>> On Tue, Aug 8, 2017 at 7:18 AM, Surinder Kumar <
>>> surinder(dot)kumar(at)enterprisedb(dot)com> wrote:
>>>
>>>> Hi
>>>>
>>>> On Ubuntu-14.04, I got error Application Server couldn't be contacted:
>>>>
>>>> *Steps performed:*
>>>>
>>>> - I have already installed pgAdmin4-1.4 which come with
>>>> PostgreSQL-9.6 installer.
>>>> then I run root(at)ubuntu:/opt/PostgreSQL/9.6/pgAdmin 4/bin#
>>>> ./pgAdmin4./.
>>>> - Now took latest git pull from HEAD
>>>> - Apply unified_config.diff patch.
>>>> - Then compiled pgAdmin4 in runtime and then run ./pgAdmin.
>>>> - Got error Application Server couldn't be contacted.
>>>>
>>>> But when I ran ./pgAdmin4 for the second time. pgAdmin4 runs without
>>>> any issue.
>>>> I didn’t get any error on the terminal and log file.
>>>> ​ I couldn't find why it gives this error.​
>>>>
>>>
>>> I know Fahar has run into this with existing releases on Ubuntu. If you
>>> enable debugging, can you see any clues? I assume it's going round the
>>> retry loop before aborting?
>>>
>>>> *Another issue related to Alembic:*
>>>>
>>>> If I am running pgAdmin4 already installed on my machine, then I
>>>> upgrade pgAdmin4 using Python wheel:
>>>>
>>>> (test_p27) surinder(at)ubuntu:~/virtualenvs/test_p27$ python
>>>> ~/virtualenvs/test_p27/lib/python2.7/site-packages/pgadmin4/pgAdmin4.py
>>>>
>>>> *I am getting error:*
>>>>
>>>> alembic.util.exc.CommandError: Can't locate revision identified by
>>>> 'd85a62333272'
>>>>
>>>> To fix this, I have to delete existing pgadmin4.db file. I don’t know
>>>> if it is a valid case or should I log an RM if it is?
>>>>
>>> That's an annoying side-effect of the move to Alembic. The previous DB
>>> code would quite happily (and intentionally) run with a newer version of
>>> the database, however, the new code does not. You'll see this issue
>>> whenever you've run a newer version of pgAdmin and then go back to an older
>>> version that uses an older schema version.
>>>
>>> It would be nice to change this so it doesn't complain - but we'd have
>>> to be cautious to only break compatibility on major releases.
>>>
>>>
>>>
>>>> Apart for this, I didn’t see any functionality break. It works!!
>>>> ​
>>>>
>>>> I liked the approach to set SEVER_MODE in runtime using built-ins.
>>>>
>>> Thanks. Do you think it warrants a 2.0 version number, given the
>>> potential for breaking existing installations (I do, but would like other
>>> feedback)? If we do that - and thus allow a parallel installation of 1.x
>>> and 2.x), how would we resolve the Alembic issue you noted such that both
>>> versions could be run against the same DB?
>>>
>> To resolve this issue while upgrading to newer version of pgAdmin4, we
>> can add a parameter version_table: 'alembic_version-1.6 in
>> web/migrations/env.py of upcoming pgAdmin4 source code(reference link
>> <https://issues.asterisk.org/jira/secure/attachment/51610/ASTERISK-24311-set-version-table.diff>
>> )
>> It will create a new version table in pgadmin4.db database and it will
>> instruct the Alembic to run migrations for the version stored in
>> table(alembic_version-1.6) when new pgAdmin4 will be installed.
>>
>> I tried to check if it works or not. But somehow I couldn’t install
>> pgAdmin4-1.5 from wheel.
>>
>> Alternatively I will try to generate new schema using in pgAdmin4 source
>> code(Git HEAD). It will generate a new migration version and then i will
>> run pgAdmin4-1.6 from wheel having different migration number. It will
>> create pgadmin4.db for 1.5.
>> Then
>> I will run pgAdmin4-1.6 from source code, so migration will take place
>> and hopefully
>> ​the ​
>> sa
>> me
>> ​ ​
>> pgadmin4.db will work for newer pgAdmin4 version.
>>
>> Reference link <https://issues.asterisk.org/jira/browse/ASTERISK-24311>
>> where the same issue has been discussed.
>>
>
> Sounds good - please investigate further.
>
I spent some time, understands how Alembic works and I thought it is easy
using the reference link but not, It needs more R&D. I will look how can we
fix it.

>
> Thanks!
>
>
>> Thanks,
>> Surinder
>>
>>
>>>
>>>
>>>> Thanks,
>>>> Surinder
>>>> ​
>>>>
>>>> On Mon, Aug 7, 2017 at 7:01 PM, Dave Page <dpage(at)pgadmin(dot)org> wrote:
>>>>
>>>>>
>>>>>
>>>>> On Mon, Aug 7, 2017 at 1:43 PM, Surinder Kumar <
>>>>> surinder(dot)kumar(at)enterprisedb(dot)com> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> The patch seems to work in Runtime mode, but fails in Server mode
>>>>>> with error:
>>>>>>
>>>>>> (pgAdmin_27)Laptop195:pgadmin4 surinder$ python web/pgAdmin4.py
>>>>>> Traceback (most recent call last):
>>>>>> File "web/pgAdmin4.py", line 55, in <module>
>>>>>> exec(open(file_quote(setupfile), 'r').read())
>>>>>> File "<string>", line 35, in <module>
>>>>>> File "/Users/surinder/Documents/Projects/pgadmin4/web/pgadmin/setup/data_directory.py", line 23, in create_app_data_directory
>>>>>> _create_directory_if_not_exists(os.path.dirname(config.SQLITE_PATH))
>>>>>> File "/Users/surinder/Documents/Projects/pgadmin4/web/pgadmin/setup/data_directory.py", line 15, in _create_directory_if_not_exists
>>>>>> os.mkdir(_path)
>>>>>> OSError: [Errno 13] Permission denied: '/var/lib/pgadmin'
>>>>>> (pgAdmin_27)Laptop195:pgadmin4 surinder$
>>>>>>
>>>>>> This is because the directory /var/lib/ has root only access and I
>>>>>> am running pgAdmin4 with the non-root user.
>>>>>>
>>>>>> Also pgadmin directory is not created.
>>>>>>
>>>>>> (pgAdmin_35)Laptop195:pgadmin4 surinder$ ls /var/lib/pgadmin
>>>>>> ls: /var/lib/pgadmin: No such file or directory
>>>>>>
>>>>>> I got same error with MacOSX and Ubuntu-14.04 machines irrespective
>>>>>> of Python version.
>>>>>>
>>>>>> Meanwhile, I am testing patch with other test cases.
>>>>>>
>>>>> That's fully expected. In the case of Linux, the packages will be
>>>>> responsible for creating those directories with the appropriate ownership.
>>>>> In other cases, the user would.
>>>>>
>>>> ​ok, I got it.​
>>>>
>>>>>
>>>>> There's not really much we can do about that - and it's exactly what
>>>>> would happen if you try to run many other packages yourself when standard
>>>>> *nix paths are used.
>>>>>
>>>>
>>>>>
>>>>>
>>>>>> Thanks,
>>>>>> Surinder
>>>>>> ​
>>>>>>
>>>>>> On Mon, Aug 7, 2017 at 4:33 PM, Surinder Kumar <
>>>>>> surinder(dot)kumar(at)enterprisedb(dot)com> wrote:
>>>>>>
>>>>>>> On Mon, Aug 7, 2017 at 4:11 PM, Ashesh Vashi <
>>>>>>> ashesh(dot)vashi(at)enterprisedb(dot)com> wrote:
>>>>>>>
>>>>>>>> On Mon, Aug 7, 2017 at 3:59 PM, Dave Page <dpage(at)pgadmin(dot)org>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Anyone?
>>>>>>>>>
>>>>>>>> Surinder - please give this one priority.
>>>>>>>>
>>>>>>> Sure
>>>>>>> ​.​
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>>
>>>>>>>> Thanks & Regards,
>>>>>>>>
>>>>>>>> Ashesh Vashi
>>>>>>>> EnterpriseDB INDIA: Enterprise PostgreSQL Company
>>>>>>>> <http://www.enterprisedb.com/>
>>>>>>>>
>>>>>>>>
>>>>>>>> *http://www.linkedin.com/in/asheshvashi*
>>>>>>>> <http://www.linkedin.com/in/asheshvashi>
>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Thu, Jul 20, 2017 at 5:03 PM, Dave Page <dpage(at)pgadmin(dot)org>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> All,
>>>>>>>>>>
>>>>>>>>>> Attached is a patch that aims to allow us to have a standardised
>>>>>>>>>> config that will work out of the box for both web and desktop modes. It
>>>>>>>>>> does this by doing two things:
>>>>>>>>>>
>>>>>>>>>> 1) The runtime sets SERVER_MODE in the Python environment before
>>>>>>>>>> starting the app. If this value is set, then it overrides the default value
>>>>>>>>>> of SERVER_MODE in the config.
>>>>>>>>>>
>>>>>>>>>> 2) The config file then offers default values for the various
>>>>>>>>>> file locations for both server and desktop mode, setting them appropriately
>>>>>>>>>> based on the derived SERVER_MODE value.
>>>>>>>>>>
>>>>>>>>>> The only downsides I can see from this are:
>>>>>>>>>>
>>>>>>>>>> - You cannot run in server mode in the runtime without manually
>>>>>>>>>> reconfiguring SERVER_MODE and likely a bunch of paths in config_local.py
>>>>>>>>>>
>>>>>>>>>> - If you want to override SERVER_MODE, you'll probably also need
>>>>>>>>>> to redefine the various paths in config_local.py.
>>>>>>>>>>
>>>>>>>>>> I don't see either being something 99.9% of users would need
>>>>>>>>>> though.
>>>>>>>>>>
>>>>>>>>>> Can anyone see if the patch breaks anything, or if I missed any
>>>>>>>>>> side effects?
>>>>>>>>>>
>>>>>>>>>> Is it likely to break things during upgrades? I suspect so... so
>>>>>>>>>> maybe this should prompt v2.0?
>>>>>>>>>>
>>>>>>>>>> I'd appreciate multiple reviews of this, as it could break
>>>>>>>>>> things. Note that I haven't yet updated the docs.
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Dave Page
>>>>>>>>>> Blog: http://pgsnake.blogspot.com
>>>>>>>>>> Twitter: @pgsnake
>>>>>>>>>>
>>>>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com
>>>>>>>>>> The Enterprise PostgreSQL Company
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Dave Page
>>>>>>>>> Blog: http://pgsnake.blogspot.com
>>>>>>>>> Twitter: @pgsnake
>>>>>>>>>
>>>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com
>>>>>>>>> The Enterprise PostgreSQL Company
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Dave Page
>>>>> Blog: http://pgsnake.blogspot.com
>>>>> Twitter: @pgsnake
>>>>>
>>>>> EnterpriseDB UK: http://www.enterprisedb.com
>>>>> The Enterprise PostgreSQL Company
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Dave Page
>>> Blog: http://pgsnake.blogspot.com
>>> Twitter: @pgsnake
>>>
>>> EnterpriseDB UK: http://www.enterprisedb.com
>>> The Enterprise PostgreSQL Company
>>>
>> ​
>>
>
>
>
> --
> Dave Page
> Blog: http://pgsnake.blogspot.com
> Twitter: @pgsnake
>
> EnterpriseDB UK: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>