Re: Problem related to volume creation to pgadmin 4 Docker image

Hi Rodrigo,

Did you get a chance to verify the snapshot and steps are already mentioned
by Akshay?

On Mon, Apr 25, 2022 at 2:53 PM Akshay Joshi <akshay(dot)joshi(at)enterprisedb(dot)com>
wrote:

> Hi Rodrigo
>
> We need your small help to confirm the fix
> https://redmine.postgresql.org/issues/6958. We have fixed the issue but
> can you please test it on the snapshot build?
> You need to use "*image: dpage/pgadmin4:snapshot*" in your
> docker-compose.yml file.
>
> On Mon, Oct 25, 2021 at 3:33 PM Dave Page <dpage(at)pgadmin(dot)org> wrote:
>
>> Hi
>>
>> Issue created: https://redmine.postgresql.org/issues/6958
>>
>> On Fri, Oct 22, 2021 at 4:24 PM Rodrigo Mariano <rodmariano13(at)gmail(dot)com>
>> wrote:
>>
>>> Hi Dave,
>>>
>>> I understand the situation and I believe both options, that you
>>> suggested, could improve the container.
>>>
>>> If you could leave this issue marked on somewhere to be analyzed in the
>>> future, I thank you so much.
>>>
>>> Thank you for your help.
>>>
>>> Best regards,
>>> Rodrigo
>>> On 22/10/2021 11:31, Dave Page wrote:
>>>
>>> Hi
>>>
>>> On Fri, Oct 22, 2021 at 3:12 PM Rodrigo Mariano <rodmariano13(at)gmail(dot)com>
>>> wrote:
>>>
>>>> Hi Dave,
>>>>
>>>> I tested the ACL command, as you suggested, and it worked when docker
>>>> container was turned off, but when I lauched pgadmin, it reset the folder
>>>> permissions again.
>>>>
>>> That's very odd - pgAdmin only resets the permission bits. It doesn't
>>> have any code to touch the ACL.
>>>
>>>>
>>>> Could you consider, in future versions, to give access to host user to
>>>> */var/lib/pgadmin/storage* folder?
>>>> For example, other files and folders (e.g. sessions and pgadmin4.db)
>>>> could be restricted, but storage, as a folder to user files, could have
>>>> read and execute permissions in order to host user be able to access it.
>>>>
>>> That may be safe in your environment, but perhaps not in others (and we
>>> always aim for secure-by-default). Perhaps a suitable compromise would be
>>> to either have a config option to avoid the chmod at startup, or to only
>>> perform it when the directory is first created (so that you can change it
>>> after first launch, and not have it reset in the future).
>>>
>>>>
>>>> Thank you for your help.
>>>>
>>>> Best regards,
>>>> Rodrigo
>>>> On 22/10/2021 06:31, Dave Page wrote:
>>>>
>>>> Hi
>>>>
>>>> On Thu, Oct 21, 2021 at 7:51 PM Rodrigo Mariano <rodmariano13(at)gmail(dot)com>
>>>> wrote:
>>>>
>>>>> Hi Dave,
>>>>>
>>>>> Which OS do you use? I'm using Ubuntu 18.
>>>>>
>>>> macOS, primarily.
>>>>
>>>>>
>>>>> Nautilus is the file manager to Ubuntu.
>>>>>
>>>> Ah, OK.
>>>>
>>>>>
>>>>> I updated my image to dpage/pgadmin4:6.0 in order to avoid old
>>>>> versions. I add a new volume and I executed the chown command (i.e. sudo
>>>>> chown -R 5050:5050 <host_directory>).
>>>>>
>>>>> I tried to add my user to 5050 group, but it did not work, because
>>>>> when pgadmin4 Docker container is executed, it allows just 5050 user to
>>>>> edit the folder and not other ones from the same group (i.e.
>>>>> *drwx------*).
>>>>>
>>>>> *drwx------* is the default permission that pgadmin4 Docker container
>>>>> gives to volume it creates, in other words, just 5050 user can edit the
>>>>> volume data, not other ones, even if that user belongs to 5050 group.
>>>>>
>>>> OK, now I understand what you mean. Yes, when pgAdmin launches, it'll
>>>> check the directories it needs, and always tries to fix the permissions to
>>>> ensure they're secure (i.e. 0700 permissions).
>>>>
>>>> You might be able to use the extended ACL to work around that, e.g.
>>>>
>>>> setfacl -Rm u:rodrigo:rwX,d:u:rodrigo:rwX <host_directory>
>>>>
>>>> I believe that will recursively give you permissions on the directory
>>>> on the host (assuming your username is rodrigo), and set it up so
>>>> permissions are inherited. You may need to ensure your host filesystem is
>>>> mounted with the 'acl' option.
>>>>
>>>>>
>>>>> Thank you.
>>>>>
>>>>> Best regards,
>>>>> Rodrigo
>>>>> On 21/10/2021 10:20, Dave Page wrote:
>>>>>
>>>>>
>>>>>
>>>>> On Thu, Oct 21, 2021 at 1:33 PM Rodrigo Mariano <
>>>>> rodmariano13(at)gmail(dot)com> wrote:
>>>>>
>>>>>> Hi Dave,
>>>>>>
>>>>>> *> I've never needed to do that with plain Docker or Kubernetes. I've
>>>>>> never used Docker Compose though. *
>>>>>>
>>>>>> Have you ever tried to create a volume to */var/lib/pgadmin/storage*
>>>>>> folder using newer image versions and you were able to access it via host
>>>>>> in the nautilus? Using plain Docker.
>>>>>>
>>>>> I have no idea what "the nautilus" is, but yes, I've mapped
>>>>> /var/lib/pgadmin to the host many times (including 30 seconds ago with
>>>>> 6.1), and it works fine. As long as appropriate permissions are set on the
>>>>> directory on the host, I can access it from there as well.
>>>>>
>>>>>>
>>>>>> If you have, how could I do that?
>>>>>>
>>>>> As you suggested, you could add yourself to the 5050 group, and ensure
>>>>> the directory on the host is group readable.
>>>>>
>>>>>>
>>>>>> I did not have this kind of issue with older versions of pgadmin4
>>>>>> Docker image (e.g. *dpage/pgadmin4:4.15*), this issue has started
>>>>>> with recent images that I need to change folder permission to 5050:5050
>>>>>> (e.g. *dpage/pgadmin4:5.4*).
>>>>>>
>>>>> 4.15 is very old. We've long since had additional checks in pgAdmin to
>>>>> ensure that we can successfully write to the storage directory, and to stop
>>>>> running the processes in the container as root that was a) quite dangerous
>>>>> and b) could allow it to override permissions on the host. In particular,
>>>>> you're probably hitting the issue mentioned in the callout box at the top
>>>>> of https://www.pgadmin.org/docs/pgadmin4/6.1/release_notes_4_16.html
>>>>>
>>>>>
>>>>>>
>>>>>> Thank you.
>>>>>>
>>>>>> Best regards,
>>>>>> Rodrigo
>>>>>>
>>>>>> On 21/10/2021 08:36, Dave Page wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Thu, Oct 21, 2021 at 12:27 PM Rodrigo Mariano <
>>>>>> rodmariano13(at)gmail(dot)com> wrote:
>>>>>>
>>>>>>> Hi Aditya,
>>>>>>>
>>>>>>> According to the documentation, I need to change user and group of
>>>>>>> my host folder to *5050:5050* through *chown*.
>>>>>>>
>>>>>>> If my default user and group is *rodrigo:rodrigo*, how could my
>>>>>>> default user access a folder that belongs to another one (i.e.
>>>>>>> *5050:5050*)?
>>>>>>>
>>>>>> The pgAdmin processes in the container run under uid 5050, gid 5050.
>>>>>>
>>>>>>>
>>>>>>> As far as I know, I cannot access a folder that belongs to other
>>>>>>> user normally.
>>>>>>>
>>>>>>> Maybe should I add my default user (i.e. *rodrigo*) to pgadmin
>>>>>>> group (i.e. *5050*)?
>>>>>>>
>>>>>> I've never needed to do that with plain Docker or Kubernetes. I've
>>>>>> never used Docker Compose though.
>>>>>>
>>>>>>> If I should, I believe this information could be written on the
>>>>>>> documentation.
>>>>>>>
>>>>>>> Thank you.
>>>>>>>
>>>>>>> Best regards,
>>>>>>> Rodrigo
>>>>>>> On 21/10/2021 02:06, Aditya Toshniwal wrote:
>>>>>>>
>>>>>>> Hi Rodrigo,
>>>>>>>
>>>>>>> pgAdmin just needs a readable and writable directory. pgAdmin cannot
>>>>>>> change any permission on its own. It might be some other ownership issue on
>>>>>>> your system then.
>>>>>>>
>>>>>>> On Wed, Oct 20, 2021 at 11:29 PM Rodrigo Mariano <
>>>>>>> rodmariano13(at)gmail(dot)com> wrote:
>>>>>>>
>>>>>>>> Hi Aditya,
>>>>>>>>
>>>>>>>> I did both.
>>>>>>>>
>>>>>>>> First, I changed the folder permissions to 5050:5050 and the Docker
>>>>>>>> container worked, but I was not able to get into the folder; the folder is
>>>>>>>> locked and I cannot access its subfolders, even through terminal. For
>>>>>>>> example:
>>>>>>>>
>>>>>>>> After that, I tried using default permissions, however that error
>>>>>>>> message appeared.
>>>>>>>>
>>>>>>>> Thank you.
>>>>>>>>
>>>>>>>> Best regards,
>>>>>>>> Rodrigo
>>>>>>>> On 20/10/2021 10:08, Aditya Toshniwal wrote:
>>>>>>>>
>>>>>>>> Hi Rodrigo,
>>>>>>>>
>>>>>>>> Did you run sudo chown -R 5050:5050 ./volumes/pgadmin4 and sudo
>>>>>>>> chown -R 5050:5050 ./volumes/pgadmin4_storage As per -
>>>>>>>> https://www.pgadmin.org/docs/pgadmin4/6.0/container_deployment.html#mapped-files-and-directories
>>>>>>>> ?
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Oct 20, 2021 at 6:14 PM Rodrigo Mariano <
>>>>>>>> rodmariano13(at)gmail(dot)com> wrote:
>>>>>>>>
>>>>>>>>> Hi Aditya,
>>>>>>>>>
>>>>>>>>> I tried to create the volume to sub directory as well (i.e. */var/lib/pgadmin/storage/postgres_localhost.com
>>>>>>>>> <http://postgres_localhost.com>*), but the same error message
>>>>>>>>> appears.
>>>>>>>>>
>>>>>>>>> I send below the traceback.
>>>>>>>>>
>>>>>>>>> Thank you for your help.
>>>>>>>>>
>>>>>>>>> Best regards,
>>>>>>>>> Rodrigo
>>>>>>>>>
>>>>>>>>> -
>>>>>>>>>
>>>>>>>>> Traceback (most recent call last):
>>>>>>>>> File "/venv/lib/python3.8/site-packages/gunicorn/arbiter.py",
>>>>>>>>> line 589, in spawn_worker
>>>>>>>>> worker.init_process()
>>>>>>>>> File
>>>>>>>>> "/venv/lib/python3.8/site-packages/gunicorn/workers/gthread.py", line 92,
>>>>>>>>> in init_process
>>>>>>>>> super().init_process()
>>>>>>>>> File
>>>>>>>>> "/venv/lib/python3.8/site-packages/gunicorn/workers/base.py", line 134, in
>>>>>>>>> init_process
>>>>>>>>> self.load_wsgi()
>>>>>>>>> File
>>>>>>>>> "/venv/lib/python3.8/site-packages/gunicorn/workers/base.py", line 146, in
>>>>>>>>> load_wsgi
>>>>>>>>> self.wsgi = self.app.wsgi()
>>>>>>>>> File "/venv/lib/python3.8/site-packages/gunicorn/app/base.py",
>>>>>>>>> line 67, in wsgi
>>>>>>>>> self.callable = self.load()
>>>>>>>>> File
>>>>>>>>> "/venv/lib/python3.8/site-packages/gunicorn/app/wsgiapp.py", line 58, in
>>>>>>>>> load
>>>>>>>>> return self.load_wsgiapp()
>>>>>>>>> File
>>>>>>>>> "/venv/lib/python3.8/site-packages/gunicorn/app/wsgiapp.py", line 48, in
>>>>>>>>> load_wsgiapp
>>>>>>>>> return util.import_app(self.app_uri)
>>>>>>>>> File "/venv/lib/python3.8/site-packages/gunicorn/util.py", line
>>>>>>>>> 359, in import_app
>>>>>>>>> mod = importlib.import_module(module)
>>>>>>>>> File "/usr/lib/python3.8/importlib/__init__.py", line 127, in
>>>>>>>>> import_module
>>>>>>>>> return _bootstrap._gcd_import(name[level:], package, level)
>>>>>>>>> File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
>>>>>>>>> File "<frozen importlib._bootstrap>", line 991, in _find_and_load
>>>>>>>>> File "<frozen importlib._bootstrap>", line 975, in
>>>>>>>>> _find_and_load_unlocked
>>>>>>>>> File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
>>>>>>>>> File "<frozen importlib._bootstrap_external>", line 848, in
>>>>>>>>> exec_module
>>>>>>>>> File "<frozen importlib._bootstrap>", line 219, in
>>>>>>>>> _call_with_frames_removed
>>>>>>>>> File "/pgadmin4/run_pgadmin.py", line 4, in <module>
>>>>>>>>> from pgAdmin4 import app
>>>>>>>>> File "/pgadmin4/pgAdmin4.py", line 98, in <module>
>>>>>>>>> app = create_app()
>>>>>>>>> File "/pgadmin4/pgadmin/__init__.py", line 441, in create_app
>>>>>>>>> paths.init_app(app)
>>>>>>>>> File "/pgadmin4/pgadmin/utils/paths.py", line 103, in init_app
>>>>>>>>> raise InternalServerError(
>>>>>>>>> werkzeug.exceptions.InternalServerError: 500 Internal Server
>>>>>>>>> Error: The user does not have permission to read and write to the specified
>>>>>>>>> storage directory.
>>>>>>>>> On 20/10/2021 09:08, Aditya Toshniwal wrote:
>>>>>>>>>
>>>>>>>>> Hi Rodrigo,
>>>>>>>>>
>>>>>>>>> /var/lib/pgadmin/storage is the base directory. A sub directory
>>>>>>>>> for each user will be created for storing user files.
>>>>>>>>>
>>>>>>>>> On Wed, Oct 20, 2021 at 5:10 PM Rodrigo Mariano <
>>>>>>>>> rodmariano13(at)gmail(dot)com> wrote:
>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> I'm having a trouble related to pgadmin 4 Docker image
>>>>>>>>>> <https://hub.docker.com/r/dpage/pgadmin4>
>>>>>>>>>> <https://hub.docker.com/r/dpage/pgadmin4>.
>>>>>>>>>>
>>>>>>>>>> I would like to create a volume to */var/lib/pgadmin/storage*
>>>>>>>>>> folder, in order to access backup files created by pgadmin 4 interface,
>>>>>>>>>> however error messages about permission denied are raised, for example:
>>>>>>>>>>
>>>>>>>>>> werkzeug.exceptions.InternalServerError: 500 Internal Server
>>>>>>>>>> Error: The user does not have permission to read and write to the specified
>>>>>>>>>> storage directory.
>>>>>>>>>>
>>>>>>>>>> Is there a way to create this volume?
>>>>>>>>>>
>>>>>>>>>> I had to use a command to change user and group of my volume to
>>>>>>>>>> 5050:5050 (i.e. *sudo chown -R 5050:5050 pgadmin4*), but now I'm
>>>>>>>>>> not able to get into the folder anymore, even when I try creating a volume
>>>>>>>>>> to */var/lib/pgadmin/storage* folder directly.
>>>>>>>>>>
>>>>>>>>>> I send below my Docker compose file with default values.
>>>>>>>>>>
>>>>>>>>>> Thank you in advance.
>>>>>>>>>>
>>>>>>>>>> Best regards,
>>>>>>>>>> Rodrigo
>>>>>>>>>>
>>>>>>>>>> -
>>>>>>>>>>
>>>>>>>>>> *docker-compose.yml*
>>>>>>>>>>
>>>>>>>>>> version: '3'
>>>>>>>>>>
>>>>>>>>>> services:
>>>>>>>>>> cdsr_postgis:
>>>>>>>>>> container_name: cdsr_postgis
>>>>>>>>>> image: kartoza/postgis:11.0-2.5
>>>>>>>>>> restart: on-failure
>>>>>>>>>> environment:
>>>>>>>>>> - POSTGRES_USER=postgres
>>>>>>>>>> - POSTGRES_PASS=postgres
>>>>>>>>>> - ALLOW_IP_RANGE=0.0.0.0/0
>>>>>>>>>> -
>>>>>>>>>> POSTGRES_MULTIPLE_EXTENSIONS=postgis,hstore,postgis_topology,pgrouting
>>>>>>>>>> volumes:
>>>>>>>>>> - ./volumes/postgresql:/var/lib/postgresql
>>>>>>>>>> networks:
>>>>>>>>>> - cdsr
>>>>>>>>>> ports:
>>>>>>>>>> - 6000:5432
>>>>>>>>>>
>>>>>>>>>> cdsr_pgadmin4:
>>>>>>>>>> container_name: cdsr_pgadmin4
>>>>>>>>>> image: dpage/pgadmin4:5.4
>>>>>>>>>> restart: on-failure
>>>>>>>>>> environment:
>>>>>>>>>> - PGADMIN_DEFAULT_EMAIL=postgres(at)localhost(dot)com
>>>>>>>>>> - PGADMIN_DEFAULT_PASSWORD=postgres
>>>>>>>>>> volumes:
>>>>>>>>>> # to fix permission bugs:
>>>>>>>>>> # sudo chown -R 5050:5050 pgadmin4
>>>>>>>>>> - ./volumes/pgadmin4:/var/lib/pgadmin
>>>>>>>>>> - ./volumes/pgadmin4_storage:/var/lib/pgadmin/storage
>>>>>>>>>> networks:
>>>>>>>>>> - cdsr
>>>>>>>>>> depends_on:
>>>>>>>>>> - cdsr_postgis
>>>>>>>>>> ports:
>>>>>>>>>> - 6001:80
>>>>>>>>>>
>>>>>>>>>> networks:
>>>>>>>>>> cdsr:
>>>>>>>>>> driver: bridge
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Thanks,
>>>>>>>>> Aditya Toshniwal
>>>>>>>>> pgAdmin Hacker | Software Architect | *edbpostgres.com*
>>>>>>>>> <http://edbpostgres.com>
>>>>>>>>> "Don't Complain about Heat, Plant a TREE"
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Thanks,
>>>>>>>> Aditya Toshniwal
>>>>>>>> pgAdmin Hacker | Software Architect | *edbpostgres.com*
>>>>>>>> <http://edbpostgres.com>
>>>>>>>> "Don't Complain about Heat, Plant a TREE"
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Thanks,
>>>>>>> Aditya Toshniwal
>>>>>>> pgAdmin Hacker | Software Architect | *edbpostgres.com*
>>>>>>> <http://edbpostgres.com>
>>>>>>> "Don't Complain about Heat, Plant a TREE"
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> --
>>>>>> Dave Page
>>>>>> Blog: https://pgsnake.blogspot.com
>>>>>> Twitter: @pgsnake
>>>>>>
>>>>>> EDB: https://www.enterprisedb.com
>>>>>>
>>>>>>
>>>>>
>>>>> --
>>>>> Dave Page
>>>>> Blog: https://pgsnake.blogspot.com
>>>>> Twitter: @pgsnake
>>>>>
>>>>> EDB: https://www.enterprisedb.com
>>>>>
>>>>>
>>>>
>>>> --
>>>> Dave Page
>>>> Blog: https://pgsnake.blogspot.com
>>>> Twitter: @pgsnake
>>>>
>>>> EDB: https://www.enterprisedb.com
>>>>
>>>>
>>>
>>> --
>>> Dave Page
>>> Blog: https://pgsnake.blogspot.com
>>> Twitter: @pgsnake
>>>
>>> EDB: https://www.enterprisedb.com
>>>
>>>
>>
>> --
>> Dave Page
>> Blog: https://pgsnake.blogspot.com
>> Twitter: @pgsnake
>>
>> EDB: https://www.enterprisedb.com
>>
>>
>
> --
> *Thanks & Regards*
> *Akshay Joshi*
> *pgAdmin Hacker | Principal Software Architect*
> *EDB Postgres <http://edbpostgres.com>*
>
> *Mobile: +91 976-788-8246*
>

--
Fahar Abbas
pgAdmin4 team
EnterpriseDB Corporation
Mobile: +92-333-5409707
Skype ID: *live:fahar.abbas*
Website: www.enterprisedb.com