Re: [pgAdmin4][Patch] - RM 5457 - Kerberos Authentication - Phase 1

Hi Khushboo

Jenkins build for OSX is failing with the below error can you please fix
and send the patch:

runTest (pgadmin.browser.tests.test_master_password.MasterPasswordTestCase)
TestCase for Create master password dialog ... 2021-01-18
12:07:32,542: ERROR flask.app: 400 Bad Request: The CSRF tokens do not
match.
Traceback (most recent call last):
File "/Users/jenkins/workspace/pgadmin4-macos-qa/venv/lib/python3.9/site-packages/flask_wtf/csrf.py",
line 256, in protect
validate_csrf(self._get_csrf_token())
File "/Users/jenkins/workspace/pgadmin4-macos-qa/venv/lib/python3.9/site-packages/flask_wtf/csrf.py",
line 106, in validate_csrf
raise ValidationError('The CSRF tokens do not match.')
wtforms.validators.ValidationError: The CSRF tokens do not match.

On Mon, Jan 18, 2021 at 4:40 PM Akshay Joshi <akshay(dot)joshi(at)enterprisedb(dot)com>
wrote:

> Thanks, patch applied.
>
> On Mon, Jan 18, 2021 at 4:07 PM Khushboo Vashi <
> khushboo(dot)vashi(at)enterprisedb(dot)com> wrote:
>
>>
>>
>> On Mon, Jan 18, 2021 at 3:26 PM Dave Page <dpage(at)pgadmin(dot)org> wrote:
>>
>>>
>>>
>>> On Mon, Jan 18, 2021 at 9:37 AM Khushboo Vashi <
>>> khushboo(dot)vashi(at)enterprisedb(dot)com> wrote:
>>>
>>>>
>>>>
>>>> On Mon, Jan 18, 2021 at 2:45 PM Dave Page <dpage(at)pgadmin(dot)org> wrote:
>>>>
>>>>> Hi
>>>>>
>>>>> On Mon, Jan 18, 2021 at 7:30 AM Khushboo Vashi <
>>>>> khushboo(dot)vashi(at)enterprisedb(dot)com> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> Please find the attached updated patch with the below changes:
>>>>>>
>>>>>> - Dependencies are added into Linux packages in the RPM/DEBs.
>>>>>> - Dev packages are added in the setup scripts for Linux.
>>>>>> - The required packages are added in the Dockerfile.
>>>>>> - Conditional gssapi 1.6.2 dependency is added for Python 3.5 in
>>>>>> requirements.txt.
>>>>>>
>>>>>
>>>>> 1.6.9 is the last release that supports Python 3.4+. We should use
>>>>> that rather than older versions.
>>>>>
>>>> As per the https://pypi.org/project/gssapi/*1.6.9*/, it says Requires: Python
>>>> >=3.6.*
>>>>
>>>
>>> I think that's the metadata for the latest package version on the left.
>>> If you read the main text, it says:
>>> Requirements
>>> Basic
>>>
>>> - A working implementation of GSSAPI (such as from MIT Kerberos)
>>> which includes header files
>>> - a C compiler (such as GCC)
>>> - either the enum34 Python package or Python 3.4+
>>> - the six and decorator python package
>>>
>>>
>>> For 1.6.10, that changed to:
>>> Requirements
>>> Basic
>>>
>>> - A working implementation of GSSAPI (such as from MIT Kerberos)
>>> which supports delegation and includes header files
>>> - a C compiler (such as GCC)
>>> - Python 3.6+ (older releases support older versions, but are
>>> unsupported)
>>> - the decorator python package
>>>
>>>
>>> I got the error as below for all the versions till 1.6.2.
>>
>> [image: Screen Shot 2021-01-18 at 3.27.59 PM.png]
>>
>> So, as per our conversation on slack, we will go with 1.6.2.
>>
>>
>>>>
>>>>>
>>>>>
>>>>>> - krb5 libs are not bundled with the Desktop packages, so added the
>>>>>> gssapi dependency into the try/catch block.
>>>>>> - .dockerignore is introduced to ignore unwanted files/folders like
>>>>>> node_modules etc., which will make the docker build fast. (By Ashesh Vashi)
>>>>>>
>>>>>
>>>>> Aside from that one comment above, eyeball review of the build changes
>>>>> looks good.
>>>>>
>>>>>
>>>>>
>>>>>>
>>>>>> Thanks,
>>>>>> Khushboo
>>>>>>
>>>>>> On Fri, Jan 15, 2021 at 3:48 PM Dave Page <dpage(at)pgadmin(dot)org> wrote:
>>>>>>
>>>>>>> And another thought...
>>>>>>>
>>>>>>> Some of the Jenkins QA jobs setup the virtual environment for
>>>>>>> running tests themselves. I believe these might actually be the cause of
>>>>>>> some of the failures we saw initially with the commit - I'll review those,
>>>>>>> and ensure they won't try to build the gssapi module from source on Windows.
>>>>>>>
>>>>>>> On Thu, Jan 14, 2021 at 4:34 PM Dave Page <dpage(at)pgadmin(dot)org> wrote:
>>>>>>>
>>>>>>>> FYI, I did a quick test (and browse of PyPI):
>>>>>>>>
>>>>>>>> - On Windows, it seems there is a binary wheel available:
>>>>>>>>
>>>>>>>> (gssapi) C:\Users\dpage>pip install gssapi
>>>>>>>> Collecting gssapi
>>>>>>>> Downloading gssapi-1.6.12-cp39-cp39-win_amd64.whl (670 kB)
>>>>>>>> |████████████████████████████████| 670 kB 3.3 MB/s
>>>>>>>> Collecting decorator
>>>>>>>> Downloading decorator-4.4.2-py2.py3-none-any.whl (9.2 kB)
>>>>>>>> Installing collected packages: decorator, gssapi
>>>>>>>> Successfully installed decorator-4.4.2 gssapi-1.6.12
>>>>>>>>
>>>>>>>> - On macOS, the wheel is built by pip, but it doesn't seem to have
>>>>>>>> any additional binary dependencies.
>>>>>>>>
>>>>>>>> This should simplify things a lot - we just need to ensure the
>>>>>>>> build scripts use the binary package on Windows, and install the build deps
>>>>>>>> on the Linux/Docker environments (and update the package builds with the
>>>>>>>> additional dependencies of course).
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, Jan 14, 2021 at 4:04 PM Dave Page <dpage(at)pgadmin(dot)org>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi Khushboo,
>>>>>>>>>
>>>>>>>>> As you know, this has been rolled back as the buildfarm blew up. I
>>>>>>>>> think there are a number of TODOs that need to be addressed, given that the
>>>>>>>>> gssapi Python module is dependent on MIT Kerberos:
>>>>>>>>>
>>>>>>>>> In the patch:
>>>>>>>>>
>>>>>>>>> - Linux packages will need the additional dependencies to be
>>>>>>>>> declared in the RPM/DEBs.
>>>>>>>>> - The setup scripts for Linux will need to have the -dev packages
>>>>>>>>> added as appropriate.
>>>>>>>>> - The various READMEs that describe how to build packages will
>>>>>>>>> need to be updated.
>>>>>>>>> - The Dockerfile will need to be modified to add the required
>>>>>>>>> packages.
>>>>>>>>> - The Windows build will need to be updated so the installer ships
>>>>>>>>> additional required DLLs.
>>>>>>>>> - Are there any additional macOS dependencies? If so, they need to
>>>>>>>>> be handled.
>>>>>>>>>
>>>>>>>>> In the buildfarm:
>>>>>>>>>
>>>>>>>>> - All Linux build VMs need to be updated with the additional
>>>>>>>>> dependencies.
>>>>>>>>> - On Windows, we need to figure out how to build/ship KfW. It's a
>>>>>>>>> pain to build, which we would typically do ourselves to ensure we're
>>>>>>>>> consistently using the same buildchain. If we do build it ourselves:
>>>>>>>>> - Will the Python package find it during it's build?
>>>>>>>>> - We'll need to create a Jenkins job to perform the build.
>>>>>>>>> - Is any work required on macOS, or does it ship with everything
>>>>>>>>> that's needed? If not, we'll need to build it, and create the Jenkins job.
>>>>>>>>>
>>>>>>>>> One final thought: on Windows/macOS, can we force a binary
>>>>>>>>> installation from PIP (pip install --only-binary=gssapi gssapi)? If so,
>>>>>>>>> will that include the required libraries, as psycopg2-binary does?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Thu, Jan 14, 2021 at 8:18 AM Akshay Joshi <
>>>>>>>>> akshay(dot)joshi(at)enterprisedb(dot)com> wrote:
>>>>>>>>>
>>>>>>>>>> Thanks, patch applied.
>>>>>>>>>>
>>>>>>>>>> On Thu, Jan 14, 2021 at 1:42 PM Khushboo Vashi <
>>>>>>>>>> khushboo(dot)vashi(at)enterprisedb(dot)com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> Please ignore my previous patch, attached the updated one.
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Khushboo
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Jan 14, 2021 at 12:17 PM Khushboo Vashi <
>>>>>>>>>>> khushboo(dot)vashi(at)enterprisedb(dot)com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi,
>>>>>>>>>>>>
>>>>>>>>>>>> Please find the attached updated patch.
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> Khushboo
>>>>>>>>>>>>
>>>>>>>>>>>> On Thu, Jan 14, 2021 at 12:00 PM Akshay Joshi <
>>>>>>>>>>>> akshay(dot)joshi(at)enterprisedb(dot)com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Khushboo
>>>>>>>>>>>>>
>>>>>>>>>>>>> Seems you have attached the wrong patch. Please send the
>>>>>>>>>>>>> updated patch.
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Jan 13, 2021 at 2:35 PM Khushboo Vashi <
>>>>>>>>>>>>> khushboo(dot)vashi(at)enterprisedb(dot)com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Please find the attached updated patch.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> Khushboo
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Fri, Jan 1, 2021 at 1:07 PM Aditya Toshniwal <
>>>>>>>>>>>>>> aditya(dot)toshniwal(at)enterprisedb(dot)com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi Khushboo,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I've just done the code review. Apart from below, the patch
>>>>>>>>>>>>>>> looks good to me:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 1) Move the auth source constants -ldap, kerberos out of app
>>>>>>>>>>>>>>> object. They don't belong there. You can create the constants
>>>>>>>>>>>>>>> somewhere else and import them.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> +app.PGADMIN_LDAP_AUTH_SOURCE = 'ldap'
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> +app.PGADMIN_KERBEROS_AUTH_SOURCE = 'kerberos'
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Done
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 2) Are we going to make kerberos default for wsgi ?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> *--- a/web/pgAdmin4.wsgi*
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> *+++ b/web/pgAdmin4.wsgi*
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> @@ -24,6 +24,10 @@ builtins.SERVER_MODE = True
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> import config
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> +config.AUTHENTICATION_SOURCES = ['kerberos']
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> +config.KERBEROS_AUTO_CREATE_USER = True
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Removed, it was only for testing.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 3) Remove the commented code.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> + # if self.form.data['email'] and
>>>>>>>>>>>>>>> self.form.data['password'] and \
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> + # source.get_source_name() ==\
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> + #
>>>>>>>>>>>>>>> current_app.PGADMIN_KERBEROS_AUTH_SOURCE:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> + # continue
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Removed the comment, it is actually the part of the code.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 4) KERBEROSAuthentication could be KerberosAuthentication
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> class KERBEROSAuthentication(BaseAuthentication):
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Done.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 5) You can use the constants (ldap, kerberos) you had
>>>>>>>>>>>>>>> defined when creating a user.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> + 'auth_source': 'kerberos'
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Done.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 6) The below URLs belong to the authenticate module.
>>>>>>>>>>>>>>> Currently they are in the browser module. I would also suggest rephrasing
>>>>>>>>>>>>>>> the URL from /kerberos_login to /login/kerberos. Same for logout.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Done the rephrasing as well as moved to the authentication
>>>>>>>>>>>>>> module.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Also, even though the method GET works, we should use the
>>>>>>>>>>>>>>> POST method for login and DELETE for logout.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Kerberos_login just redirects the page to the actual login,
>>>>>>>>>>>>>> so no need for the POST method.
>>>>>>>>>>>>>> I followed the same method for the Logout user we have used
>>>>>>>>>>>>>> for the normal user.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> +(at)blueprint(dot)route("/kerberos_login",
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> + endpoint="kerberos_login",
>>>>>>>>>>>>>>> methods=["GET"])
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> +(at)blueprint(dot)route("/kerberos_logout",
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> + endpoint="kerberos_logout",
>>>>>>>>>>>>>>> methods=["GET"])
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Tue, Dec 22, 2020 at 6:07 PM Akshay Joshi <
>>>>>>>>>>>>>>> akshay(dot)joshi(at)enterprisedb(dot)com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi Aditya
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Can you please do the code review?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Tue, Dec 22, 2020 at 3:44 PM Khushboo Vashi <
>>>>>>>>>>>>>>>> khushboo(dot)vashi(at)enterprisedb(dot)com> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Please find the attached patch to support Kerberos
>>>>>>>>>>>>>>>>> Authentication in pgAdmin RM 5457.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> The patch introduces a new pluggable option for Kerberos
>>>>>>>>>>>>>>>>> authentication, using SPNEGO to forward kerberos tickets through a browser
>>>>>>>>>>>>>>>>> which will bypass the login page entirely if the Kerberos Authentication
>>>>>>>>>>>>>>>>> succeeds.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> The complete setup of the Kerberos Server + pgAdmin
>>>>>>>>>>>>>>>>> Server + Client is documented in a separate file and attached.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> This patch also includes the small fix related to logging
>>>>>>>>>>>>>>>>> #5829
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>> Khushboo
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>> *Thanks & Regards*
>>>>>>>>>>>>>>>> *Akshay Joshi*
>>>>>>>>>>>>>>>> *pgAdmin Hacker | Principal Software Architect*
>>>>>>>>>>>>>>>> *EDB Postgres <http://edbpostgres.com>*
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> *Mobile: +91 976-788-8246*
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>> Aditya Toshniwal
>>>>>>>>>>>>>>> pgAdmin hacker | Sr. Software Engineer | *edbpostgres.com*
>>>>>>>>>>>>>>> <http://edbpostgres.com>
>>>>>>>>>>>>>>> "Don't Complain about Heat, Plant a TREE"
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> *Thanks & Regards*
>>>>>>>>>>>>> *Akshay Joshi*
>>>>>>>>>>>>> *pgAdmin Hacker | Principal Software Architect*
>>>>>>>>>>>>> *EDB Postgres <http://edbpostgres.com>*
>>>>>>>>>>>>>
>>>>>>>>>>>>> *Mobile: +91 976-788-8246*
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> *Thanks & Regards*
>>>>>>>>>> *Akshay Joshi*
>>>>>>>>>> *pgAdmin Hacker | Principal Software Architect*
>>>>>>>>>> *EDB Postgres <http://edbpostgres.com>*
>>>>>>>>>>
>>>>>>>>>> *Mobile: +91 976-788-8246*
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Dave Page
>>>>>>>>> Blog: http://pgsnake.blogspot.com
>>>>>>>>> Twitter: @pgsnake
>>>>>>>>>
>>>>>>>>> EDB: http://www.enterprisedb.com
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Dave Page
>>>>>>>> Blog: http://pgsnake.blogspot.com
>>>>>>>> Twitter: @pgsnake
>>>>>>>>
>>>>>>>> EDB: http://www.enterprisedb.com
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Dave Page
>>>>>>> Blog: http://pgsnake.blogspot.com
>>>>>>> Twitter: @pgsnake
>>>>>>>
>>>>>>> EDB: http://www.enterprisedb.com
>>>>>>>
>>>>>>>
>>>>>
>>>>> --
>>>>> Dave Page
>>>>> Blog: http://pgsnake.blogspot.com
>>>>> Twitter: @pgsnake
>>>>>
>>>>> EDB: http://www.enterprisedb.com
>>>>>
>>>>>
>>>
>>> --
>>> Dave Page
>>> Blog: http://pgsnake.blogspot.com
>>> Twitter: @pgsnake
>>>
>>> EDB: http://www.enterprisedb.com
>>>
>>>
>
> --
> *Thanks & Regards*
> *Akshay Joshi*
> *pgAdmin Hacker | Principal Software Architect*
> *EDB Postgres <http://edbpostgres.com>*
>
> *Mobile: +91 976-788-8246*
>

--
*Thanks & Regards*
*Akshay Joshi*
*pgAdmin Hacker | Principal Software Architect*
*EDB Postgres <http://edbpostgres.com>*

*Mobile: +91 976-788-8246*