Request for inputs on #6208 - MaxQueuePool Issue

From: Yogesh Mahajan <yogesh(dot)mahajan(at)enterprisedb(dot)com>
To: pgadmin-hackers <pgadmin-hackers(at)postgresql(dot)org>
Cc: Nikhil Mohite <nikhil(dot)mohite(at)enterprisedb(dot)com>
Subject: Request for inputs on #6208 - MaxQueuePool Issue
Date: 2023-08-08 08:17:16
Message-ID: CAMa=N=NVxKMyofyEL-53i8DV1=6PmgEjmomrXs1hya7BB8kcyw@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgadmin-hackers

Hi Hackers,

For the #6208 <https://github.com/pgadmin-org/pgadmin4/issues/6208>, below
are the observations about this issue -

On each API request, an application db connection is created with state
'idle in transaction'. Connection state is changed to idle only after a
successful response from the web server. If an exception occurs while
processing a request which is not handled and response is not sent, the
application db connection remains orphaned. This connection is only reset
on application restart.

Issue - In case of pgAdmin, if the user openes 15 query tool tabs & all of
them have long running transactions like pg_sleep(), then opening new query
tool/or any operation on pgAdmin which hits API request to backend will
throw 'QueuePool Limit reached' error. (Because by default, SQLAlchemy
allows 15 connections total: 5 connections in pool & 10 in overflow and
pgAdmin uses default setting.) OR if the user executes a query in the
query tool & while query execution is in progress, hits F5(keyboard
shortcut to execute query) 16 times continuously, then pgAdmin throws
'QueuePool Limit reached' error. After that, the query tool gives incorrect
responses to the queries.

Solutions -
1.Provide configurable settings for 'pool_size' & 'max_overflow' parameters
for SQLAlchemy.
2.Disable pooling using NullPool.A Pool which does not pool connections.
Instead it literally opens and closes the underlying DB-API connection per
each connection open/close. Using NullPool may impact the performance.

What approach should be followed to fix the issue?

@Nikhil Mohite <nikhil(dot)mohite(at)enterprisedb(dot)com> Please add if anything is
missed.

Thanks,
Yogesh Mahajan
EnterpriseDB

Responses

Browse pgadmin-hackers by date

  From Date Subject
Next Message Yogesh Mahajan 2023-08-09 05:19:59 [pgadmin-org/pgadmin4] 0b24a8: Ensure keyboard shortcut for query execution is di...
Previous Message MysticBoy 2023-08-08 06:26:45 [pgadmin-org/pgadmin4] 8afa17: Chinese translation.