Re: Race condition while creating a new partition

From: Justin <zzzzz(dot)graf(at)gmail(dot)com>
To: Andrei Zhidenkov <andrei(dot)zhidenkov(at)n26(dot)com>, "pgsql general (pgsql-general(at)postgresql(dot)org)" <pgsql-general(at)postgresql(dot)org>
Subject: Re: Race condition while creating a new partition
Date: 2019-12-16 16:49:45
Message-ID: CALL-XeM-CpWpUGqJrpw21Rdun7QJQwFV5sH9Pk7i13XJyzX93w@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

Hi Andrei

General speaking any DDL (Create, Alter Drop .etc) commands issue
exclusive locks automatically, so anything this transaction touches starts
getting exclusive locks. Assuming this is a multi-threading app these two
threads are sending commands all but at the same time. The Exclusive lock
has not been committed yet as its waiting for a previous insert to finish,
and the Exclusive lock is blocking the insert as its transaction started
prior to the Insert.

the wait queue is pretty long 84, 80, 82, 83, 87, 88, 85, 86.
The time stamps are only 2 ms apart, lots of commands being sent in at
nearly the same time...

Here are my suggestions
Tweak the Python app to make sure all the threads have committed their
transactions place a lock on all the other threads, Then issue a DDL
command, when the command is committed, then unlock the other threads.
That should kill off the race condition deadlock..

Post all the SQL code that is creating this exclusive lock, we may be able
re-write the SQL code to reduce the time and/or resources this Exclusive
Lock is creating.

Pyscopg2 default mode is to require issuing a Commit it does not
automatically commit transaction. If running in auto-commit OFF try
turning this ON for DDL commands may fix the issue. It may also be that
DDL command takes far longer than 1 second to complete...

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Sar 2019-12-16 17:46:04 Experiencing error during restore - found unexpected block ID (0)
Previous Message Andrei Zhidenkov 2019-12-16 15:13:44 Re: Race condition while creating a new partition