答复: [External] Re: pgadmin--pgagent---the process hang by unknow reasons

Hi Dave

Thanks for your quick response.
This env was deploy on Jan 2019 by my team mate. Currently the pgagent have 30 jobs running. The version is pgagent_10-3.4.0
I don’t know how to upgrade the pgagent. I try to find upgrade document. But failed.
Only find that edb ppas could upgrade pgagent.
https://www.enterprisedb.com/edb-docs/d/edb-postgres-advanced-server/installation-getting-started/upgrade-guide/11/EDB_Postgres_Advanced_Server_Upgrade_Guide.1.13.html

If I re-install the pgagent with latest version. Whether the old pgagent jobs will drop with old version pgagent ?
Would you like to guide me to make pgagent using new Boost package and no impact currently working pgagent jobs ?
I don’t want to rebuild all pgagent jobs. Thanks in advance.

徐志宇（Jack）
Database Engineer

DB Team,ITS. Lenovo China
Phone: 86-18910860709
Email:xuzy13(at)lenovo(dot)com
No.6 Shangdi West Road, Haidian District Beijing, China, 100085

发件人: Dave Page <dpage(at)pgadmin(dot)org>
发送时间: 2020年11月26日 19:39
收件人: Zhiyu ZY13 Xu <xuzy13(at)lenovo(dot)com>
抄送: pgadmin-support(at)postgresql(dot)org
主题: [External] Re: pgadmin--pgagent---the process hang by unknow reasons

Hi

Given the libwx* references in your stacktrace, you appear to be using an old version of pgagent - we removed the dependency on wxWidgets nearly 2.5 years ago and replaced it with Boost.

Please upgrade and try again.

Thanks.

On Thu, Nov 26, 2020 at 8:05 AM Zhiyu ZY13 Xu <xuzy13(at)lenovo(dot)com<mailto:xuzy13(at)lenovo(dot)com>> wrote:
Hi Support

I using pgagent over 2 years. There are over 30 jobs running by pgagent. Recently. I found a problem that sometime the pgagent hang by unknow reasons.
From the stack information. Look like the pagent experience dead-lock issue in code.
The stack display many thread stop on this function “in __lll_lock_wait”
If you need more information. Please let me know. I suspect this is a bug.

I collect to pgagent trace log and stack information on the attachment.

pgagent trace log
pg_agent_11_24.log
pg_agent_11_26.log
pgagent process stack
others information.

version:
pgagent_10-3.4.0-10.rhel6.x86_64
PG 10.5

The typical stack information.

[postgres(at)sltfjfrauxq pgagent_pd]$ cat 23389.stark.1
Thread 7 (Thread 0x7ff745f5c700 (LWP 906)):
#0 0x00007ff74b003334 in __lll_lock_wait () from /lib64/libpthread.so.0
#1 0x00007ff74affe5d8 in _L_lock_854 () from /lib64/libpthread.so.0
#2 0x00007ff74affe4a7 in pthread_mutex_lock () from /lib64/libpthread.so.0
#3 0x00007ff74ba979c9 in wxMutexInternal::Lock() () from /usr/lib64/libwx_baseu-2.8.so.0
#4 0x00007ff74c15b819 in DBconn::Return() ()
#5 0x00007ff74c161217 in Job::Execute() ()
#6 0x00007ff74c162899 in JobThread::Entry() ()
#7 0x00007ff74ba99021 in wxThreadInternal::PthreadStart(wxThread*) () from /usr/lib64/libwx_baseu-2.8.so.0
#8 0x00007ff74affcaa1 in start_thread () from /lib64/libpthread.so.0
#9 0x00007ff74ad49c4d in clone () from /lib64/libc.so.6
Thread 6 (Thread 0x7ff72ffff700 (LWP 908)):
#0 0x00007ff74b003334 in __lll_lock_wait () from /lib64/libpthread.so.0
#1 0x00007ff74affe5d8 in _L_lock_854 () from /lib64/libpthread.so.0
#2 0x00007ff74affe4a7 in pthread_mutex_lock () from /lib64/libpthread.so.0
#3 0x00007ff74ba979c9 in wxMutexInternal::Lock() () from /usr/lib64/libwx_baseu-2.8.so.0
#4 0x00007ff74c15b819 in DBconn::Return() ()
#5 0x00007ff74c161217 in Job::Execute() ()
#6 0x00007ff74c162899 in JobThread::Entry() ()
#7 0x00007ff74ba99021 in wxThreadInternal::PthreadStart(wxThread*) () from /usr/lib64/libwx_baseu-2.8.so.0
#8 0x00007ff74affcaa1 in start_thread () from /lib64/libpthread.so.0
#9 0x00007ff74ad49c4d in clone () from /lib64/libc.so.6
Thread 5 (Thread 0x7ff74695d700 (LWP 910)):
#0 0x00007ff74b003334 in __lll_lock_wait () from /lib64/libpthread.so.0
#1 0x00007ff74affe5d8 in _L_lock_854 () from /lib64/libpthread.so.0
#2 0x00007ff74affe4a7 in pthread_mutex_lock () from /lib64/libpthread.so.0
#3 0x00007ff74ba979c9 in wxMutexInternal::Lock() () from /usr/lib64/libwx_baseu-2.8.so.0
#4 0x00007ff74c15b819 in DBconn::Return() ()
#5 0x00007ff74c161217 in Job::Execute() ()
#6 0x00007ff74c162899 in JobThread::Entry() ()
#7 0x00007ff74ba99021 in wxThreadInternal::PthreadStart(wxThread*) () from /usr/lib64/libwx_baseu-2.8.so.0
#8 0x00007ff74affcaa1 in start_thread () from /lib64/libpthread.so.0
#9 0x00007ff74ad49c4d in clone () from /lib64/libc.so.6
Thread 4 (Thread 0x7ff74735e700 (LWP 1565)):
#0 0x00007ff74b003334 in __lll_lock_wait () from /lib64/libpthread.so.0
#1 0x00007ff74affe5d8 in _L_lock_854 () from /lib64/libpthread.so.0
#2 0x00007ff74affe4a7 in pthread_mutex_lock () from /lib64/libpthread.so.0
#3 0x00007ff74ba979c9 in wxMutexInternal::Lock() () from /usr/lib64/libwx_baseu-2.8.so.0
#4 0x00007ff74c15b819 in DBconn::Return() ()
#5 0x00007ff74c161217 in Job::Execute() ()
#6 0x00007ff74c162899 in JobThread::Entry() ()
#7 0x00007ff74ba99021 in wxThreadInternal::PthreadStart(wxThread*) () from /usr/lib64/libwx_baseu-2.8.so.0
#8 0x00007ff74affcaa1 in start_thread () from /lib64/libpthread.so.0
#9 0x00007ff74ad49c4d in clone () from /lib64/libc.so.6
Thread 3 (Thread 0x7ff74555b700 (LWP 1567)):
#0 0x00007ff74ad40403 in poll () from /lib64/libc.so.6
#1 0x00007ff74bd1c28f in ?? () from /usr/lib64/libpq.so.5
#2 0x00007ff74bd1c310 in ?? () from /usr/lib64/libpq.so.5
#3 0x00007ff74bd178e2 in ?? () from /usr/lib64/libpq.so.5
#4 0x00007ff74bd1865f in PQconnectdb () from /usr/lib64/libpq.so.5
#5 0x00007ff74c15ad71 in DBconn::Connect(wxString const&) ()
#6 0x00007ff74c15af73 in DBconn::DBconn(wxString const&, wxString const&) ()
#7 0x00007ff74c15bfe8 in DBconn::Get(wxString const&, wxString const&) ()
#8 0x00007ff74c16108f in Job::Execute() ()
#9 0x00007ff74c162899 in JobThread::Entry() ()
#10 0x00007ff74ba99021 in wxThreadInternal::PthreadStart(wxThread*) () from /usr/lib64/libwx_baseu-2.8.so.0
#11 0x00007ff74affcaa1 in start_thread () from /lib64/libpthread.so.0
#12 0x00007ff74ad49c4d in clone () from /lib64/libc.so.6
Thread 2 (Thread 0x7ff744b5a700 (LWP 1569)):
#0 0x00007ff74b003334 in __lll_lock_wait () from /lib64/libpthread.so.0
#1 0x00007ff74affe5d8 in _L_lock_854 () from /lib64/libpthread.so.0
#2 0x00007ff74affe4a7 in pthread_mutex_lock () from /lib64/libpthread.so.0
#3 0x00007ff74ba979c9 in wxMutexInternal::Lock() () from /usr/lib64/libwx_baseu-2.8.so.0
#4 0x00007ff74c15bf6b in DBconn::Get(wxString const&, wxString const&) ()
#5 0x00007ff74c16108f in Job::Execute() ()
#6 0x00007ff74c162899 in JobThread::Entry() ()
#7 0x00007ff74ba99021 in wxThreadInternal::PthreadStart(wxThread*) () from /usr/lib64/libwx_baseu-2.8.so.0
#8 0x00007ff74affcaa1 in start_thread () from /lib64/libpthread.so.0
#9 0x00007ff74ad49c4d in clone () from /lib64/libc.so.6
Thread 1 (Thread 0x7ff74c3507e0 (LWP 23389)):
#0 0x00007ff74b003334 in __lll_lock_wait () from /lib64/libpthread.so.0
#1 0x00007ff74affe5d8 in _L_lock_854 () from /lib64/libpthread.so.0
#2 0x00007ff74affe4a7 in pthread_mutex_lock () from /lib64/libpthread.so.0
#3 0x00007ff74ba979c9 in wxMutexInternal::Lock() () from /usr/lib64/libwx_baseu-2.8.so.0
#4 0x00007ff74c15a99d in DBconn::ClearConnections(bool) ()
#5 0x00007ff74c15e908 in MainRestartLoop(DBconn*) ()
#6 0x00007ff74c15f2a3 in MainLoop() ()
#7 0x00007ff74c15e016 in main ()

徐志宇（Jack）
Database Engineer

DB Team,ITS. Lenovo China
Phone: 86-18910860709
Email:xuzy13(at)lenovo(dot)com<mailto:Email%3Axuzy13(at)lenovo(dot)com>
No.6 Shangdi West Road, Haidian District Beijing, China, 100085

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EDB: http://www.enterprisedb.com