Re: Londiste 3 pgq events_1_1 table huge

From: Saiful Muhajir <saifulmuhajir(at)gmail(dot)com>
To: "Rene (dot)" <rene0_3(at)hotmail(dot)com>
Cc: Leonardo M(dot) Ramé <l(dot)rame(at)griensu(dot)com>, PostgreSql-general <pgsql-general(at)postgresql(dot)org>
Subject: Re: Londiste 3 pgq events_1_1 table huge
Date: 2016-05-19 15:39:58
Message-ID: CAA0dH_uHP3i6CN=VNajP2C7Rh9bvVp2am_tWRg-S38JXjg8jzQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

This has happened to us where we have dead or unmanaged consumer. Turns out
londiste is keeping the event even if the consumer is unreachable. This is
to ensure that the consumer gets what it should.

To clean this up, delete the unused/dead consumer, with qadmin or manually
if necessary. The table won't be deleted immediately though. We have to
restart pgqd and workers and wait for two days.

~
Saiful Muhajir
On 19 May 2016 20:01, "Rene ." <rene0_3(at)hotmail(dot)com> wrote:

> Queue - Event are stored in queue tables i.e queues. Several producers can
> write
> into same queue and several consumers can read from the queue. Events are
> kept
> in queue until all the consumers have seen them.
>
> Maybe you have some inactive consumers holding a event tables.
>
> qadmin -h <root_node_host> -p 5432 -U postgres -d <database name> -Q
> <queue name>
>
> Use 'show help;' to see available commands.
> copy output of show consumer command
>
> show consumer;
>
> Rene
>
> ________________________________________
> From: pgsql-general-owner(at)postgresql(dot)org <
> pgsql-general-owner(at)postgresql(dot)org> on behalf of Leonardo M. Ramé <
> l(dot)rame(at)griensu(dot)com>
> Sent: Thursday, May 19, 2016 2:43 PM
> To: PostgreSql-general
> Subject: Re: [GENERAL] Londiste 3 pgq events_1_1 table huge
>
> El 18/05/16 a las 19:03, Rene . escribió:
> > Hi, Check for long running Idle in transaction sessions. Idle in
> transaction sessions may holding events table from cleaning itself up.
> > If there is more then days long running idle in transaction sessions,
> kill them, event table should be cleaned automatically after that.
> >
> > "select pid,state, query_start from pg_stat_activity where state='idle
> in transaction';" for checking sessions.
> >
> > Rene
>
> Thanks Rene, I found only one "idle in transaction" and it dates from
> just a couple of minutes ago, so, the reason of huge event table must be
> something else.
>
> By looking at the event_1_1 table I found records from march, but
> londiste3 status shows everything is already in sync:
>
> nodo_master (root)
> | Tables: 146/0/0
> | Lag: 0s, Tick: 1112197
> +--: node_esclavo (leaf)
> Tables: 146/0/0
> Lag: 0s, Tick: 1112197
>
> So, what if I manually delete old events?.
>
>
> Regards,
> --
> Leonardo M. Ramé
> Medical IT - Griensu S.A.
> Av. Colón 636 - Piso 8 Of. A
> X5000EPT -- Córdoba
> Tel.: +54(351)4246924 +54(351)4247788 +54(351)4247979 int. 19
> Cel.: +54 9 (011) 40871877
>
>
> --
> Sent via pgsql-general mailing list (pgsql-general(at)postgresql(dot)org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-general
>
> --
> Sent via pgsql-general mailing list (pgsql-general(at)postgresql(dot)org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-general
>

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Cameron Smith 2016-05-19 18:30:34 PostgreSQL with BDR - PANIC: could not create replication identifier checkpoint
Previous Message Peter Juhasz 2016-05-19 14:37:48 PQcancel may hang in the recv call