Re: logical replication snapshots

From: Andres Freund <andres(at)anarazel(dot)de>
To: Dimitri Maziuk <dmaziuk(at)bmrb(dot)wisc(dot)edu>, Petr Jelinek <petr(at)2ndquadrant(dot)com>, Alvaro Herrera <alvherre(at)2ndquadrant(dot)com>, Peter Eisentraut <peter_e(at)gmx(dot)net>
Cc: pgsql-general <pgsql-general(at)lists(dot)postgresql(dot)org>
Subject: Re: logical replication snapshots
Date: 2018-07-23 23:51:00
Message-ID: 20180723235100.qvztneum7ipdlyir@alap3.anarazel.de
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

Hi,

On 2018-07-23 15:23:04 -0500, Dimitri Maziuk wrote:
> I'm playing with logical replication in postgres-10 and I got it to hang
> on startup with
>
> > 2018-07-23 13:15:06.577 CDT [18624] FATAL: the database system is starting up
> > 2018-07-23 13:15:06.577 CDT [18624] DEBUG: shmem_exit(1): 0 before_shmem_exit callbacks to make
> > 2018-07-23 13:15:06.577 CDT [18624] DEBUG: shmem_exit(1): 0 on_shmem_exit callbacks to make
> > 2018-07-23 13:15:06.577 CDT [18624] DEBUG: proc_exit(1): 1 callbacks to make
> > 2018-07-23 13:15:06.577 CDT [18624] DEBUG: exit(1)
> > 2018-07-23 13:15:06.577 CDT [18624] DEBUG: shmem_exit(-1): 0 before_shmem_exit callbacks to make
> > 2018-07-23 13:15:06.577 CDT [18624] DEBUG: shmem_exit(-1): 0 on_shmem_exit callbacks to make
> > 2018-07-23 13:15:06.577 CDT [18624] DEBUG: proc_exit(-1): 0 callbacks to make
> ... forever ...

What precedes that "loop"?

Could you describe what exactly you did into that situation?

> After poking around:
>
> # find /var/lib/pgsql/10/data/pg_logical/snapshots -printf . | wc -c
> 12727975

Ugh, something absolutely clearly isn't right here.

> It looks like ZFS simply can't handle this many files: `ls` etc. hang
> forever.

ls probably works in general, it's just the sorting that makes it
hang. Should work with -f or such.

Greetings,

Andres Freund

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Michael Paquier 2018-07-24 05:58:09 Re: Waiting connections postgres 10
Previous Message Kevin Brannen 2018-07-23 21:54:07 RE: Improving pg_dump performance