Re: logical replication snapshots

On 07/25/2018 11:10 AM, Dimitri Maziuk wrote:
> On 07/25/2018 11:16 AM, Adrian Klaver wrote:
>
>> Where you using pg_export_snapshot() by any chance?:
>
> If it's not in chapter 31 of TFM then I have not touched it. The only
> notable thing I changed from 31.9 quickstart is that replica identity is
> full on all tables (the data's a mess).

So no.

>
>> Where there any relevant error messages in the log before the database
>> hung?
>
> Do you seriously expect me to know what "relevant" is? ;) -- It's 7
> days' worth of

Generally speaking relevancy increases the closer you get to the fall
over event. So we are most interested in entries close to the point
where the database stopped/failed.

>
>> 2018-07-23 09:43:12.851 CDT [7975] STATEMENT: COPY macromolecules."Entity_common_name" TO STDOUT
>> 2018-07-23 09:43:12.925 CDT [7982] LOG: logical decoding found consistent point at 19/E6942408
>> 2018-07-23 09:43:12.925 CDT [7982] DETAIL: There are no running transactions.
>> 2018-07-23 09:43:12.935 CDT [7982] ERROR: permission denied for schema macromolecules
>> 2018-07-23 09:43:12.935 CDT [7982] STATEMENT: COPY macromolecules."Assembly" TO STDOUT
>> 2018-07-23 09:43:13.034 CDT [7987] LOG: logical decoding found consistent point at 19/E6942440
>> 2018-07-23 09:43:13.034 CDT [7987] DETAIL: There are no running transactions.
>> 2018-07-23 09:43:13.044 CDT [7987] ERROR: permission denied for schema macromolecules
>> 2018-07-23 09:43:13.044 CDT [7987] STATEMENT: COPY macromolecules."Spectral_transition" TO STDOUT
>> 2018-07-23 09:43:13.135 CDT [7990] LOG: logical decoding found consistent point at 19/E6942478
>> 2018-07-23 09:43:13.135 CDT [7990] DETAIL: There are no running transactions.
>> 2018-07-23 09:43:13.145 CDT [7990] ERROR: permission denied for schema macromolecules
>> 2018-07-23 09:43:13.145 CDT [7990] STATEMENT: COPY macromolecules."Assembly_db_link" TO STDOUT
>
> that ends with
>
>> 2018-07-23 09:45:40.280 CDT [6032] LOG: database system was interrupted; last known up at 2018-07-06 17:04:28 CDT
>> 2018-07-23 09:45:40.281 CDT [6033] FATAL: the database system is starting up
>> 2018-07-23 09:45:40.282 CDT [6034] FATAL: the database system is starting up
> ... ad infinitum ...
>
> Would "permission denied" be relevant?

Logical decoding is something I am still learning. The "permission
denied" would to me be relevant only to the extent that it seems to be
provoking:

"LOG: logical decoding found consistent point at 19/E6942440"
DETAIL: There are no running transactions."

Others with more experience in this area would need to fill whether that
might account for the 13 million files in the ~snapshot/ directory.

>
> (Not sure where that is coming from: I did initial sync as postgres then
> altered it to replication user. Can't track it down now since the
> database is dead.)
>

In your previous post you said:

"(To somebody else as I'm obviously not turning logical replication back
on until I know it won't kill my server again.) "

So what are you working off now?

--
Adrian Klaver
adrian(dot)klaver(at)aklaver(dot)com