Re: logical replication snapshots

On 07/26/2018 10:54 AM, Dimitri Maziuk wrote:
> On 07/25/2018 07:57 PM, Andres Freund wrote:
>> On 2018-07-25 12:31:01 -0700, Adrian Klaver wrote:
>
>>>> Would "permission denied" be relevant?
>>>
>>> Logical decoding is something I am still learning. The "permission denied"
>>> would to me be relevant only to the extent that it seems to be provoking:
>>
>> Yes, it looks related. Looks like logical rep was trying to perform the
>> intiial sync of those tables, and couldn't due to permission errors.
>
> I'm not sure what happened, I remember the initial sync of that
> particular schema failing on one table only, but looking at it now, all
> tables are empty on the subscriber.

To me that indicates all the syncs failed.

>
>>> "LOG: logical decoding found consistent point at 19/E6942440"
>>> DETAIL: There are no running transactions."
>>>
>>> Others with more experience in this area would need to fill whether that
>>> might account for the 13 million files in the ~snapshot/ directory.
>>
>> That indicates there's some error handling issues to be resolved. Petr?
>
> It logs "COPY TO STDOUT", does that actually cache to disk (in ~snapshot/)?

That is getting deeper into the code then I have been, so I don't know.
FYI the code is below and the part that triggered the error is around
line 762 (copy_table(Relation rel)).

https://git.postgresql.org/gitweb/?p=postgresql.git;a=blob;f=src/backend/replication/logical/tablesync.c;h=acc6498567d07c3e93d0adf5039a10c693e38f64;hb=HEAD

>
> Would it be safe to delete every file in ~snapshot/ older than some
> X-number of minutes? (And what a good X might be?)
>

Again I don't know the answer to this. Are you trying this on a test
setup or production one?

--
Adrian Klaver
adrian(dot)klaver(at)aklaver(dot)com