Re: pgsql: Introduce replication slots.

On Sun, Feb 2, 2014 at 11:06 PM, Andres Freund <andres(at)2ndquadrant(dot)com> wrote:
> On 2014-02-02 14:44:17 +0900, Fujii Masao wrote:
>> On Sat, Feb 1, 2014 at 6:37 PM, Andres Freund <andres(at)2ndquadrant(dot)com> wrote:
>> > On 2014-02-01 18:33:08 +0900, Fujii Masao wrote:
>> >> On Sat, Feb 1, 2014 at 6:09 PM, Andres Freund <andres(at)2ndquadrant(dot)com> wrote:
>> >> > On 2014-02-01 16:47:47 +0900, Fujii Masao wrote:
>> >> >> This patch changed basebackup.c so that it skips pg_replslot. It's OK
>> >> >> to skip all files in that directory, but an empty pg_replslot must be
>> >> >> included in the backup. Otherwise we cannot start PostgreSQL from
>> >> >> the backup taken via pg_basebackup. Attached patch fixes this problem.
>> >> >
>> >> > That's a pretty fair point. Not sure how that could escape my
>> >> > notice. The patch does look sane to me.
>> >> >
>> >> > I wonder if we additionally should add code to recreate pg_replslot on
>> >> > startup, similar to pg_xlog?
>> >>
>> >> Similar to pg_xlog/archive_status, not pg_xlog? That might be an option.
>> >> But I'm not inclined to do that for now. The fact that the essential
>> >> directory like pg_replslot doesn't exist indicates the symptom of
>> >> something strange. And, changing that way might prevent us from
>> >> detecting such symptom.
>> >
>> > The reason I am wondering is that it makes a fair bit of sense to
>> > exclude it in open-coded base backups as well, and excluding the
>> > entire directory might be the easiest way there. But I guess people
>> > manage for pg_xlog/, so it's really not something that would reduce pain
>> > measurably.
>>
>> On second thought, we should always delete all files in pg_replslot
>> when starting recovery from the backup?
>
> Are you suggesting to always delete them when in standby_mode? If so,
> no, that'd be bad, we intentially *do* want to support situations in
> which we stream from the standby, i.e. cascading types of setups.
>
> If you mean doing so when initially starting with a backup label,
> hm. Maybe. That would make it impossible to keep replication slots when
> moving to a new server with a short downtime, which seems a bit
> annoying.

Yeah, I was thinking the latter.

>> The users who are
>> using their own backup script instead of pg_basebackup might
>> include pg_replslot files in the backup. Currently, in this case,
>> the replication slots which were created before would be available
>> even after the recovery. Is this OK? If not, all files in pg_replslot
>> should be removed at the beginning of the PITR. OTOH, if that's OK,
>> I think that pg_basebackup should not skip pg_replslot files.
>> Thought?
>
> Robert raised this previously in
> http://archives.postgresql.org/message-id/CA%2BTgmoar6BLb%2B7BQUYEmkmdFSE1f8khCZCDP-aCojOrESiNLBg%40mail.gmail.com :
>> - Exclude pg_replslot from base backups. This might need more thought
>> and documentation; people who use the filesystem method to perform
>> backups might need to be advised to remove this directory in some
>> cases also, or people who use pg_basebackup might want to keep it in
>> some cases (not sure).
>
> I can see usecases for removing and keeping them. Removing them has the
> big advantage that the user won't be surprised by a slot's existance
> which prevents resources (WAL, xmin horizon/VACUUM) from being
> reclaimed.

Agreed.

Regards,

--
Fujii Masao