Re: Minimal logical decoding on standbys

On Mon, 27 May 2019 at 19:26, Andres Freund <andres(at)anarazel(dot)de> wrote:
>
> On 2019-05-27 17:04:44 +0530, Amit Khandekar wrote:
> > On Fri, 24 May 2019 at 21:00, Amit Khandekar <amitdkhan(dot)pg(at)gmail(dot)com> wrote:
> > >
> > > On Fri, 24 May 2019 at 19:26, Amit Khandekar <amitdkhan(dot)pg(at)gmail(dot)com> wrote:
> > > > Working on the patch now ....
> > >
> > > Attached is an incremental WIP patch
> > > handle_wal_level_changes_WIP.patch to be applied over the earlier main
> > > patch logical-decoding-on-standby_v4_rebased.patch.
> >
> > I found an issue with these changes : When we change master wal_level
> > from logical to hot_standby, and again back to logical, and then
> > create a logical replication slot on slave, it gets created; but when
> > I do pg_logical_slot_get_changes() with that slot, it seems to read
> > records *before* I created the logical slot, so it encounters
> > parameter-change(logical=>hot_standby) record, so returns an error as
> > per the patch, because now in DecodeXLogOp() I error out when
> > XLOG_PARAMETER_CHANGE is found :
>
>
> > @@ -190,11 +190,23 @@ DecodeXLogOp(LogicalDecodingContext *ctx,
> > XLogRecordBuffer *buf)
> > * can restart from there.
> > */
> > break;
> > + case XLOG_PARAMETER_CHANGE:
> > + {
> > + xl_parameter_change *xlrec =
> > + (xl_parameter_change *) XLogRecGetData(buf->record);
> > +
> > + /* Cannot proceed if master itself does not have logical data */
> > + if (xlrec->wal_level < WAL_LEVEL_LOGICAL)
> > + ereport(ERROR,
> > + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
> > + errmsg("logical decoding on standby requires "
> > + "wal_level >= logical on master")));
> > + break;
> > + }
> >
> > I thought it won't read records *before* the slot was created. Am I
> > missing something ?
>
> That's why I had mentioned that you'd need to adapt
> ReplicationSlotReserveWal(), to use the replay LSN or such.

Yeah ok. I tried to do this :

@@ -1042,7 +1042,8 @@ ReplicationSlotReserveWal(void)
if (!RecoveryInProgress() && SlotIsLogical(slot))
{
....
}
else
{
- restart_lsn = GetRedoRecPtr();
+ restart_lsn = SlotIsLogical(slot) ?
+ GetXLogReplayRecPtr(&ThisTimeLineID) : GetRedoRecPtr();

But then when I do pg_create_logical_replication_slot(), it hangs in
DecodingContextFindStartpoint(), waiting to find new records
(XLogReadRecord).

Working on it ...

--
Thanks,
-Amit Khandekar
EnterpriseDB Corporation
The Postgres Database Company