Re: Disable WAL logging to speed up data loading

()

On Mon, Nov 30, 2020 at 2:39 PM Kyotaro Horiguchi
<horikyota(dot)ntt(at)gmail(dot)com> wrote:
>
> At Fri, 27 Nov 2020 13:34:49 +0000, "osumi(dot)takamichi(at)fujitsu(dot)com" <osumi(dot)takamichi(at)fujitsu(dot)com> wrote in
> > Thank you, Horiguchi-San
> >
> > > I haven't seen a criteria of whether a record is emitted or not for
> > > wal_leve=none.
> > >
> > > We're emitting only redo logs. So I think theoretically we don't need anything
> > > other than the shutdown checkpoint record because we don't perform
> > > recovery and checkpoint record is required at startup.
> > >
> > > RM_XLOG_ID:
> > > XLOG_FPI_FOR_HINT - not needed?
> > > XLOG_FPI - not needed?
> > >
> > > XLOG_CHECKPOINT_SHUTDOWN - must have
> > >
> > > So how about the followings?
> > > XLOG_CHECKPOINT_ONLINE
> > > XLOG_NOOP
> > > XLOG_NEXTOID
> > > XLOG_SWITCH
> > > XLOG_BACKUP_END
> > > XLOG_PARAMETER_CHANGE
> > > XLOG_RESTORE_POINT
> > > XLOG_FPW_CHANGE
> > > XLOG_END_OF_RECOVERY
> > >
> > >
> > > RM_XACT_ID:
> > > XLOG_XACT_COMMIT
> > > XLOG_XACT_PREPARE
> > > XLOG_XACT_ABORT
> > > XLOG_XACT_COMMIT_PREPARED
> > > XLOG_XACT_ABORT_PREPARED
> > > XLOG_XACT_ASSIGNMENT
> > > XLOG_XACT_INVALIDATIONS
> > >
> > > Do we need all of these?
> > No. Strictly speaking, you are right.
> > We still have types of WAL that are not necessarily needed.
> > For example, XLOG_END_OF_RECOVERY is not useful
> > because wal_level=none doesn't recover from any accidents.
> > Or, XLOG_CHECKPOINT_ONLINE is used when we execute CHECKPOINT
> > not for shutting down. Thus we could eliminate more.
>
> Yeah, although it's enough only to restrict non-harmful records
> practically, if we find that only a few kinds of records are needed,
> maybe it's cleaner to allow only required record type(s).
>
> > > And, currenly what decides whether to emit a wal record according to
> > > wal_level is the caller of XLogInsert.
> > Yes.
> >
> > > So doing this at XLogInsert-level means
> > > that we bring the criteria of the necessity of wal-record into xlog layer only for
> > > wal_level=none. I'm not sure it is the right direction.
> > I'm sorry. I didn't understand what "doing this" and "xlog layer" meant.
>
> "doing this" meant filtering record types.
>
> > Did you mean that fixing the caller side of XLogInsert (e.g. CreateCheckPoint)
> > is not the right direction ? Or, fixing the function of XLogInsert is not the right direction ?
>
> Maybe it's right that if we can filter-out records looking only rmid,
> since the xlog facility doesn't need to know about record types of a
> resource manager. But if we need to finer-grained control on the
> record types, I'm afraid that that's wrong.

I also think we should have the caller of XLogInsert() control whether
or not to write a WAL record according to the setting of wal_level
like XLogStandbyInfoActive() and XLogLogicalInfoActive(). Since most
callers seem to assume the wal record will be written when calling to
XLogInsert() I think it's better not to break that assumption.

> However, if we need only
> the XLOG_CHECKPOINT_SHUTDOWN record, it might be better to let
> XLogInsert filter records rather than inserting that filtering code to
> all the caller sites.
>
> > > At Fri, 27 Nov 2020 07:01:16 +0000, "tsunakawa(dot)takay(at)fujitsu(dot)com"
> > > <tsunakawa(dot)takay(at)fujitsu(dot)com> wrote in
> > > > I'm afraid "none" doesn't represent the behavior because RM_XLOG_ID and
> > > RM_XACT_ID WAL records, except for XLOG_FPI_*, are emitted. What's the
> > > good name? IIUC, "minimal" is named after the fact that the minimal
> > > amount of WAL necessary for crash recovery is generated. "norecovery" or
> > > "unrecoverable"?
> > Lastly, I found another name which expresses the essential characteristic of this wal_level.
> > How about the name of wal_level="crash_unsafe" ?
> > What did you think ?
>
> I don't dislike "none" since it seems to me practically "none".

Me too.

> It seems rather correct if we actually need only the shutdown checkpoint
> record.
>
> "unrecoverable" is apparently misleading. "crash_unsafe" is precise
> but seems somewhat alien being among "logical", "replica" and
> "minimal".

Since the WAL protocol is for durability I'm concerned the name
'crash_unsafe' (and 'unrecoverable') leads to a contradiction.

Regards,

--
Masahiko Sawada
EnterpriseDB: https://www.enterprisedb.com/