Re: Disable WAL logging to speed up data loading

At Fri, 27 Nov 2020 07:01:16 +0000, "tsunakawa(dot)takay(at)fujitsu(dot)com" <tsunakawa(dot)takay(at)fujitsu(dot)com> wrote in
> From: Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com>
> > While testing the patch on some workload, I realized that
> > XLOG_FPI_FOR_HINT record could still be emitted even when wal_level =
> > none. IIUC that WAL record is not necessary during wal_level = none
> > since the server cannot be the primary server and the server crash
> > ends up requiring to restore the whole database.
>
> Nice catch! XLOG_FPI_FOR_HINT and XLOG_FPI should be eliminated, otherwise large amount of WAL may be written. (It seems that other RMIDs than RM_XLOG_ID and RM_XACT_ID do not have to be written.)
>
> I'm afraid "none" doesn't represent the behavior because RM_XLOG_ID and RM_XACT_ID WAL records, except for XLOG_FPI_*, are emitted. What's the good name? IIUC, "minimal" is named after the fact that the minimal amount of WAL necessary for crash recovery is generated. "norecovery" or "unrecoverable"?

I haven't seen a criteria of whether a record is emitted or not for
wal_leve=none.

We're emitting only redo logs. So I think theoretically we don't need
anything other than the shutdown checkpoint record because we don't
perform recovery and checkpoint record is required at startup.

RM_XLOG_ID:
XLOG_FPI_FOR_HINT - not needed?
XLOG_FPI - not needed?

XLOG_CHECKPOINT_SHUTDOWN - must have

So how about the followings?
XLOG_CHECKPOINT_ONLINE
XLOG_NOOP
XLOG_NEXTOID
XLOG_SWITCH
XLOG_BACKUP_END
XLOG_PARAMETER_CHANGE
XLOG_RESTORE_POINT
XLOG_FPW_CHANGE
XLOG_END_OF_RECOVERY

RM_XACT_ID:
XLOG_XACT_COMMIT
XLOG_XACT_PREPARE
XLOG_XACT_ABORT
XLOG_XACT_COMMIT_PREPARED
XLOG_XACT_ABORT_PREPARED
XLOG_XACT_ASSIGNMENT
XLOG_XACT_INVALIDATIONS

Do we need all of these?

And, currenly what decides whether to emit a wal record according to
wal_level is the caller of XLogInsert. So doing this at
XLogInsert-level means that we bring the criteria of the necessity of
wal-record into xlog layer only for wal_level=none. I'm not sure it is
the right direction.

regards.

--
Kyotaro Horiguchi
NTT Open Source Software Center