Re: Disable WAL logging to speed up data loading

At Fri, 27 Nov 2020 15:07:34 +0900, Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> wrote in
> On Tue, Nov 24, 2020 at 11:19 AM osumi(dot)takamichi(at)fujitsu(dot)com
> <osumi(dot)takamichi(at)fujitsu(dot)com> wrote:
> >
> > Hi
> >
> > On Monday, Nov 23, 2020 12:17 PM Tsunakawa, Takayuki <tsunakawa(dot)takay(at)fujitsu(dot)com> wrote:
> > > PREPARE TRANSACTION is the same as COMMIT in that it persists
> > > transaction updates. A crash during wal_level = none loses both of them.
> > > So, I don't think PREPARE TRANSACTION needs special treatment.
> > Yeah, I got it. That makes sense.
> > Attached is the one I removed the special treatment.
> >
> >
> > > Does PREPARE TRANSACTION complete successfully? I remember
> > > Fujii-san said it PANICed if --enable-asserts is used in configure.
> > Yes. That assertion failure Fujii-San reported in the past
> > was solved by having rmid != RM_XACT_ID in XLogInsert()
> > to add WAL generation for transaction completes.
> > That I missed the condition was the cause but fine now.
> >
> > Plus, PREPARE TRANSACTION and COMMIT PREPARED
> > works successfully at present when no happening occurs.
> >
>
> Thank you for updating the patch.
>
> - (errmsg("WAL was generated with wal_level=minimal,
> data may be missing"),
> + (errmsg("WAL was generated with wal_level<=minimal,
> data may be missing"),
> errhint("This happens if you temporarily set
> wal_level=minimal without taking a new base backup.")));
>
> 'wal_level=minimal' in errhint also needs to be changed to 'wal_level<=minimal'?
>
> While testing the patch on some workload, I realized that
> XLOG_FPI_FOR_HINT record could still be emitted even when wal_level =
> none. IIUC that WAL record is not necessary during wal_level = none
> since the server cannot be the primary server and the server crash
> ends up requiring to restore the whole database.

It seems to be on purpose.

@@ -449,6 +449,14 @@ XLogInsert(RmgrId rmid, uint8 info)
return EndPos;
}

+ /* Issues WAL related to XLOG resources and transactions only */
+ if (wal_level == WAL_LEVEL_NONE &&
+ rmid != RM_XLOG_ID && rmid != RM_XACT_ID)
+ {
+ XLogResetInsertion();
+ return GetXLogInsertRecPtr();

What is the reason for those kinds of records to be emitted?

I think we must emit at least the shutdown checkpoint record, which
has redo-LSN that points to the record itself. I'm not sure what
other kinds of records are needed.

regards.

--
Kyotaro Horiguchi
NTT Open Source Software Center