RE: Disable WAL logging to speed up data loading

From: "osumi(dot)takamichi(at)fujitsu(dot)com" <osumi(dot)takamichi(at)fujitsu(dot)com>
To: 'Kyotaro Horiguchi' <horikyota(dot)ntt(at)gmail(dot)com>, "tsunakawa(dot)takay(at)fujitsu(dot)com" <tsunakawa(dot)takay(at)fujitsu(dot)com>
Cc: "sawada(dot)mshk(at)gmail(dot)com" <sawada(dot)mshk(at)gmail(dot)com>, "masao(dot)fujii(at)oss(dot)nttdata(dot)com" <masao(dot)fujii(at)oss(dot)nttdata(dot)com>, "laurenz(dot)albe(at)cybertec(dot)at" <laurenz(dot)albe(at)cybertec(dot)at>, "ashutosh(dot)bapat(dot)oss(at)gmail(dot)com" <ashutosh(dot)bapat(dot)oss(at)gmail(dot)com>, "pgsql-hackers(at)lists(dot)postgresql(dot)org" <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: RE: Disable WAL logging to speed up data loading
Date: 2020-11-27 13:34:49
Message-ID: OSBPR01MB48884BCEEBC453BA0DA8FEAFEDF80@OSBPR01MB4888.jpnprd01.prod.outlook.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Thank you, Horiguchi-San

> I haven't seen a criteria of whether a record is emitted or not for
> wal_leve=none.
>
> We're emitting only redo logs. So I think theoretically we don't need anything
> other than the shutdown checkpoint record because we don't perform
> recovery and checkpoint record is required at startup.
>
> RM_XLOG_ID:
> XLOG_FPI_FOR_HINT - not needed?
> XLOG_FPI - not needed?
>
> XLOG_CHECKPOINT_SHUTDOWN - must have
>
> So how about the followings?
> XLOG_CHECKPOINT_ONLINE
> XLOG_NOOP
> XLOG_NEXTOID
> XLOG_SWITCH
> XLOG_BACKUP_END
> XLOG_PARAMETER_CHANGE
> XLOG_RESTORE_POINT
> XLOG_FPW_CHANGE
> XLOG_END_OF_RECOVERY
>
>
> RM_XACT_ID:
> XLOG_XACT_COMMIT
> XLOG_XACT_PREPARE
> XLOG_XACT_ABORT
> XLOG_XACT_COMMIT_PREPARED
> XLOG_XACT_ABORT_PREPARED
> XLOG_XACT_ASSIGNMENT
> XLOG_XACT_INVALIDATIONS
>
> Do we need all of these?
No. Strictly speaking, you are right.
We still have types of WAL that are not necessarily needed.
For example, XLOG_END_OF_RECOVERY is not useful
because wal_level=none doesn't recover from any accidents.
Or, XLOG_CHECKPOINT_ONLINE is used when we execute CHECKPOINT
not for shutting down. Thus we could eliminate more.

> And, currenly what decides whether to emit a wal record according to
> wal_level is the caller of XLogInsert.
Yes.

> So doing this at XLogInsert-level means
> that we bring the criteria of the necessity of wal-record into xlog layer only for
> wal_level=none. I'm not sure it is the right direction.
I'm sorry. I didn't understand what "doing this" and "xlog layer" meant.
Did you mean that fixing the caller side of XLogInsert (e.g. CreateCheckPoint)
is not the right direction ? Or, fixing the function of XLogInsert is not the right direction ?

> At Fri, 27 Nov 2020 07:01:16 +0000, "tsunakawa(dot)takay(at)fujitsu(dot)com"
> <tsunakawa(dot)takay(at)fujitsu(dot)com> wrote in
> > I'm afraid "none" doesn't represent the behavior because RM_XLOG_ID and
> RM_XACT_ID WAL records, except for XLOG_FPI_*, are emitted. What's the
> good name? IIUC, "minimal" is named after the fact that the minimal
> amount of WAL necessary for crash recovery is generated. "norecovery" or
> "unrecoverable"?
Lastly, I found another name which expresses the essential characteristic of this wal_level.
How about the name of wal_level="crash_unsafe" ?
What did you think ?

Best,
Takamichi Osumi

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Fujii Masao 2020-11-27 13:39:27 Re: Feature improvement for pg_stat_statements
Previous Message Ajin Cherian 2020-11-27 13:05:04 Re: [HACKERS] logical decoding of two-phase transactions