From: | Kyotaro Horiguchi <horikyota(dot)ntt(at)gmail(dot)com> |
---|---|
To: | noah(at)leadboat(dot)com |
Cc: | pgsql-hackers(at)postgresql(dot)org, 9erthalion6(at)gmail(dot)com, andrew(dot)dunstan(at)2ndquadrant(dot)com, hlinnaka(at)iki(dot)fi, robertmhaas(at)gmail(dot)com, michael(at)paquier(dot)xyz |
Subject: | Re: [HACKERS] WAL logging problem in 9.4.3? |
Date: | 2019-08-28 06:42:10 |
Message-ID: | 20190828.154210.204505676.horikyota.ntt@gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
Hello, Noah.
At Tue, 27 Aug 2019 15:49:32 +0900 (Tokyo Standard Time), Kyotaro Horiguchi <horikyota(dot)ntt(at)gmail(dot)com> wrote in <20190827(dot)154932(dot)250364935(dot)horikyota(dot)ntt(at)gmail(dot)com>
> I'm not sure whether the knob shows apparent performance gain and
> whether we can offer the criteria to identify the proper
> value. But I'll add this feature with a GUC
> effective_io_block_size defaults to 64kB as the threshold in the
> next version. (The name and default value are arguable, of course.)
This is a new version of the patch based on the discussion.
The differences from v19 are the follows.
- Removed the new stuff in two-phase.c.
The action on PREPARE TRANSACTION is now taken in
PrepareTransaction(). Instead of storing pending syncs in
two-phase files, the function immediately syncs all files that
can survive the transaction end. (twophase.c, xact.c)
- Separate pendingSyncs from pendingDeletes.
pendingSyncs gets handled differently from pendingDeletes so it
is separated.
- Let smgrDoPendingSyncs() to avoid performing fsync on
to-be-deleted files.
In previous versions the function syncs all recorded files even
if it is being deleted. Since we use WAL-logging as the
alternative of fsync now, performance gets more significance
g than before. Thus this version avoids uesless fsyncs.
- Use log_newpage instead of fsync for small tables.
As in the discussion up-thread, I think I understand how
WAL-logging works better than fsync. smgrDoPendingSync issues
log_newpage for all blocks in the table smaller than the GUC
variable "effective_io_block_size". I found
log_newpage_range() that does exact what is needed here but it
requires Relation that is no available there. I removed an
assertion in CreateFakeRelcacheEntry so that it works while
non-recovery mode.
- Rebased and fixed some bugs.
I'm trying to measure performance difference on WAL/fsync.
By the way, smgrDoPendingDelete is called from CommitTransaction
and AbortTransaction directlry, and from AbortSubTransaction via
AtSubAbort_smgr(), which calls only smgrDoPendingDeletes() and is
called only from AbortSubTransaction. I think these should be
unified either way. Any opinions?
CommitTransaction()
+ msgrDoPendingDelete()
AbortTransaction()
+ msgrDoPendingDelete()
AbortSubTransactoin()
AtSubAbort_smgr()
+ msgrDoPendingDelete()
# Looking around, the prefixes AtEOact/PreCommit/AtAbort don't
# seem to be used keeping a principle.
regards.
--
Kyotaro Horiguchi
NTT Open Source Software Center
Attachment | Content-Type | Size |
---|---|---|
v20-0001-TAP-test-for-copy-truncation-optimization.patch | text/x-patch | 11.3 KB |
v20-0002-Fix-WAL-skipping-feature.patch | text/x-patch | 50.0 KB |
v20-0003-Documentation-for-effective_io_block_size.patch | text/x-patch | 1.6 KB |
v20-0004-Additional-test-for-new-GUC-setting.patch | text/x-patch | 1.8 KB |
From | Date | Subject | |
---|---|---|---|
Next Message | Moon, Insung | 2019-08-28 06:43:02 | Performance improvement of WAL writing? |
Previous Message | Fabien COELHO | 2019-08-28 05:36:45 | Re: pgbench - implement strict TPC-B benchmark |