Re: WAL Re-Writes

On Mon, Feb 8, 2016 at 12:08 AM, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> I think deciding it automatically without user require to configure it,
> certainly has merits, but what about some cases where user can get
> benefits by configuring themselves like the cases where we use
> PG_O_DIRECT flag for WAL (with o_direct, it will by bypass OS
> buffers and won't cause misaligned writes even for smaller chunk sizes
> like 512 bytes or so). Some googling [1] reveals that other databases
> also provides user with option to configure wal block/chunk size (as
> BLOCKSIZE), although they seem to decide chunk size based on
> disk-sector size.

Well, if you can prove that we need that flexibility, then we should
have a GUC. Where's the benchmarking data to support that conclusion?

> An additional thought, which is not necessarily related to this patch is,
> if user chooses and or we decide to write in 512 bytes sized chunks,
> which is usually a disk sector size, then can't we think of avoiding
> CRC for each record for such cases, because each WAL write in
> it-self will be atomic. While reading, if we process in wal-chunk-sized
> units, then I think it should be possible to detect end-of-wal based
> on data read.

Gosh, taking CRCs off of WAL records sounds like a terrible idea. I'm
not sure why you think that writing in sector-sized chunks would make
that any more safe, because to me it seems like it wouldn't. But even
if it does, it's hard to believe that we don't derive some reliability
from CRCs that we would lose without them.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company