Re: AW: beta testing version

On Wed, Dec 06, 2000 at 11:15:26AM -0500, Tom Lane wrote:
> Zeugswetter Andreas SB <ZeugswetterA(at)Wien(dot)Spardat(dot)at> writes:
> > Yes, but there would need to be a way to verify the last page or
> > record from txlog when running on crap hardware.
> How exactly *do* we determine where the end of the valid log data is,
> anyway?

I don't know how pgsql does it, but the only safe way I know of is to
include an "end" marker after each record. When writing to the log,
append the records after the last end marker, ending with another end
marker, and fdatasync the log. Then overwrite the previous end marker
to indicate it's not the end of the log any more and fdatasync again.

To ensure that it is written atomically, the end marker must not cross a
hardware sector boundary (typically 512 bytes). This can be trivially
guaranteed by making the marker a single byte.

Any other way I've seen discussed (here and elsewhere) either
- Requires atomic multi-sector writes, which are possible only if all
the sectors are sequential on disk, the kernel issues one large write
for all of them, and you don't powerfail in the middle of the write.
- Assume that a CRC is a guarantee. A CRC would be a good addition to
help ensure the data wasn't broken by flakey drive firmware, but
doesn't guarantee consistency.

--
Bruce Guenter <bruceg(at)em(dot)ca> http://em.ca/~bruceg/