Re: POC and rebased patch for CSN based snapshots

On 2020/06/19 14:54, Fujii Masao wrote:
>
>
> On 2020/06/19 13:36, movead(dot)li(at)highgo(dot)ca wrote:
>>
>>  >You mean that the last generated CSN needs to be WAL-logged because any smaller
>>> CSN than the last one should not be reused after crash recovery. Right?
>> Yes that's it.
>>
>>> If right, that WAL-logging seems not necessary because CSN mechanism assumes
>>> CSN is increased monotonically. IOW, even without that WAL-logging, CSN afer
>>> crash recovery must be larger than that before. No?
>>
>> CSN collected based on time of  system in this patch, but time is not reliable all the
>> time. And it designed for Global CSN(for sharding) where it may rely on CSN from
>> other node , which generated from other machine.
>>
>> So monotonically is not reliable and it need to keep it's largest CSN in wal in case
>> of crash.
>
> Thanks for the explanaion! Understood.

I have another question about this patch;

When checking each tuple visibility, we always have to get the CSN
corresponding to XMIN or XMAX from CSN SLRU. In the past discussion,
there was the suggestion that CSN should be stored in the tuple header
or somewhere (like hint bit) to avoid the overhead by very frequehntly
lookup for CSN SLRU. I'm not sure the conclusion of this discussion.
But this patch doesn't seem to adopt that idea. So did you confirm that
such performance overhead by lookup for CSN SLRU is negligible?

Of course I know that idea has big issue, i.e., there is no enough space
to store CSN in a tuple header if CSN is 64 bits. If CSN is 32 bits, we may
be able to replace XMIN or XMAX with CSN corresponding to them. But
it means that we have to struggle with one more wraparound issue
(CSN wraparound issue). So it's not easy to adopt that idea...

Sorry if this was already discussed and concluded...

Regards,

--
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION