Re: Pluggable storage

On Fri, Oct 27, 2017 at 4:06 PM, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:

> On Wed, Oct 25, 2017 at 1:59 PM, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
> wrote:
> >> Another thing to consider is that, if we could replace satisfiesfunc,
> >> it would probably break some existing code. There are multiple places
> >> in the code that compare snapshot->satisfies to
> >> HeapTupleSatisfiesHistoricMVCC and HeapTupleSatisfiesMVCC.
> >>
> >> I think the storage API should just leave snapshots alone. If a
> >> storage engine wants to call HeapTupleSatisfiesVisibility() with that
> >> snapshot, it can do so. Otherwise it can switch on
> >> snapshot->satisfies and handle each case however it likes.
> >>
> >
> > How will it switch satisfies at runtime? There are places where we
> > might know which visibility function (*MVCC , *Dirty, etc) needs to be
> > called, but I think there are other places (like heap_fetch) where it
> > is not clear and we decide based on what is stored in
> > snapshot->satisfies.
>
> An alternative storage engine needs to provide its own implementation
> of heap_fetch, and that replacement implementation can implement MVCC
> and other snapshot behavior in any way it likes.

In the current set of patches, I changed the snapshot->satisfies function
point to an enum type, Based on the snapshot visibility type, internally
the storage AM will call the corresponding visibility function.

Additional changes that are done in the patches compared to earlier
patches apart from rebase.

0004-Adding tuple visibility:

Tuple visibility API functions are reduced to 3. It still needs further
optimization.

Tuple satisfies visibility check is added to heap function that don't have
currently.

0006-tuple-insert

Move the index tuple insertion logic inside storage Am with a function
pointer, applicable to insert and updates. yet to handle the insert index
tuples
for multi insert scenario.

Removed the speculative finish API. Yet to remove abort API.

Known pending items:

1. Provide a generic new API like heap_fetch to remove heap_hot_search
and visibility functions usage.
2. Move toast table details into storage AM, Toast method depends on
storage.
and also toast flattening function needs to be replaced with some generic
functions.
3. Provide new API to get the heaptuple from index or building the index.
may be the same API like heap_fetch may satisfy this requirement also.
4. Bulk insert functionality needs a separate API to deal with all storage
AMs.
5. Provide a framework to add reloptions based on storage.
6. Needs a generic API to support rewrite the heap, (cluster command)

Regards,
Hari Babu
Fujitsu Australia