Re: [ANNOUNCE] IMCS: In Memory Columnar Store for PostgreSQL

On Thu, Jan 9, 2014 at 2:09 PM, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> On Thu, Jan 9, 2014 at 12:21 AM, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
>> On Tue, Jan 7, 2014 at 10:20 PM, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
>>> On Tue, Jan 7, 2014 at 2:46 AM, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
>>>>
>>>> Well, right now we just reopen the same object from all of the
>>>> processes, which seems to work fine and doesn't require any of this
>>>> complexity. The only problem I don't know how to solve is how to make
>>>> a segment stick around for the whole postmaster lifetime. If
>>>> duplicating the handle into the postmaster without its knowledge gets
>>>> us there, it may be worth considering, but that doesn't seem like a
>>>> good reason to rework the rest of the existing mechanism.
>>>
>>> I think one has to try this to see if it works as per the need. If it's not
>>> urgent, I can try this early next week?
>>
>> Anything we want to get into 9.4 has to be submitted by next Tuesday,
>> but I don't know that we're going to get this into 9.4.
>
> Using DuplicateHandle(), we can make segment stick for Postmaster
> lifetime. I have used below test (used dsm_demo module) to verify:
> Session - 1
> select dsm_demo_create('this message is from session-1');
> dsm_demo_create
> -----------------
> 827121111
>
> Session - 2
> -----------------
> select dsm_demo_read(827121111);
> dsm_demo_read
> ----------------------------
> this message is from session-1
> (1 row)
>
> Session-1
> \q
>
> -- till here it will work without DuplicateHandle as well
>
> Session -2
> select dsm_demo_read(827121111);
> dsm_demo_read
> ----------------------------
> this message is from session-1
> (1 row)
>
> Session -2
> \q
>
> Session -3
> select dsm_demo_read(827121111);
> dsm_demo_read
> ----------------------------
> this message is from session-1
> (1 row)
>
> -- above shows that handle stays around.
>
> Note -
> Currently I have to bypass below code in dam_attach(), as it assumes
> segment will not stay if it's removed from control file.
>
> /*
> * If we didn't find the handle we're looking for in the control
> * segment, it probably means that everyone else who had it mapped,
> * including the original creator, died before we got to this point.
> * It's up to the caller to decide what to do about that.
> */
> if (seg->control_slot == INVALID_CONTROL_SLOT)
> {
> dsm_detach(seg);
> return NULL;
> }
>
>
> Could you let me know what exactly you are expecting in patch,
> just a call to DuplicateHandle() after CreateFileMapping() or something
> else as well?

Well, I guess what I was thinking is that we could have a call
dsm_keep_segment() which would be invoked on an already-created
dsm_segment *. On Linux, that would just bump the reference count in
the control segment up by one so that it doesn't get destroyed until
postmaster shutdown. On Windows it may as well still do that for
consistency, but will also need to do this DuplicateHandle() trick.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company