Re: Regarding Postgres Dynamic Shared Memory (DSA)

Hi Thomas,

When you create or attach to a DSA area, a detach callback is

automatically registered on the control segment (or containing

segment, for an in-place DSA area). See the code like this in dsa.c:

/* Clean up when the control segment detaches. */

on_dsm_detach(segment, &dsa_on_dsm_detach_release_in_place,

PointerGetDatum(dsm_segment_address(segment)));

So the reference counting is automatically looked after and you don't

need to do anything. There are four possibilities: (1) you explicitly

detach from the area, (2) the ResourceManager active when you created

or attached goes out of scope, detaching you automatically (you

probably want to disable that with dsa_pin_mapping(area)), (3) your

backend exits normally and it's automatically detached (4) you crash

and the postmaster restarts the whole cluster on the basis that shared

memory could be arbitrarily corrupted.

The third possibility is our use case. We want the dsa_area to be alive through out the lifetime of the cluster and we want each backend to hold the attached dsa through out the lifetime of the backend irrespective of ResourceManager scope.

Hence we are calling dsa_pin(area) after dsa_create() and dsa_pin_mapping(area) after dsa_create() and dsa_attach().

DSA implementation in autovacuum.c helped me in understanding these concepts :)

So in my case, i could safely assume that postgres will automatically takes care of dsa detaching and reference count decrements during backend exit.

It's so nice of you, taking time and explaining and helping to solve the use cases.

Best Regards,

-Mahi
Teamwork divides the task and multiplies the success.

---- On Thu, 22 Jun 2017 17:08:01 +0530 Thomas Munro <thomas(dot)munro(at)enterprisedb(dot)com> wrote ----

On Thu, Jun 22, 2017 at 10:59 PM, Mahendranath Gurram

<mahendranath(at)zohocorp(dot)com> wrote:

> I'm implementing the In-Memory index as per your suggestion. So far it's

> good.

Great news.

> As of now, only one thing is unclear for me. How could i detach the

> dsa(dsa_detach() call) in backend (typically during backend quit).

>

> Of-course when process quits, all it's associated memory will be

> cleared/destroyed. But we have to decrement dsm reference counts as part of

> the potgres dsa framework implementation. which can not be done now.

>

> I have gone through the postgres source code. But unfortunately, i couldn't

> see anything in handling this case.

When you create or attach to a DSA area, a detach callback is

automatically registered on the control segment (or containing

segment, for an in-place DSA area). See the code like this in dsa.c:

/* Clean up when the control segment detaches. */

on_dsm_detach(segment, &dsa_on_dsm_detach_release_in_place,

PointerGetDatum(dsm_segment_address(segment)));

So the reference counting is automatically looked after and you don't

need to do anything. There are four possibilities: (1) you explicitly

detach from the area, (2) the ResourceManager active when you created

or attached goes out of scope, detaching you automatically (you

probably want to disable that with dsa_pin_mapping(area)), (3) your

backend exits normally and it's automatically detached (4) you crash

and the postmaster restarts the whole cluster on the basis that shared

memory could be arbitrarily corrupted.

Note that if you call dsa_pin(area) after creating the DSA area the

reference count cannot reach zero until you either call

dsa_unpin(area) or the cluster exits. That's the right thing to do

for a case where backends might come and go and it's possible for no

one to be attached for periods of time, but you want the DSA area to

continue to exist. I think that's probably what you need for a

DSA-backed in-memory index.

--

Thomas Munro

http://www.enterprisedb.com

--

Sent via pgsql-hackers mailing list (pgsql-hackers(at)postgresql(dot)org)

To make changes to your subscription:

http://www.postgresql.org/mailpref/pgsql-hackers