Re: Error: dsa_area could not attach to a segment that has been freed

On 20 September 2017 at 16:55, Thomas Munro <thomas(dot)munro(at)enterprisedb(dot)com>
wrote:

> On Wed, Sep 20, 2017 at 6:14 PM, Gaddam Sai Ram
> <gaddamsairam(dot)n(at)zohocorp(dot)com> wrote:
> > Thank you very much! That fixed my issue! :)
> > I was in an assumption that pinning the area will increase its lifetime
> but
> > yeah after taking memory context into consideration its working fine!
>
> So far the success rate in confusing people who first try to make
> long-lived DSA areas and DSM segments is 100%. Basically, this is all
> designed to ensure automatic cleanup of resources in short-lived
> scopes.
>

90% ;)

I got it working with no significant issues for a long lived segment used
to store a pool of shm_mq pairs used for a sort of "connection listener"
bgworker. Though I only used DSM+ToC, not DSA. But TBH that may well be
luck, as I tend to routinely use memory contexts scoped to the operational
lifetime of a subsystem, making most problems like this just vanish without
my realising they were there in the first place. Usually.

I pretty much shamelessly cribbed from test_shm_mq for the ToC stuff
though. It's simple enough when you read it in use, but I'd be lucky to do
it without an example.

I had lots more problems with shm_mq than DSM. shm_mq is very obviously
designed for short-lived scopes, and falls down badly if you have a pool of
queues you want to re-use after the peer detaches. You have to track "in
use" flags separately to the shm_mq's own, because it doesn't clear its
stored PGPROC entries for receiver/sender on detach. Once you know neither
sender nor receiver is still attached, you can memset() the area and create
a new queue in it.

You can't just reset the queue for a new peer, and have to do quite a dance
to make sure it's safe detach from, overwrite, re-create and re-attach to.

> Good luck for your graph project. I think you're going to have to
> expend a lot of energy trying to avoid memory leaks if your DSA lives
> as long as the database cluster, since error paths won't automatically
> free any memory you allocated in it.

Yeah, that's going to be hard. You might land up having lots and lots of
little DSM segments.

> There is a kind of garbage collection for palloc'd memory and
> also for other resources like file handles, but if you're using a big
> long lived DSA area you have nothing like that.

We need, IMO, a DSA-backed heirachical MemoryContext system.

We can't use the exact MemoryContext API as-is due to the need for far
pointers though :(

--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services