Re: Copy data to DSA area

On Tue, Nov 13, 2018 at 9:45 AM Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
> On Thu, Nov 8, 2018 at 9:05 PM Thomas Munro
> <thomas(dot)munro(at)enterprisedb(dot)com> wrote:
> > * I had some ideas about some kind of "allocation rollback" interface:
> > you begin an "allocation transaction", allocate a bunch of stuff
> > (perhaps indirectly, by calling some API that makes query plans or
> > whatever and is totally unaware of this stuff). Then if there is an
> > error, whatever was allocated so far is freed in the usual cleanup
> > paths by a rollback that happens via the resource manager machinery.
> > If you commit, then the allocation becomes permanent. Then you only
> > commit stuff that you promise not to leak (perhaps stuff that has been
> > added to a very carefully managed cluster-wide plan cache). I am not
> > sure of the details, and this might be crazy...
>
> Hmm, my first thought was that you were just reinventing memory
> contexts, but it's really not quite the same thing, because you want
> the allocations to end up owned by a long-lived context when you
> succeed but a transient context when you fail. Still, if it weren't
> for the fact that the memory context interface is hostile to dynamic
> shared memory's map-this-anywhere semantics, I suspect we'd try to
> find a way to make memory contexts fit the bill, maybe by reparenting
> contexts or even individual allocations. You could imagine using the
> sorts of union algorithms that are described in
> https://en.wikipedia.org/wiki/Disjoint-set_data_structure to get very
> low asymptotic complexity here.

Yeah, I was imagining something that still works with MemoryContexts.
Interesting idea re: unions. I haven't got as far as thinking about
how you'd actually make that work. But I think we're both describing
the same general kind of semantics; you need to be able to build stuff
with automatic clean-up on non-local exit, but also keep that stuff
for longer once you decide you're ready.

Anyway, avoiding all the hard questions about new kinds of foot gun
for now, here is a stupid hack that shows a DSA area inside the
traditional fixed-address shared memory region, wrapped in a custom
(and somewhat defective for now) MemoryContext. It shows a "List"
being manipulated by standard PostgreSQL list code that is entirely
unaware that it is working in shared memory:

postgres=# call hoge_list(3);
NOTICE: Contents of list:
NOTICE: 1
NOTICE: 2
NOTICE: 3
CALL

You can call that procedure from various different backends to add and
remove integers from a List. The point is that it could just as
easily be a query plan or any other palloc-based stuff in our tree,
and the pointers work in any backend.

--
Thomas Munro
http://www.enterprisedb.com