Re: ERROR: invalid memory alloc request size <a_big_number_here>

Alvaro Herrera <alvherre(at)alvh(dot)no-ip(dot)org> writes:
> Ok. I had hoped to reproduce the problem with pristine sources, in
> order to verify that I was able to show it not appearing with my patch.
> However I have been unable to create a situation in which the problem
> appears. So I attach the patch that I came up with. Please test it.

On further reflection, this isn't gonna work :-(. The problem with the
waste-a-slot approach is that it creates an ambiguity near the offsets
wraparound point: if you are looking at an mxid with starting offset
just under 2^32, and you see the next mxid has start offset 1, did your
mxid include the xid in offset 0 or not?

We could possibly fix that by decreeing that wrapped-around mxids never
use slot 0, but it seems pretty darn messy: that would affect fetching
and storing loops as well as the code that allocates space.

I'm currently experimenting with an alternative approach, which leaves
the nextOffset arithmetic as it was and instead special-cases the zero
offset case this way:

* 2. The next multixact may still be in process of being filled in:
* that is, another process may have done GetNewMultiXactId but not yet
* written the offset entry for that ID. In that scenario, it is
* guaranteed that the offset entry for that multixact exists (because
* GetNewMultiXactId won't release MultiXactGenLock until it does)
* but contains zero (because we are careful to pre-zero offset pages).
* So, if we read zero as the next multixact offset, we have to treat
* it with suspicion. It could be valid, though. We deal with this
* ambiguity by requiring processes that are creating a multixact with
* starting offset zero to set the creatingOffsetZero flag in the shared
* data structure; we sleep until we see that cleared before trusting
* a zero offset. This is all pretty messy, but the mess occurs only
* in infrequent corner cases, so it seems better than holding the
* MultiXactGenLock for a long time on every multixact creation.

creatingOffsetZero will be a bool that gets set before releasing
MultiXactGenLock if offset 0 is being returned, and then we clear it
after updating the slru data structures if we had starting offset 0.

Thoughts?

regards, tom lane