Re: [PATCH] Infinite loop while acquiring new TOAST Oid

Andres Freund <andres(at)anarazel(dot)de> writes:
> I think the first step to improve the situation is to not use a global oid
> counter for toasted values. One way to do that would be to use the sequence
> code to do oid assignment, but we likely can find a more efficient
> representation.

I don't particularly buy that, because the only real fix is this:

> Eventually we should do the obvious thing and make toast ids 64bit wide

and if we do that there'll be no need to worry about multiple counters.

> - to
> combat the space usage we likely should switch to representing the ids as
> variable width integers or such, otherwise the space increase would likely be
> prohibitive.

And I don't buy that either. An extra 4 bytes with a 2K payload is not
"prohibitive", it's more like "down in the noise".

I think if we switch to int8 keys and widen the global OID counter to 8
bytes (using just the low 4 bytes for other purposes), we'll have a
perfectly fine solution. There is still plenty of work to be done under
that plan, because of the need to maintain backward compatibility for
existing TOAST tables --- and maybe people would want an option to keep on
using them, for non-enormous tables? If we add additional work on top of
that, it'll just mean that it will take longer to have any solution.

>> Quick fix for this problem is limiting GetNewOidWithIndex loops to some
>> reasonable amount defined by related macro and returning error if there is
>> still no available Oid. Please check attached patch, any feedback is
>> appreciated.

> This feels like the wrong spot to tackle the issue.

Yeah, that is completely horrid. It does not remove the existing failure
mode, just changes it to have worse consequences.

regards, tom lane