Re: [PATCH] Infinite loop while acquiring new TOAST Oid

Hi hackers!

I've prepared a patch with a 64-bit TOAST Value ID. It is a kind of
prototype
and needs some further work, but it is already working and ready to play
with.

I've introduced 64-bit value ID field in varatt_external, but to keep it
compatible
with older bases 64-bit value is stored as a structure of two 32-bit fields
and value
ID moved to be the last in the varatt_external structure. Also, I've
introduced
different ID assignment function - instead of using global OID assignment
function
I use individual counters for each TOAST table, automatically cached and
after
server restart when new value is inserted into TOAST table maximum used
value
is searched and used to assign the next one.

>Andres Freund:
>I think we'll need to do something about the width of varatt_external to
make
>the conversion to 64bit toast oids viable - and if we do, I don't think
>there's a decent argument for staying with 4 byte toast OIDs. I think the
>varatt_external equivalent would end up being smaller in just about all
cases.
>And as you said earlier, the increased overhead inside the toast table /
index
>is not relevant compared to the size of toasted datums.

There is some small overhead due to indexing 64-bit values. Also, for
smaller
tables we can use 32-bit ID instead of 64-bit, but then we would have a
problem
when we reach the limit of 2^32.

Pg_upgrade seems to be not a very difficult case if we go keeping old TOAST
tables using 32-bit values,

Please have a look. I'd be grateful for some further directions.

GIT branch with this feature, rebased onto current master:
https://github.com/postgrespro/postgres/tree/toast_64bit

On Wed, Dec 7, 2022 at 12:00 AM Nikita Malakhov <hukutoc(at)gmail(dot)com> wrote:

> Hi hackers!
>
> Here's some update on the subject - I've made a branch based on master from
> 28/11 and introduced 64-bit TOAST value ID there. It is not complete now
> but
> is already working and has some features:
> - extended structure for TOAST pointer (varatt_long_external) with 64-bit
> TOAST value ID;
> - individual ID counters for each TOAST table, being cached for faster
> access
> and lookup of maximum TOAST ID in use after server restart.
> https://github.com/postgrespro/postgres/tree/toast_64bit
>
> --
> Regards,
> Nikita Malakhov
> Postgres Professional
> https://postgrespro.ru/
>

--
Regards,
Nikita Malakhov
Postgres Professional
https://postgrespro.ru/