Re: PG14: Avoid checking output-buffer-length for every encoded byte during pg_hex_encode

Welcome.
Em seg., 16 de ago. de 2021 às 05:46, Hans Buschmann <buschmann(at)nidsa(dot)net>
escreveu:

> During some development on encoding-related parts of postgres I stumbled
> over the new length-checking-code in common/hex.c/pg_hex_encode.
>
> Differently when comparing it to all versions before the
> output-buffer-length is checked during encoding of every individual source
> byte.
>
Is always good to remove immutables from loops, if safe and secure.

> This may impose quite a regression when using pg_dump on databases with
> many/big bytea or lo columns.
>
Decode has regression too.

> Because all criteria to check buffer-length are known in advance of
> encoding (srclen and destlen) I propose doing the check only once before
> starting the while-loop.
>
Good.

>
> Please find the attached patch for pg14 and master, older versions did not
> have this behavior.
>
I think that we can take one more step here.
pg_hex_decode can be improved too.
We can remove limitations from all functions that use hex functions.
byteaout from (varlena.c) has an artificial limitation to handle only
MaxAllocSize length, but
nothing prevents her from being prepared for that limitation to be removed
one day.

> Tested on pg14-beta3, but applies also on master.
>
> PS: This is my very first patch, I am in no way an experienced C-developer
> and don't have a smoothly running development environment or experience
> yet. (originally mostly dealing with postgres on Windows).
>
It seems good to me.

Please, take a look at the new version attached.
If possible can you review the tests if there is an overflow at
pg_hex_encode and pg_hex_decode functions?

regards,
Ranier Vilela