Re: [HACKERS] [bug-fix] Cannot select big bytea values (~600MB)

Andres Freund <andres(at)anarazel(dot)de> writes:
> On 2018-02-16 09:58:29 -0500, Tom Lane wrote:
>> Anna Akenteva <a(dot)akenteva(at)postgrespro(dot)ru> writes:
>>> [ widen StringInfoData max length to size_t ]

>> I find this scary as heck. Have you spent any time looking at the
>> side effects? There are probably hundreds of places that expect that
>> stringinfos won't get larger than 1GB.

> FWIW, I think we're going to have to bite that bullet sooner rather than
> later. I do agree it's not going to fix this issue for real, and that
> we're not going to backpatch it.

I'm not necessarily saying we shouldn't consider widening this.
I'm just saying it's going to take a good deal of cross-checking for
consequences, in particular that nothing is at risk of integer overflow
if it's presented with a very long StringInfo.

One way to limit the side effects would be to have StringInfos default to
only allowing 1GB of content as before, and you have to do something extra
at creation time to let one get bigger.

There's still the problem that the wire protocol will limit us to 2GB
(or maybe 4GB if you want to be brave^Wfoolhardy and assume clients think
the width fields are unsigned). I can't get hugely excited about moving
the goalposts only from 1GB to 2GB ...

regards, tom lane