Re: ERROR: missing chunk number 0 for toast value

On 01/06/2014 08:29 PM, Andres Freund wrote:
> On 2014-01-06 12:40:25 -0500, Robert Haas wrote:
>> On Mon, Jan 6, 2014 at 11:47 AM, Andres Freund <andres(at)2ndquadrant(dot)com> wrote:
>>> On 2014-01-06 11:08:41 -0500, Robert Haas wrote:
>>> Yea. But at least it would fail reliably instead of just under
>>> concurrency and other strange circumstances - and there'd be a safe way
>>> out. Currently there seem to be all sorts of odd behaviour possible.
>>>
>>> I simply don't have a better idea :(
>>
>> Is "forcibly detoast everything" a complete no-go? I realize there
>> are performance concerns with that approach, but I'm not sure how
>> realistic a worry it actually is.
>
> The scenario I am primarily worried about is turning a record assignment
> which previously took up to BLOCK_SIZE + slop amount of memory into
> something taking up to a gigabyte. That's a pretty damn hefty
> change.
> And there's no good way of preventing it short of using a variable for
> each actually desired column which imnsho isn't really a solution.

We could mitigate that somewhat by doing an optimization pass of the
PL/pgSQL code after compilation, and check which fields of a row
variable are never referenced, and skip the detoasting for those fields.
It would only work for named row variables, not anonymous record
variables, and you would still unnecessarily detoast fields that are
sometimes accessed but usually not. But it would avoid the detoasting in
the most egregious cases, e.g where you fetch a whole row into a
variable just to access one field.

Overall, I'm leaning towards biting the bullet and always detoasting
everything in master. Probably best to just leave the stable branches alone.

- Heikki