Re: how to limit statement memory allocation

On Tue, Mar 9, 2021 at 6:03 PM Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:

> Radoslav Nedyalkov <rnedyalkov(at)gmail(dot)com> writes:
> > Occasionally we get bad queries on our db that consume a lot of memory.
> > These typically are full joins by mistake or just too large result sets.
> > My understanding is these should go to a temp file but apparently memory
> > allocation is preferred.
>
> Perhaps the accumulation is happening on the client side? libpq doesn't
> have any provision for spilling a result set to disk.
>
> If that's it, you could consider revising your application to read results
> row-at-a-time, although that might require a good deal of effort.
>
> https://www.postgresql.org/docs/current/libpq-single-row-mode.html
>
> regards, tom lane
>

Ah, I named it result set wrongly perhaps.
These are queries , part of a larger ETL function or statement which at the
end just write to a table.
The huge join is an intermediate step.

So I'm still wondering.

Thanks and regards,
Rado