Re: Size estimation of postgres core files

On 2019-02-15 13:01:50 -0600, Jeremy Finzel wrote:
> It doesn't write out all of RAM, only the amount in use by the
> particular backend that crashed (plus all the shared segments attached
> by that backend, including the main shared_buffers, unless you disable
> that as previously mentioned).

> Based on the Alvaro's response, I thought it is reasonably possible that that
> *could* include nearly all of RAM, because that was my original question.  If
> shared buffers is say 50G and my OS has 1T, shared buffers is a small portion
> of that.  But really my question is what should we reasonably assume is
> possible

The size of the core dump will be roughly the same as the VM used by the
process - so that will be the initial size of the process plus shared
buffers plus a (usually small) multiple of work_mem or
maintenance_work_mem plus whatever memory the process allocates.

The big unknown is that "(usually small) multiple of work_mem". I've
seen a process use 8 times work_mem for a moderately complex query, so
depending on what you do it might be worse.

The extra memory allocated by processes is usually small (after all, if
some datastructure were expected to be potentially large it would
probably be limited by work_mem), but if there is a bug (and that's what
you are looking for) it might really grow without bounds.

If you know some upper bound for a reasonable size of your processes you
could set the address space limit - not only will this limit the core
dump size, but it will also prevent a single process from consuming all
RAM and triggering the OOM killer.

You probably don't want to limit the core dump size directly (another
process limit you can set) as that will result in a truncated (and
possibly useless) core dump. For similar reasons I'm not convinced that
omitting the shared memory is a good idea.

hp

--
_ | Peter J. Holzer | we build much bigger, better disasters now
|_|_) | | because we have much more sophisticated
| | | hjp(at)hjp(dot)at | management tools.
__/ | http://www.hjp.at/ | -- Ross Anderson <https://www.edge.org/>