Re: (resolution?) Re: [HACKERS] memory problem again

Daniel Kalchev <daniel(at)digsys(dot)bg> writes:
> I run the postmaster after unlimit, which sets limits thus:
> datasize 262144 kbytes

Oh well, so much for the small-DSIZ theory. But I still don't much
care for the other theory (sort ignores -S) because (a) I can't
reproduce any such behavior here, (b) I've been through that code
recently and didn't see anything that looked like it would cause
that behavior, and (c) if it were true then we ought to be seeing
more complaints.

I think there's probably some platform-specific issue that's causing
the misbehavior you see, but I'm at a loss to guess what it is.
Anyone have any ideas?

> BTW, this postgres is compiled with default of 64 backends - I saw recently
> note here that this may interfere with the -S option somehow....

I must've missed that --- I don't know any reason for number of backends
to interfere with -S, because -S just sets the amount of memory that
any one backend thinks it can expend for local working storage (per
sort or hash node). Can you recall where/when this discussion was?

> (another not related bug, but still on memory allocation)
> Still - this does not explain why postgres cannot allocated more than
> 76 MB (according to top) on BSD/OS (never did, actually - any previous
> version too), while a simple malloc(1 MB) loop allocates up to the
> process limit.

That does seem odd. Could it be that the shared memory segment used
by Postgres gets placed at 64M or so in your process's virtual address
space, thus preventing the malloc arena from expanding past that point?
If so, is there anything we can do to force a higher placement?

> Maybe at some time postrges tries to allocate 'larger' chunk, which
> the BSD/OS malloc does not like?

There is some code in aset.c that asks for larger and larger chunks,
but it should fall back to asking for a smaller chunk if it can't
get a bigger one. More to the point, the sort operation invoked by
SELECT DISTINCT shouldn't ask for more than (roughly) your -S setting.
So I'm still clueless where the problem is :-(

regards, tom lane