From: | Simon Riggs <simon(at)2ndquadrant(dot)com> |
---|---|
To: | Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> |
Cc: | Andres Freund <andres(at)anarazel(dot)de>, Tomas Vondra <tv(at)fuzzy(dot)cz>, pgsql-committers <pgsql-committers(at)postgresql(dot)org> |
Subject: | Re: pgsql: Generational memory allocator |
Date: | 2017-11-26 18:45:33 |
Message-ID: | CANP8+jKCX0sTqR8ZexTMxr03Eu3wsuwV2qit0EB0TVorMeSXBg@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-committers |
On 27 November 2017 at 04:46, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Simon Riggs <simon(at)2ndquadrant(dot)com> writes:
>> On 26 November 2017 at 08:46, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>>> I've confirmed that the attached is sufficient to stop the valgrind crash
>>> on my machine. But as I said, I think we should be more aggressive at
>>> resizing the buffer, to reduce resize cycles. I'm inclined to start out
>>> with a buffer size of 128 or 256 or so bytes and double it when needed.
>>> Anybody have a feeling for a typical size for the "main data" part
>>> of a WAL record?
>
>> We reuse the buffer and only pfree/palloc when we need to enlarge the
>> buffer, so not sure we need to do the doubling thing and it probably
>> doesn't matter what the typical size is.
>
> Well, I'm concerned about the possibility of a lot of palloc thrashing
> if the first bunch of records it reads happen to have steadily increasing
> sizes. However, rather than doubling, it might be sufficient to set a
> robust minimum on the first allocation, ie use something along the lines
> of Max(1024, MAXALIGN(state->main_data_len)).
Agreed.
I was just researching what that number should be... and I was
thinking that we should use the maximum normal tuple size, which I
think is
TOAST_TUPLE_THRESHOLD +
SizeOfXLogRecord +
SizeOfXLogRecordDataHeaderLong
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
From | Date | Subject | |
---|---|---|---|
Next Message | Tom Lane | 2017-11-26 18:53:37 | Re: pgsql: Generational memory allocator |
Previous Message | Joe Conway | 2017-11-26 17:52:22 | pgsql: Make has_sequence_privilege support WITH GRANT OPTION |