Re: Speed up Clog Access by increasing CLOG buffers

On 10/31/2016 02:24 PM, Tomas Vondra wrote:
> On 10/31/2016 05:01 AM, Jim Nasby wrote:
>> On 10/30/16 1:32 PM, Tomas Vondra wrote:
>>>
>>> Now, maybe this has nothing to do with PostgreSQL itself, but maybe it's
>>> some sort of CPU / OS scheduling artifact. For example, the system has
>>> 36 physical cores, 72 virtual ones (thanks to HT). I find it strange
>>> that the "good" client counts are always multiples of 72, while the
>>> "bad" ones fall in between.
>>>
>>> 72 = 72 * 1 (good)
>>> 108 = 72 * 1.5 (bad)
>>> 144 = 72 * 2 (good)
>>> 180 = 72 * 2.5 (bad)
>>> 216 = 72 * 3 (good)
>>> 252 = 72 * 3.5 (bad)
>>> 288 = 72 * 4 (good)
>>>
>>> So maybe this has something to do with how OS schedules the tasks, or
>>> maybe some internal heuristics in the CPU, or something like that.
>>
>> It might be enlightening to run a series of tests that are 72*.1 or *.2
>> apart (say, 72, 79, 86, ..., 137, 144).
>
> Yeah, I've started a benchmark with client a step of 6 clients
>
> 36 42 48 54 60 66 72 78 ... 252 258 264 270 276 282 288
>
> instead of just
>
> 36 72 108 144 180 216 252 288
>
> which did a test every 36 clients. To compensate for the 6x longer runs,
> I'm only running tests for "group-update" and "master", so I should have
> the results in ~36h.
>

So I've been curious and looked at results of the runs executed so far,
and for the group_update patch it looks like this:

clients tps
-----------------
36 117663
42 139791
48 129331
54 144970
60 124174
66 137227
72 146064
78 100267
84 141538
90 96607
96 139290
102 93976
108 136421
114 91848
120 133563
126 89801
132 132607
138 87912
144 129688
150 87221
156 129608
162 85403
168 130193
174 83863
180 129337
186 81968
192 128571
198 82053
204 128020
210 80768
216 124153
222 80493
228 125503
234 78950
240 125670
246 78418
252 123532
258 77623
264 124366
270 76726
276 119054
282 76960
288 121819

So, similar saw-like behavior, perfectly periodic. But the really
strange thing is the peaks/valleys don't match those observed before!

That is, during the previous runs, 72, 144, 216 and 288 were "good"
while 108, 180 and 252 were "bad". But in those runs, all those client
counts are "good" ...

Honestly, I have no idea what to think about this ...

regards

--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services