Re: Relation extension scalability

On Thu, Mar 31, 2016 at 12:59 AM, Dilip Kumar <dilipbalaut(at)gmail(dot)com> wrote:
> On Tue, Mar 29, 2016 at 10:08 AM, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
> wrote:
>> Yes, that makes sense. One more point is that if the reason for v13
>> giving better performance is extra blocks (which we believe in certain cases
>> can leak till the time Vacuum updates the FSM tree), do you think it makes
>> sense to once test by increasing lockWaiters * 20 limit to may be
>> lockWaiters * 25 or lockWaiters * 30.
>
>
> I tested COPY 10000 record, by increasing the number of blocks just to find
> out why we are not as good as V13
> with extraBlocks = Min( lockWaiters * 40, 2048) and got below results..
>
> COPY 10000
> --------------------
> Client Patch(extraBlocks = Min( lockWaiters * 40, 2048))
> -------- ---------
> 16 752
> 32 708
>
> This proves that main reason of v13 being better is its adding extra blocks
> without control.
> though v13 is better than these results, I think we can get that also by
> changing multiplier and max limit .
>
> But I think we are ok with the max size as 4MB (512 blocks) right?

Yeah, kind of. But obviously if we could make the limit smaller
without hurting performance, that would be better.

Per my note yesterday about performance degradation with parallel
COPY, I wasn't able to demonstrate that this patch gives a consistent
performance benefit on hydra - the best result I got was speeding up a
9.5 minute load to 8 minutes where linear scalability would have been
2 minutes. And I found cases where it was actually slower with the
patch. Now maybe hydra is just a crap machine, but I'm feeling
nervous.

What machines did you use to test this? Have you tested really large
data loads, like many MB or even GB of data?

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company