Re: Relation extension scalability

On Sun, Mar 27, 2016 at 8:00 AM, Dilip Kumar <dilipbalaut(at)gmail(dot)com> wrote:
> Relation Size
> -----------------
> INSERT : 16000 transaction from 32 Client
>
> Base v13 v14_1
> --------- --------- --------
> TPS 37 255 112
> Rel Size 17GB 17GB 18GB
>
> COPY: 32000 transaction from 32 client
> Base v13 v14_1
> --------- --------- ---------
> TPS 121 823 427
> Rel Size 11GB 11GB 11GB
>
>
> Script are attached in the mail
> ----------------------------------------=
> test_size_ins.sh --> run insert test and calculate relation size.
> test_size_copy --> run copy test and relation size
> copy_script -> copy pg_bench script used by test_size_copy.sh
> insert_script --> insert pg_bench script used by test_size_ins.sh
> multi_extend_v14_poc_v1.patch --> modified patch of v14.
>
> I also tried modifying v14 from different different angle.
>
> One is like below-->
> -------------------------
> In AddExtraBlock
> {
> I add page to FSM one by one like v13 does.
> then update the full FSM tree up till root
> }

Not following this. Did you attach this version?

> Results:
> ----------
> 1. With this performance is little less than v14 but the problem of extra
> relation size is solved.
> 2. With this we can conclude that extra size of relation in v14 is because
> some while extending the pages, its not immediately available and at end
> some of the pages are left unused.

I agree with that conclusion. I'm not quite sure where that leaves
us, though. We can go back to v13, but why isn't that producing extra
pages? It seems like it should: whenever a bulk extend rolls over to
a new FSM page, concurrent backends will search either the old or the
new one but not both.

Maybe we could do this - not sure if it's what you were suggesting above:

1. Add the pages one at a time, and do RecordPageWithFreeSpace after each one.
2. After inserting them all, go back and update the upper levels of
the FSM tree up the root.

Another idea is:

If ConditionalLockRelationForExtension fails to get the lock
immediately, search the last *two* pages of the FSM for a free page.

Just brainstorming here.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company