Re: spinlocks on powerpc

2012/1/3 Manabu Ori <manabu(dot)ori(at)gmail(dot)com>
> With current git head:
> (Note that Power750 generates executables with hint bit from
> current git head.)
>
> pgbench -c 1 -j 1 -S -T 300 tps = 11436.679064 (including ...
> pgbench -c 2 -j 1 -S -T 300 tps = 21922.031158 (including ...
> pgbench -c 4 -j 2 -S -T 300 tps = 41801.573397 (including ...
> pgbench -c 8 -j 4 -S -T 300 tps = 76581.573285 (including ...
> pgbench -c 16 -j 8 -S -T 300 tps = 154154.380180 (including ...
> pgbench -c 32 -j 16 -S -T 300 tps = 280654.751280 (including ...
> pgbench -c 64 -j 32 -S -T 300 tps = 263800.861178 (including ...
> pgbench -c 96 -j 48 -S -T 300 tps = 212199.887237 (including ...
> pgbench -c 128 -j 64 -S -T 300 tps = 170627.652759 (including ...
>
> With current git head with TAS_SPIN patch:
> (executables have hint bit and TAS_SPIN change)
>
> pgbench -c 1 -j 1 -S -T 300 tps = 11323.652326 (including ...
> pgbench -c 2 -j 1 -S -T 300 tps = 22123.674189 (including ...
> pgbench -c 4 -j 2 -S -T 300 tps = 43616.374433 (including ...
> pgbench -c 8 -j 4 -S -T 300 tps = 86350.153176 (including ...
> pgbench -c 16 -j 8 -S -T 300 tps = 166122.891575 (including ...
> pgbench -c 32 -j 16 -S -T 300 tps = 269379.747507 (including ...
> pgbench -c 64 -j 32 -S -T 300 tps = 361657.417319 (including ...
> pgbench -c 96 -j 48 -S -T 300 tps = 333483.557846 (including ...
> pgbench -c 128 -j 64 -S -T 300 tps = 299554.099510 (including ...
>
> I'm running another cases includes LWARX stuff etc and send the
> results later.

I got additional results.

I've been using current git head of commit 631beeac35 in all the
tests, so that git head already have hint bit in LWARX and
SYNC/ISYNC are replaced with LWSYNC.

Note that in my graph legend "+ TAS_SPIN path" means including
TAS_SPIN patch and "- hint bit" means undefining
USE_PPC_LWARX_MUTEX_HINT.

With current git head without hint bit and with TAS_SPIN patch:
(executables have only TAS_SPIN change)

pgbench -c 1 -j 1 -S -T 300 tps = 11516.896929 (including ...
pgbench -c 2 -j 1 -S -T 300 tps = 21992.169336 (including ...
pgbench -c 4 -j 2 -S -T 300 tps = 40533.578654 (including ...
pgbench -c 8 -j 4 -S -T 300 tps = 84993.346937 (including ...
pgbench -c 16 -j 8 -S -T 300 tps = 157629.680382 (including ...
pgbench -c 32 -j 16 -S -T 300 tps = 273234.960592 (including ...
pgbench -c 64 -j 32 -S -T 300 tps = 348557.006484 (including ...
pgbench -c 96 -j 48 -S -T 300 tps = 326664.510722 (including ...
pgbench -c 128 -j 64 -S -T 300 tps = 299446.708323 (including ...

With current git head without hint bit:
(executables have neither hint bit nor TAS_SPIN change)

pgbench -c 1 -j 1 -S -T 300 tps = 11531.003861 (including ...
pgbench -c 2 -j 1 -S -T 300 tps = 22510.372046 (including ...
pgbench -c 4 -j 2 -S -T 300 tps = 40116.804994 (including ...
pgbench -c 8 -j 4 -S -T 300 tps = 83516.160169 (including ...
pgbench -c 16 -j 8 -S -T 300 tps = 160835.396856 (including ...
pgbench -c 32 -j 16 -S -T 300 tps = 228697.183575 (including ...
pgbench -c 64 -j 32 -S -T 300 tps = 177939.580160 (including ...
pgbench -c 96 -j 48 -S -T 300 tps = 160282.234748 (including ...
pgbench -c 128 -j 64 -S -T 300 tps = 143993.382895 (including ...

We can see LWARX hint bit and TAS_SPIN patch have meaningful
benefit on Power750, and having both get best performance.

Substituing LWSYNC for SYNC in S_UNLOCK:
(executables have hint bit)

pgbench -c 1 -j 1 -S -T 300 tps = 11387.633820 (including ...
pgbench -c 2 -j 1 -S -T 300 tps = 22270.036504 (including ...
pgbench -c 4 -j 2 -S -T 300 tps = 41268.705101 (including ...
pgbench -c 8 -j 4 -S -T 300 tps = 83487.940761 (including ...
pgbench -c 16 -j 8 -S -T 300 tps = 142243.077283 (including ...
pgbench -c 32 -j 16 -S -T 300 tps = 263584.870238 (including ...
pgbench -c 64 -j 32 -S -T 300 tps = 204520.685955 (including ...
pgbench -c 96 -j 48 -S -T 300 tps = 174361.369468 (including ...
pgbench -c 128 -j 64 -S -T 300 tps = 149120.192708 (including ...

Substituing LWSYNC for both SYNC and ISYNC:
(executables have hint bit)

pgbench -c 1 -j 1 -S -T 300 tps = 11470.622346 (including ...
pgbench -c 2 -j 1 -S -T 300 tps = 22403.955716 (including ...
pgbench -c 4 -j 2 -S -T 300 tps = 43919.047680 (including ...
pgbench -c 8 -j 4 -S -T 300 tps = 80186.836149 (including ...
pgbench -c 16 -j 8 -S -T 300 tps = 146265.129834 (including ...
pgbench -c 32 -j 16 -S -T 300 tps = 259638.888656 (including ...
pgbench -c 64 -j 32 -S -T 300 tps = 202791.211830 (including ...
pgbench -c 96 -j 48 -S -T 300 tps = 173524.291680 (including ...
pgbench -c 128 -j 64 -S -T 300 tps = 148385.991706 (including ...

In my test, replacing both SYNC and ISYNC with LWSYNC is
effective even when tye system is overloaded.

Test environment:
Power 750 (32 physical cores, virtually 128 cores using SMT4)
mem: 256GB
OS: RHEL6.1 kernel 2.6.32-131.0.15.el6.ppc64
gcc version 4.4.5 20110214 (Red Hat 4.4.5-6)
PostgreSQL git head (631beeac3598a73dee2c2afa38fa2e734148031b)

Regards,
Manabu Ori