Re: JIT compiling - v4.0

On 2017-10-04 11:56:47 +0300, Ants Aasma wrote:
> On Wed, Oct 4, 2017 at 9:48 AM, Andres Freund <andres(at)anarazel(dot)de> wrote:
> > Here's an updated version of the patchset. There's some substantial
> > changes here, but it's still very obviously very far from committable as
> > a whole. There's some helper commmits that are simple and independent
> > enough to be committable earlier on.
>
> Looks pretty impressive already.

Thanks!

> I wanted to take it for a spin, but got errors about the following
> symbols being missing:
>
> LLVMOrcUnregisterPerf
> LLVMOrcRegisterGDB
> LLVMOrcRegisterPerf
> LLVMOrcGetSymbolAddressIn
> LLVMLinkModules2Needed
>
> As far as I can tell these are not in mainline LLVM. Is there a branch
> or patchset of LLVM available somewhere that I need to use this?

Oops, I'd forgotten about the modifications. Sorry. I've attached them
here. The GDB and Perf stuff should now be an optional dependency,
too. The required changes are fairly small, so they hopefully shouldn't
be too hard to upstream.

Please check the git tree for a rebased version of the pg patches, with
a bunch bugfixes (oops, some last minute "cleanups") and performance
fixes.

Here's some numbers for a a TPC-H scale 5 run. Obviously the Q01 numbers
are pretty nice in partcular. But it's also visible that the shorter
query can loose, which is largely due to the JIT overhead - that can be
ameliorated to some degree, but JITing obviously isn't always going to
be a win.

It's pretty impressive that in q01, even after all of this, expression
evaluation *still* is 35% of the total time (25% in the aggregate
transition function). That's partially just because the query does
primarily aggregation, but also because the generated code can stand a
good chunk of improvements.

master q01 min: 14146.498 dev min: 11479.05 [diff -23.24] dev-jit min: 8659.961 [diff -63.36] dev-jit-deform min: 7279.395 [diff -94.34] dev-jit-deform-inline min: 6997.956 [diff -102.15]
master q02 min: 1234.229 dev min: 1208.102 [diff -2.16] dev-jit min: 1292.983 [diff +4.54] dev-jit-deform min: 1580.505 [diff +21.91] dev-jit-deform-inline min: 1809.046 [diff +31.77]
master q03 min: 6220.814 dev min: 5424.107 [diff -14.69] dev-jit min: 5175.125 [diff -20.21] dev-jit-deform min: 4257.368 [diff -46.12] dev-jit-deform-inline min: 4218.115 [diff -47.48]
master q04 min: 947.476 dev min: 970.608 [diff +2.38] dev-jit min: 969.944 [diff +2.32] dev-jit-deform min: 999.006 [diff +5.16] dev-jit-deform-inline min: 1033.78 [diff +8.35]
master q05 min: 4729.9 dev min: 4059.665 [diff -16.51] dev-jit min: 4182.941 [diff -13.08] dev-jit-deform min: 4147.493 [diff -14.04] dev-jit-deform-inline min: 4284.473 [diff -10.40]
master q06 min: 1603.708 dev min: 1592.107 [diff -0.73] dev-jit min: 1556.216 [diff -3.05] dev-jit-deform min: 1516.078 [diff -5.78] dev-jit-deform-inline min: 1579.839 [diff -1.51]
master q07 min: 4549.738 dev min: 4331.565 [diff -5.04] dev-jit min: 4475.654 [diff -1.66] dev-jit-deform min: 4645.773 [diff +2.07] dev-jit-deform-inline min: 4885.781 [diff +6.88]
master q08 min: 1394.428 dev min: 1350.363 [diff -3.26] dev-jit min: 1434.366 [diff +2.78] dev-jit-deform min: 1716.65 [diff +18.77] dev-jit-deform-inline min: 1938.152 [diff +28.05]
master q09 min: 5958.198 dev min: 5700.329 [diff -4.52] dev-jit min: 5491.683 [diff -8.49] dev-jit-deform min: 5582.431 [diff -6.73] dev-jit-deform-inline min: 5797.475 [diff -2.77]
master q10 min: 5228.69 dev min: 4475.154 [diff -16.84] dev-jit min: 4269.365 [diff -22.47] dev-jit-deform min: 3767.888 [diff -38.77] dev-jit-deform-inline min: 3962.084 [diff -31.97]
master q11 min: 281.201 dev min: 280.132 [diff -0.38] dev-jit min: 351.85 [diff +20.08] dev-jit-deform min: 455.885 [diff +38.32] dev-jit-deform-inline min: 532.093 [diff +47.15]
master q12 min: 4289.268 dev min: 4082.359 [diff -5.07] dev-jit min: 4007.199 [diff -7.04] dev-jit-deform min: 3752.396 [diff -14.31] dev-jit-deform-inline min: 3916.653 [diff -9.51]
master q13 min: 7110.545 dev min: 6898.576 [diff -3.07] dev-jit min: 6579.554 [diff -8.07] dev-jit-deform min: 6304.15 [diff -12.79] dev-jit-deform-inline min: 6135.952 [diff -15.88]
master q14 min: 678.024 dev min: 650.943 [diff -4.16] dev-jit min: 682.387 [diff +0.64] dev-jit-deform min: 746.354 [diff +9.16] dev-jit-deform-inline min: 878.437 [diff +22.81]
master q15 min: 1641.897 dev min: 1650.57 [diff +0.53] dev-jit min: 1661.591 [diff +1.19] dev-jit-deform min: 1821.02 [diff +9.84] dev-jit-deform-inline min: 1863.304 [diff +11.88]
master q16 min: 1890.246 dev min: 1819.423 [diff -3.89] dev-jit min: 1838.079 [diff -2.84] dev-jit-deform min: 1962.274 [diff +3.67] dev-jit-deform-inline min: 2096.154 [diff +9.82]
master q17 min: 502.605 dev min: 462.881 [diff -8.58] dev-jit min: 495.648 [diff -1.40] dev-jit-deform min: 537.666 [diff +6.52] dev-jit-deform-inline min: 613.144 [diff +18.03]
master q18 min: 12863.972 dev min: 11257.57 [diff -14.27] dev-jit min: 10847.61 [diff -18.59] dev-jit-deform min: 10119.769 [diff -27.12] dev-jit-deform-inline min: 10103.051 [diff -27.33]
master q19 min: 281.991 dev min: 264.191 [diff -6.74] dev-jit min: 331.102 [diff +14.83] dev-jit-deform min: 373.759 [diff +24.55] dev-jit-deform-inline min: 531.07 [diff +46.90]
master q20 min: 541.154 dev min: 511.372 [diff -5.82] dev-jit min: 565.378 [diff +4.28] dev-jit-deform min: 662.926 [diff +18.37] dev-jit-deform-inline min: 805.835 [diff +32.85]
master q22 min: 678.266 dev min: 656.643 [diff -3.29] dev-jit min: 676.886 [diff -0.20] dev-jit-deform min: 735.058 [diff +7.73] dev-jit-deform-inline min: 943.013 [diff +28.07]

master total min: 76772.848 dev min: 69125.71 [diff -11.06] dev-jit min: 65545.522 [diff -17.13] dev-jit-deform min: 62963.844 [diff -21.93] dev-jit-deform-inline min: 64925.407 [diff -18.25]

Greetings,

Andres Freund