One more problem with JIT

From: Konstantin Knizhnik <knizhnik(at)garret(dot)ru>
To: pgsql-hackers(at)lists(dot)postgresql(dot)org
Subject: One more problem with JIT
Date: 2023-08-01 16:00:10
Message-ID: b0900d21-594a-57a7-d345-9f8e3987e068@garret.ru
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi hackers,

I am using pg_embedding extension for Postgres which implements HNSW
index (some kind of ANN search).
Search query looks something like this:

    SELECT _id FROM documents ORDER BY openai <=> ARRAY[0.024466066,
-0.011110042, -0.0012917554,... , -0.008700027] LIMIT 1;

I do not put all query here because vector contains 1536 elements.

The query plans is straightforward:

  Limit  (cost=4007500.00..4007502.06 rows=1 width=33)
   ->  Index Scan using pgembedding_hnsw_idx on documents
(cost=4007500.00..6064544.00 rows=1000000 width=33)
         Order By: (openai <=>
'{0.024466066,-0.011110042,-0.0012917554,-0.025349319,-0.017500998,0.015570462,-0.015835438,-0.0067442562,0.0015977388,-0.034...
 Planning Time: 1.276 ms
  JIT:
   Functions: 5
   Options: Inlining true, Optimization true, Expressions true,
Deforming true
   Timing: Generation 0.289 ms, Inlining 3.752 ms, Optimization 34.502
ms, Emission 19.078 ms, Total 57.621 ms
 Execution Time: 70.850 ms

without JIT:

 Planning Time: 0.472 ms
 Execution Time: 13.300 ms

The problem is that JIT slowdown execution of this query 4x times: with
default setting it is 70 msec, with jit=off - 13 msec.
I wonder why do we ever try to use JIT for index scan, especially for
user defined index where no function can be inlined?
Why not to restrict JIT just to seqscan plan? But the way in this
particular case enable_seqscan=off...
May be it is naive suggestions, but 5x slowdown doesn't seems like right
behavior in any case.

Profile of query execution is the following:

-   78.52%     0.00%  postgres  postgres              [.]
standard_ExecutorRun
   - 78.52% standard_ExecutorRun
      - 55.38% 0000561dc27c89dc
         - 55.38% 0000561dc27c89dc
              0x7fc3a58dd731
            - llvm_get_function
               + 37.66% LLVMRunPassManager
               + 14.99% LLVMOrcLLJITLookup
               + 0.91% llvm_inline
                 0.90% llvm::PassManagerBuilder::populateModulePassManager
               + 0.79% LLVMRunFunctionPassManager
      - 22.79% 0000561dc27c89dc
         - ExecScan
            - 22.76% 0000561dc27c89dc
                 index_getnext_tid
               - hnsw_gettuple
                  - 22.76% hnsw_search
                     - 22.72% searchKnn

                        + 22.29% searchBaseLayer

Version of Postgres: Ubuntu 15.3-1.pgdg22.04+1)

Browse pgsql-hackers by date

  From Date Subject
Next Message Bharath Rupireddy 2023-08-01 16:30:00 Re: New Table Access Methods for Multi and Single Inserts
Previous Message Nathan Bossart 2023-08-01 16:00:06 Re: add timing information to pg_upgrade