Delay Memoize hashtable build until executor run

From: David Rowley <dgrowleyml(at)gmail(dot)com>
To: PostgreSQL Developers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Delay Memoize hashtable build until executor run
Date: 2024-01-26 06:54:16
Message-ID: CAApHDvoJktJ5XL=Kjh2a2TFr64R-7eQZV-+jcJrUwoES2GLiWg@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Currently, nodeMemoize.c builds the hashtable for the cache during
executor startup. This is not what is done in hash joins. I think we
should make the two behave the same way.

Per [1] and the corresponding discussion leading to that, making a
possibly large allocation at executor startup can lead to excessively
long EXPLAIN (not EXPLAIN ANALYZE) times. This can confuse users as
we don't mention in EXPLAIN where the time is being spent.

Although there's not yet any conclusion that Memoize is to blame,
there's another report from someone confused about where this time is
being spent in [2].

Working on the Memoize code, I originally created the hash table
during executor startup to save on having to check we have a table
each time the node is executed. However, the branch for this should
be quite predictable and I doubt it'll add any overhead that we would
notice.

The patch to do this is attached.

David

[1] https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=1e731ed12aa
[2] https://postgr.es/m/61e642df-5f48-4e4e-b4c3-58936f90ddaa@thefreecat.org

Attachment Content-Type Size
delay_memoize_hashtable_build_until_exec_run.patch text/plain 2.6 KB

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message David G. Johnston 2024-01-26 07:00:57 Re: Small fix on COPY ON_ERROR document
Previous Message Yugo NAGATA 2024-01-26 06:48:10 Re: Small fix on COPY ON_ERROR document