Re: Using Expanded Objects other than Arrays from plpgsql

From: Michel Pelletier <pelletier(dot)michel(at)gmail(dot)com>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com>, pgsql-hackers(at)lists(dot)postgresql(dot)org
Subject: Re: Using Expanded Objects other than Arrays from plpgsql
Date: 2024-12-25 20:25:18
Message-ID: CACxu=v+ZyWad6JpToJrUhTTYJneMsTGwGdgNCb39QaG7Tc359Q@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general pgsql-hackers

On Mon, Dec 23, 2024 at 8:26 AM Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:

> Michel Pelletier <pelletier(dot)michel(at)gmail(dot)com> writes:
> > ...
> > I agree it makes sense to have more use cases before making deeper
> > changes. I only work with expanded forms, but need to call wait() to
> > pre-expand the object to avoid multiple expansions in functions that can
> > take the same object in multiple parameters.
>
> Hmm. I agree that the wait() call is a bit ugly, but there are at
> least two things that seem worth looking into before we go so far
> as inventing type-support infrastructure:
>

2. If the problem is primarily with passing the same object to
> multiple parameters of a function, couldn't you detect and optimize
> that within the function? It would be messier than just blindly
> applying DatumGetWhatever() to each parameter position; but with a
> bit of thought I bet you could create some support logic that would
> hide most of the mess.
>

Ah that's a great idea, and it works beautifully! Now I can do an
efficient triangle count without even needing a function, see below
expand_matrix is only called once:

postgres=# select reduce_scalar(mxm(graph, graph, mask=>graph, c=>graph)) /
6 as tcount from vlivejournals ;
DEBUG: matrix_mxm
DEBUG: DatumGetMatrix
DEBUG: expand_matrix -- only called once!
DEBUG: new_matrix
DEBUG: DatumGetMatrixMaybeA
DEBUG: DatumGetMatrixMaybeAB
DEBUG: DatumGetMatrixMaybeABC
DEBUG: matrix_reduce_scalar
DEBUG: DatumGetMatrix
DEBUG: new_scalar
DEBUG: scalar_div_int32
DEBUG: new_scalar
DEBUG: scalar_out
┌─────────────────┐
│ tcount │
├─────────────────┤
│ int32:177820130 │
└─────────────────┘

What a wonderful Christmas present to me, thank you Tom!

That pretty much resolves my main issues. I'm still in an exploratory
phase but I think this gets me pretty far. Is this something that has to
wait for 18 to be released? Also do you need any further testing or code
reviewing from me?

-Michel

In response to

Browse pgsql-general by date

  From Date Subject
Next Message Adrian Klaver 2024-12-25 23:20:38 Re: Intermittent errors when fetching cursor rows on PostgreSQL 16
Previous Message Enrico Schenone 2024-12-24 22:23:04 Re: Intermittent errors when fetching cursor rows on PostgreSQL 16

Browse pgsql-hackers by date

  From Date Subject
Next Message Michael Paquier 2024-12-25 22:42:34 Re: An improvement of ProcessTwoPhaseBuffer logic
Previous Message Tom Lane 2024-12-25 19:49:50 Connection limits/permissions, slotsync workers, etc