Re: scalability bottlenecks with (many) partitions (and more)

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Andrew Dunstan <andrew(at)dunslane(dot)net>
Cc: Andres Freund <andres(at)anarazel(dot)de>, Tomas Vondra <tomas(at)vondra(dot)me>, Jakub Wartak <jakub(dot)wartak(at)enterprisedb(dot)com>, Robert Haas <robertmhaas(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: scalability bottlenecks with (many) partitions (and more)
Date: 2025-03-04 22:49:57
Message-ID: 201792.1741128597@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
>> I think I found a logic bug. Testing.

Oh! I bet you are looking at this 18-to-19 diff:

@@ -416,7 +416,8 @@ sub check_install_is_complete
{
$tmp_loc = "$tmp_loc/$install_dir";
$bindir = "$tmp_loc/bin";
- $libdir = "$tmp_loc/lib/postgresql";
+ $libdir = "$tmp_loc/lib";
+ $libdir .= '/postgresql' unless $libdir =~ /postgres|pgsql/;
return (-d $bindir && -d $libdir);
}
elsif (-e "$build_dir/src/Makefile.global") # i.e. not msvc
@@ -427,7 +428,8 @@ sub check_install_is_complete
chomp $suffix;
$tmp_loc = "$tmp_loc/$install_dir";
$bindir = "$tmp_loc/bin";
- $libdir = "$tmp_loc/lib/postgresql";
+ $libdir = "$tmp_loc/lib";
+ $libdir .= '/postgresql' unless $libdir =~ /postgres|pgsql/;
}

I'd dismissed that because sifaka isn't running in a directory
that has "postgres" or "pgsql" in its path, but just now I looked
at the logs of one of these steps, and guess where it's installing:

/usr/bin/make -C '../../../..' DESTDIR='/Users/buildfarm/bf-data/HEAD/pgsql.build'/tmp_install install >'/Users/buildfarm/bf-data/HEAD/pgsql.build'/tmp_install/log/install.log 2>&1

I bet the "pgsql.build" name is confusing it into doing extra
installs. This'd explain the impression I had that the test steps
were running a bit slower than they ought to. If you check
sifaka's just-posted green run against its history, that run took
13:48 versus recent times of 10:35 or thereabouts, so we're definitely
eating a good deal of time someplace...

regards, tom lane

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Andres Freund 2025-03-04 22:50:34 Re: Refactoring postmaster's code to cleanup after child exit
Previous Message Jelte Fennema-Nio 2025-03-04 22:49:01 Re: Next commitfest app release is planned for March 18th