Re: Something fishy happening on frogmouth

On Wed, Oct 30, 2013 at 09:07:43AM -0400, Robert Haas wrote:
> On Wed, Oct 30, 2013 at 8:47 AM, Andres Freund <andres(at)2ndquadrant(dot)com> wrote:
> > On 2013-10-30 08:45:03 -0400, Robert Haas wrote:
> >> If I'm reading this correctly, the last three runs on frogmouth have
> >> all failed, and all of them have failed with a complaint about,
> >> specifically, Global/PostgreSQL.851401618. Now, that really shouldn't
> >> be happening, because the code to choose that number looks like this:
> >>
> >> dsm_control_handle = random();

> > Could it be that we haven't primed the random number generator with the
> > time or something like that yet?
>
> Yeah, I think that's probably what it is.

I experienced a variation of this, namely a RHEL 7 system where initdb always
says "selecting dynamic shared memory implementation ... sysv". Each initdb
is rejecting posix shm by probing the same ten segments:

$ strace initdb -D scratch 2>&1 | grep /dev/shm/P
open("/dev/shm/PostgreSQL.1804289383", O_RDWR|O_CREAT|O_EXCL|O_NOFOLLOW|O_CLOEXEC, 0600) = -1 EEXIST (File exists)
open("/dev/shm/PostgreSQL.846930886", O_RDWR|O_CREAT|O_EXCL|O_NOFOLLOW|O_CLOEXEC, 0600) = -1 EEXIST (File exists)
open("/dev/shm/PostgreSQL.1681692777", O_RDWR|O_CREAT|O_EXCL|O_NOFOLLOW|O_CLOEXEC, 0600) = -1 EEXIST (File exists)
open("/dev/shm/PostgreSQL.1714636915", O_RDWR|O_CREAT|O_EXCL|O_NOFOLLOW|O_CLOEXEC, 0600) = -1 EEXIST (File exists)
open("/dev/shm/PostgreSQL.1957747793", O_RDWR|O_CREAT|O_EXCL|O_NOFOLLOW|O_CLOEXEC, 0600) = -1 EEXIST (File exists)
open("/dev/shm/PostgreSQL.424238335", O_RDWR|O_CREAT|O_EXCL|O_NOFOLLOW|O_CLOEXEC, 0600) = -1 EEXIST (File exists)
open("/dev/shm/PostgreSQL.719885386", O_RDWR|O_CREAT|O_EXCL|O_NOFOLLOW|O_CLOEXEC, 0600) = -1 EEXIST (File exists)
open("/dev/shm/PostgreSQL.1649760492", O_RDWR|O_CREAT|O_EXCL|O_NOFOLLOW|O_CLOEXEC, 0600) = -1 EEXIST (File exists)
open("/dev/shm/PostgreSQL.596516649", O_RDWR|O_CREAT|O_EXCL|O_NOFOLLOW|O_CLOEXEC, 0600) = -1 EEXIST (File exists)
open("/dev/shm/PostgreSQL.1189641421", O_RDWR|O_CREAT|O_EXCL|O_NOFOLLOW|O_CLOEXEC, 0600) = -1 EEXIST (File exists)

Regular postmaster runs choose a random segment, but initdb, bootstrap
postgres, and single-user postgres all start with the same segment. These
segments are months old. Perhaps I was testing something that caused a
bootstrap postgres to crash. After ten such crashes, future initdb runs
considered posix shm unusable.

> There's PostmasterRandom()
> to initialize the random-number generator on first use, but that
> doesn't help if some other module calls random(). I wonder if we
> ought to just get rid of PostmasterRandom() and instead have the
> postmaster run that initialization code very early in startup.

Usually, the first srandom() call happens early in PostmasterMain(). I plan
to add one to InitStandaloneProcess(), which substitutes for several tasks
otherwise done in PostmasterMain(). That seems like a good thing even if DSM
weren't in the picture. Also, initdb needs an srandom() somewhere;
choose_dsm_implementation() itself seems fine. Attached. With this, "make
-j20 check-world" selected posix shm and passed even when I forced DSM
creation to fail on unseeded random():

--- a/src/backend/storage/ipc/dsm_impl.c
+++ b/src/backend/storage/ipc/dsm_impl.c
@@ -249,2 +249,5 @@ dsm_impl_posix(dsm_op op, dsm_handle handle, Size request_size,

+ if (handle == 1804289383)
+ elog(ERROR, "generated handle with no randomness");
+
snprintf(name, 64, "/PostgreSQL.%u", handle);