From: | Andres Freund <andres(at)anarazel(dot)de> |
---|---|
To: | pgsql-hackers(at)postgresql(dot)org |
Cc: | Jeff Davis <pgsql(at)j-davis(dot)com>, Noah Misch <noah(at)leadboat(dot)com>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Andrew Dunstan <andrew(at)dunslane(dot)net> |
Subject: | Re: initdb and fsync |
Date: | 2012-03-13 08:42:03 |
Message-ID: | 201203130942.03551.andres@anarazel.de |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
On Tuesday, March 13, 2012 04:49:40 AM Jeff Davis wrote:
> On Sun, 2012-02-05 at 17:56 -0500, Noah Misch wrote:
> > I meant primarily to illustrate the need to be comprehensive, not comment
> > on which executable should fsync a particular file. Bootstrap-mode
> > backends do not sync anything during an initdb run on my system. With
> > your patch, we'll fsync a small handful of files and leave nearly
> > everything else vulnerable.
>
> Thank you for pointing that out. With that in mind, I have a new version
> of the patch which just recursively fsync's the whole directory
> (attached).
>
> I also introduced a new option --nosync (-N) to disable this behavior.
>
> The bad news is that it introduces a lot more time to initdb -- it goes
> from about 1s to about 10s on my machine. I tried fsync'ing the whole
> directory twice just to make sure that the second was a no-op, and
> indeed it didn't make much difference (still about 10s).
I suggest you try making it two loops:
for recursively everything in dir:
posix_fadvise(fd, POSIX_FADV_DONTNEED);
for recursively everything in dir:
fsync(fd);
In my experience that gives way much better performance due to the fact that
it does not force its own metadata/journal commit/transaction for every file
but can be batched. copydir() does the same since some releases...
Obviously its not that nice to use _DONTNEED but I havent found something that
works equally well. You could try sync_file_range(fd, 0, 0,
SYNC_FILE_RANGE_WRITE) in the first loop but my experience with that hasn't
been that good.
Andres
From | Date | Subject | |
---|---|---|---|
Next Message | Andres Freund | 2012-03-13 08:55:04 | Re: wal_buffers, redux |
Previous Message | Daniel Farina | 2012-03-13 07:35:14 | Re: pg_upgrade and statistics |