Re: Improve lseek scalability v3

On Fri, Sep 16, 2011 at 04:16:49PM +0200, Andres Freund wrote:
> I sent an email containing benchmarks from Robert Haas regarding the Subject.
> Looking at lkml.org I can't see it right now, Will recheck when I am at home.
>
> He replaced lseek(SEEK_END) with fstat() and got speedups up to 8.7 times the
> lseek performance.
> The workload was 64 clients hammering postgres with a simple readonly workload
> (pgbench -S).

Yay! Data!

> For reference see the thread in the postgres archives which also links to
> performance data: http://archives.postgresql.org/message-
> id/CA+TgmoawRfpan35wzvgHkSJ0+i-W=VkJpKnRxK2kTDR+HsanWA(at)mail(dot)gmail(dot)com

So both fstat and lseek do more work than postgres wants. lseek modifies
the file pointer while fstat copies all kinds of unnecessary information
into userspace. I imagine this is the source of the slowdown seen in
the 1-client case.

There have been various proposals to make the amount of information returned
by fstat limited to the 'cheap' (for various definitions of 'cheap') fields.

I'd like to dig into the requirement for knowing the file size a little
better. According to the blog entry it's used for "the query planner".
Does the query planner need to know the exact number of bytes in the file,
or is it after an order-of-magnitude? Or to-the-nearest-gigabyte?

--
Matthew Wilcox Intel Open Source Technology Centre
"Bill, look, we understand that you're interested in selling us this
operating system, but compare it to ours. We can't possibly take such
a retrograde step."