Re: Warm-cache prefetching

Bruce,

It (the compute intensity optimization) is what we did for copy parsing, and it sped up by a factor of 100+.

The rest of the copy path could use some work too.

Yge virtual tuples in 8.1 are another example of grouping operations into more compact chunks instead of doing bits at a time.

The l2 cache optimization could pay huge dividends for certain parts of the code path, like sorting for instance. A simple outer loop that strip mines in l2 sized chunks in sorting could speed things up 5 to 10 times in that section.
- Luke
--------------------------
Sent from my BlackBerry Wireless Device

-----Original Message-----
From: pgsql-hackers-owner(at)postgresql(dot)org <pgsql-hackers-owner(at)postgresql(dot)org>
To: Simon Riggs <simon(at)2ndquadrant(dot)com>
CC: Qingqing Zhou <zhouqq(at)cs(dot)toronto(dot)edu>; pgsql-hackers(at)postgresql(dot)org <pgsql-hackers(at)postgresql(dot)org>
Sent: Fri Dec 09 09:43:33 2005
Subject: Re: [HACKERS] Warm-cache prefetching

Do these optimizations have any affect on database software? I know
call overhead shows up as a performance bottleneck, but do these others
optimizations also measurably improve performance?

---------------------------------------------------------------------------

Simon Riggs wrote:
> This technique and others are discussed in detail in the Intel
> Optimization Manual:
> http://apps.intel.com/scripts-util/download.asp?url=/design/PentiumII/manuals/24512701.pdf&title=Intel%AE+Architecture+Optimization+Reference+Manual&fullpg=3&site=Developer
>
> Similar manual exists for AMD and other architectures.
>
> On Thu, 2005-12-08 at 23:07 -0500, Qingqing Zhou wrote:
> > I write a program try to simulate it, but I am not good at micro
> > optimization, and I just get a very weak but kind-of-stable improvement. I
> > wonder if any people here are interested to take a look.
>
> You may be trying to use the memory too early. Prefetched memory takes
> time to arrive in cache, so you may need to issue prefetch calls for N
> +2, N+3 etc rather than simply N+1.
>
> p.6-11 covers this.
>
> There's a lot of papers around coming up with interesting sounding
> techniques. AFAICS, all they've done is read the optimization guide and
> tried to apply that wisdom, so it seems a good idea to go back to the
> source.
>
> I think many of these techniques are generic across architectures, so
> there is much to be done in this area, IMHO. Though we must respect
> portability and confirm any tuning through testing.
>
> Best Regards, Simon Riggs
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 5: don't forget to increase your free space map settings
>

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

---------------------------(end of broadcast)---------------------------
TIP 3: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faq