Re: Synchrnonized Scan test

Luke Lonergan wrote:
> I've attached the results of a test with the first patch and the test. It
> uses five simultaneous scans as before, but this time the table is 120GB on
> a machine with 8GB of RAM. The data is stored on one non-raid disk, so a
> single scan should take about 33 minutes.
>
> The scans are started 5 minutes apart, so the last scan would end in about 3
> hours if they were independent (they're not). 8.2 unmodified runs the test
> in about 4 hours and 20 minutes. With the first patch, it runs in about 5
> hours and 30 minutes.
>
> The results show that the first patch actually slowed things down quite a
> bit. I'll test the last patch you sent and see what we get.
>
> I think this is worthwhile, you can definitely see a slowdown by having the
> competing scans. The OS (Linux 2.6.18) is doing a pretty good job of
> keeping the scans scheduled well, but we could still speed things up by
> about 30% if we got syncscan working.
>

Thanks for running the tests.

I don't know exactly what's happening in your case, but it may have to
do with some unbalanced scheduling. What if we actually throttled each
"follower" to read no more than N pages before it task-switched? Might
that keep the scans closer together?

If you have any way of tracing what is actually happening, that would be
very helpful.

It would be nice to know:
(1) Are the scans diverging because some processes are running forward
with fast shared_buffers page hits, and maybe other processes
task-switch when they need to read() (even if it's in the OS buffer cache)?
(2) How much difference is there really between a shared_buffer cache
hit and an OS buffer cache hit?
(3) Does an OS buffer cache hit always cause a context switch? How often?
(4) Is read-ahead per-process (like Tom suggested), or is it system wide?

Regards,
Jeff Davis