Re: Sync Scan update

On Tue, Jan 02, 2007 at 09:48:22AM -0800, Jeff Davis wrote:
> On Sat, 2006-12-30 at 13:35 -0600, Jim C. Nasby wrote:
> > > My current implementation relies on the scans to stay close together
> > > once they start close together. If one falls seriously behind, it will
> > > fall outside of the main "cache trail" and cause the performance to
> > > degrade due to disk seeking and lower cache efficiency.
> >
> > That's something else that it would be really good to have data for; in
> > some cases it will be better for the slow case to just fall behind, but
> > in other cases the added seeking will slow everything down enough that
> > it would have been faster to just stay at the speed of the slow scan.
> > The question is where those two thresholds are...
>
> Right. I will do more testing for my basic patch soon, but a lot of
> testing is required to characterize when the scans should move apart and
> when they should stay together. The problem is that there are a lot of
> variables. If you have a few scans that uses a moderate amount of CPU,
> the scans might all stay together (I/0 bound). But as soon as you get
> more scans, those scans could all become CPU bound (and could be mixed
> with other types of scans on the same table).
>
> If you have some ideas for tests I can run I'll get back to you with the
> results. However, this kind of test would probably need to be run on a
> variety of hardware.

Well, that's the real trick: ideally, syncscan would be designed in such
a way that you wouldn't have to manually tune at what point scans should
diverge instead of converge; the system should just figure it out.
--
Jim Nasby jim(at)nasby(dot)net
EnterpriseDB http://enterprisedb.com 512.569.9461 (cell)