Re: polyphase merge?

Hmm probably it's worth to try this. Has anybody try this before? If not, I
am interested to give it a try.

Regards,

Don

On Wed, Feb 4, 2009 at 11:25 PM, Gregory Stark <stark(at)enterprisedb(dot)com>wrote:

> Simon Riggs <simon(at)2ndQuadrant(dot)com> writes:
>
> > On Wed, 2009-02-04 at 10:55 +0000, Greg Stark wrote:
> >> Is this basically the same as our current algorithm but without
> >> multiplexing the tapes onto single files? I have been wondering
> >> whether we multiplex the tapes any better than filesystems can lay out
> >> separate files actually.
> >
> > I don't think you'll be able to do that more efficiently than we already
> > do. Read the top of tuplesort.c
>
> Huh?
>
> The question I posed was whether we do it any better than filesystems do,
> not
> whether we could do a better job. If filesystems can do as good a job then
> we
> might as well create separate files for each tape and let the filesystem
> decide where to allocate space. That would mean we could massively simplify
> logtape.c and just create a separate file for every tape.
>
> Possible reasons that filesystems could do better than us are that they
> have
> access to more information about actual storage layout on disk, that they
> have
> more information about hardware characteristics, and that it would give the
> filesystem cache a better idea of the access pattern which logtape.c might
> be
> confusing the picture of currently.
>
> On the other hand possible reasons why filesystems would suck at this
> include
> creating and deleting new files being a slow or locking operation on many
> filesystems, and dealing with directories of large numbers of files being
> poorly optimized on others.
>
> --
> Gregory Stark
> EnterpriseDB http://www.enterprisedb.com
> Ask me about EnterpriseDB's RemoteDBA services!
>