Re: Practical maximums (was Re: PostgreSQL theoretical

On Mon, 2006-08-07 at 16:09, Ron Johnson wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Scott Marlowe wrote:
> > On Mon, 2006-08-07 at 15:18, Ron Johnson wrote:
> >> -----BEGIN PGP SIGNED MESSAGE-----
> >> Hash: SHA1
> >>
> >> Scott Marlowe wrote:
> >>> Oh, I see where you were headed.
> >>>
> >>> I've found a bit here and there googling about for multiplex and tar,
> >>> and found an IO:Multiplex module for perl.
> >>>
> >>> I don't see why someone couldn't make use of it to create some kind of
> >>> system where you'd do:
> >>>
> >>> pg_dump dbname | perl mulitiplexscript device1,device2,device3,devicen
> >>>
> >>> No need for postgresql to support it directly. Restoring would likewise
> >>> just be a reverse operation.
> >> Interesting. Many thanks. Scary though that it hasn't been touched
> >> in 30 months.
> >>
> >> With multiple SCSI cards, each with it's own tape drive, this
> >> immensely speeds up the backup operation.
> >
> > Yeah, I read the description, and I think it's just a fancy name for
> > tee. sigh. Not REAL multiplexing, but stream duplication.
>
> This is where a multi-threaded pg_tapedump would be more effective,
> since it would be able to, for example, have 4 threads reading
> (different parts of) the database and writing to 4 separate tape drives.

Actually, I kinda prefer the idea of creating multiple streams using
something like RAID, where if you have two streams, it's a mirror, if
you have three or more then you rotate around a parity stripe like RAID
5. Then, any error on any one tape drive could be recovered. Veritas
has something like that for tape drives.

Heck, the more I think about it, the more I think it would be an
interesting project for a device driver that was like /dev/mdt or
something, and re-used the md libs from the hard drive universe.

Attach X tape drive, put in a bunch of tapes, and just pg_dump > mdt0
and you're backing up. Restore the other way around.