Re: Practical maximums (was Re: PostgreSQL theoretical

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Jeff Davis wrote:
> On Mon, 2006-08-07 at 14:51 -0500, Ron Johnson wrote:
>
>> What would be darned useful (but only, I think, with heavy usage of
>> tablespaces) is:
>> $ pg_tapedump /some/database /dev/st0,/dev/st1,/dev/st2,/dev/st3
>
> I must be missing something. What is stopping you from doing something
> along the lines of:

Missing nothing. :)

> // mux.script psuedocode
> // X is the size of a stripe of data
> top:
> read X bytes from stdin or exit if EOF
> asynchronously write those X bytes to /dev/st0
> read X bytes from stdin or exit if EOF
> asynchronously write those X bytes to /dev/st1
> goto top

Python queues and threads would make that very simple. Master
thread reads from pg_dump pipe, passing X bytes to a queue, and each
thread performs this loop:
- read from queue
- synchronously write to /dev/stN

> And then make an inverse script called demux.script.
>
> Then:
> $ pg_dump somedatabase | gzip -c | mux.script
> To restore:
> $ demux.script | gunzip -c | psql somedatabase
>
> Would that work? Obviously you'd have to document the process well to
> make sure the someone didn't get confused 12 months later trying to
> restore.

The disk system would have to be fast enough to keep all X tape
drives full.

Depending on the speed of the tape systems and disk system, having X
"threads" reading from the database/writing to tape, all at the same
time is much faster.

> You may have to do something a little more sophisticated to make it work
> more generally, like adding header information to each tape that says "I
> am the 2nd tape of 3".

Depending on the capabilities of your tape drive, CRC, error
checking, load-next-tape, etc, etc.

- --
Ron Johnson, Jr.
Jefferson LA USA

Is "common sense" really valid?
For example, it is "common sense" to white-power racists that
whites are superior to blacks, and that those with brown skins
are mud people.
However, that "common sense" is obviously wrong.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.3 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFE17XRS9HxQb37XmcRApdSAKCv390n8/eQxdodF3D6PQGBvyUSUACfR5NO
ZpL8Ac6RMHQRHNqCEb8Grwo=
=nrzR
-----END PGP SIGNATURE-----