Backing up 16TB of data (was Re: > 16TB worth of data question)

On Mon, 2003-04-21 at 13:23, Jeremiah Jahn wrote:
> I have a system that will store about 2TB+ of images per year in a PG
> database. Linux unfortunatly has the 16TB limit for 32bit systems. Not
> really sure what should be done here. Would life better to not store the
> images as BLOBS, and instead come up with some complicated way to only
> store the location in the database, or is there someway to have postgres
> handle this somehow? What are other people out there doing about this
> sort of thing?

Now that the hard disk and file system issues have been hashed around,
have you thought about how you are going to back up this much data?

You'll need multiple modern tape drives (like SuperDLT2, that can
store 160GB *raw* per tape), and having one SCSI controller per
drive, so you don't starve each. drive.

You might also think about daisy chaining a set of mini-libraries.

Then, for performance, you'd also need multiple 66MHz x 64 bit PCI
bridges so each set of 6 PCI slots doesn't get bandwidth constrained.
If PCI-X is shipping, I'd look at that.

The bottom line is that this is going to be a very big box, or
a lot of mid-sized boxes.

--
+-----------------------------------------------------------+
| Ron Johnson, Jr. Home: ron(dot)l(dot)johnson(at)cox(dot)net |
| Jefferson, LA USA http://members.cox.net/ron.l.johnson |
| |
| An ad currently being run by the NEA (the US's biggest |
| public school TEACHERS UNION) asks a teenager if he can |
| find sodium and *chloride* in the periodic table of the |
| elements. |
| And they wonder why people think public schools suck... |
+-----------------------------------------------------------+