Re: Backing up 16TB of data (was Re: > 16TB worth of

From: "Fred Moyer" <fred(at)digicamp(dot)com>
To: <ron(dot)l(dot)johnson(at)cox(dot)net>
Cc: <pgsql-general(at)postgresql(dot)org>
Subject: Re: Backing up 16TB of data (was Re: > 16TB worth of
Date: 2003-04-27 15:15:32
Message-ID: 33237.68.73.117.46.1051456532.squirrel@mail.digicamp.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

Here's a 70 TB Backup RAID at the University of Tübingen that should be
able handle the needs adequately.

http://www.tomshardware.com/storage/20030425/index.html

> On Sat, 2003-04-26 at 19:18, Jan Wieck wrote:
>> Lincoln Yeoh wrote:
>> >
>> > At 07:32 PM 4/25/2003 -0400, Jan Wieck wrote:
>> > >Of course, assuming we want to backup the total 24 Terabyte he has
>> in 2-3 years in less than a day, if we have a month to take a
>> backup we can save some money on the backup solution.
>> >
>> > If you take the month approach, the filesystem/db snapshot has to
>> stay in place for a full month whilst the backup is occuring.
>> Do-able, but can be stressful e.g. if something goes wrong after
>> trying for a full month...
>> >
>> > But anyway he said off-line backups aren't that important - I gather
>> that recreating the data is not totally impractical. Still 16TB,
>> ouch.
>>
>> I think a scenario like that, where one has a relatively small
>> percentage of really updated data and a huge portion of constantly
>> growing, is a good example for when it might be appropriate NOT to
>> store everything in one database.
>>
>> Depending on the rest of the requirements, it might need 2PC and using
>> 2 databases. The updated database then could be backed up by normal
>> means while for the constantly growing one you just archive the redo
>> logs.
>
> Here's another thought: does all 16TB of data *really* have to be
> in the database all the time??
>
> Maybe there's a business rule that anything older than 6 months isn't in
> the database itself, but is "just" sitting out on somewhere on a
> filesystem, and if the old data is requested, then, either
> programmatically or thru operator intervention, the old data is copied
> into the database.
>
> Yes, it would take longer to access "old" data, but, hey, that's
> reality (unless you want to spend *really*large* amounts of money). And
> it's not an "OSS vs.Proprietary" either.

In response to

Browse pgsql-general by date

  From Date Subject
Next Message Tom Lane 2003-04-27 15:31:29 Re: Resetting a sequence's last value...
Previous Message Bruno Baguette 2003-04-27 14:17:55 Cannot use the queries buffer of psql... :-(