Re: > 16TB worth of data question

From: Jeremiah Jahn <jeremiah(at)cs(dot)earlham(dot)edu>
To: Jonathan Bartlett <johnnyb(at)eskimo(dot)com>
Cc: postgres list <pgsql-general(at)postgresql(dot)org>
Subject: Re: > 16TB worth of data question
Date: 2003-04-22 16:39:13
Message-ID: 1051029553.4188.136.camel@bluejay.goodinassociates.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

JFS is the same, however all Linux filsystems are limit to 16TB on 32bit
HW. Due to the page caching code. That's with the 2.5 kernel. The 2.4 is
4TB iirc.

On Tue, 2003-04-22 at 11:07, Jonathan Bartlett wrote:
> > FWIW, FreeBSD's UFS filesystem doesn't suffer from the 2G limit. It's
> > limit is something like 4TB iirc.
>
> If the data itself is large (i.e. - large images not just large # of
> images), I'd go with an XFS filesystem. It has a 5 petabyte limit, and
> for large sequential reads, it has almost full-disk speed.
>
> Jon
>
> >
> > I'll also second the opinion that if you're really going to put that
> > much data in your database, x86 hardware might not be a good idea.
> > However, I'd recommend RS/6000 over Sun if you can afford it; for
> > Oracle, the standard is that you only need 1/2 the CPUs in RS/6000 as
> > you would in a Sun, because RS/6000 puts heavy emphasis on memory
> > bandwidth.
> >
> > On Mon, Apr 21, 2003 at 01:23:55PM -0500, Jeremiah Jahn wrote:
> > > I have a system that will store about 2TB+ of images per year in a PG
> > > database. Linux unfortunatly has the 16TB limit for 32bit systems. Not
> > > really sure what should be done here. Would life better to not store the
> > > images as BLOBS, and instead come up with some complicated way to only
> > > store the location in the database, or is there someway to have postgres
> > > handle this somehow? What are other people out there doing about this
> > > sort of thing?
> > >
> > > thanx,
> > > -jj-
> > >
> > > --
> > > Jeremiah Jahn <jeremiah(at)cs(dot)earlham(dot)edu>
> > >
> > >
> > > ---------------------------(end of broadcast)---------------------------
> > > TIP 1: subscribe and unsubscribe commands go to majordomo(at)postgresql(dot)org
> > >
> >
> > --
> > Jim C. Nasby (aka Decibel!) jim(at)nasby(dot)net
> > Member: Triangle Fraternity, Sports Car Club of America
> > Give your computer some brain candy! www.distributed.net Team #1828
> >
> > Windows: "Where do you want to go today?"
> > Linux: "Where do you want to go tomorrow?"
> > FreeBSD: "Are you guys coming, or what?"
> >
> >
> > ---------------------------(end of broadcast)---------------------------
> > TIP 1: subscribe and unsubscribe commands go to majordomo(at)postgresql(dot)org
> >
--
Jeremiah Jahn <jeremiah(at)cs(dot)earlham(dot)edu>

In response to

Browse pgsql-general by date

  From Date Subject
Next Message Philip Hallstrom 2003-04-22 16:47:46 Re: > 16TB worth of data question
Previous Message Jeremiah Jahn 2003-04-22 16:34:46 Re: > 16TB worth of data question