Re: Maximum table size

Tatsuo Ishii wrote:
> > Tom Lane wrote:
> > > Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> > > > Is our maximum table size limited by the maximum block number?
> > >
> > > Certainly.
> > >
> > > > Is the 16TB number a hold-over from when we weren't sure block number
> > > > was unsigned, though now we are pretty sure it is handled as unsigned
> > > > consistenly?
> > >
> > > It's a holdover. As to how certain we are that all the
> > > signed-vs-unsigned bugs are fixed, who have you heard from running a
> > > greater-than-16Tb table? And how often have they done CLUSTER, REINDEX,
> > > or even VACUUM FULL on it? AFAIK we have zero field experience to
> > > justify promising that it works.
> > >
> > > We can surely fix any such bugs that get reported, but we haven't got
> > > any infrastructure that would find or prevent 'em.
> >
> > I guess the big question is what do we report as the maximum table size?
> > Do we report 32TB and fix any bug that happen over 16TB?
>
> That seems right direction for me. I see no reason why 16TB is more
> reliable number than 32TB, since nobody has ever tried to build 16TB
> tables.

Agreed. I think the question is how large does the design support,
rather than how large have we tested. (In fact, the check for using
block numbers as unsigned was removed from the FAQ when I reviewed the
code.)

I know Tom is concerned because we haven't tested it, but I don't think
anyone has tested 16TB either, nor our 1600-column limit.

Also, I think people look at these numbers to determine if PostgreSQL
can handle their data needs 5-10 years down the road.

In fact, if you increase the page size, you can quadruple most of the
existing limits. This is already mentioned in the FAQ:

<P>The maximum table size and maximum number of columns can
be increased if the default block size is increased to 32k.</P>

I have updated the FAQ to say 32TB, and of course, larger page sizes
could make this 128TB.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073