Re: Billions of records?

Shridhar Daithankar <shridhar_daithankar(at)persistent(dot)co(dot)in> writes:
> Other than that there is no limit on size of table. Check this
> http://www.postgresql.org/docs/faqs/FAQ.html#4.5
> BTW, is it coincidence that your figure of 16TB matches the limit there?

No, because he was correctly quoting the FAQ ;-). The 16TB limit comes
from the fact that BlockNumber is 32 bits, so you cannot have a table
larger than 4 billion blocks. (The FAQ is conservatively assuming that
the limit is 2 billion blocks; 2G blocks * 8KB block size = 16TB. In
principle 4 billion should work, since BlockNumber is an unsigned int.
But there used to be places that sloppily used signed arithmetic on
block numbers, and we aren't entirely sure we've gotten rid of all of
'em. If anyone has facilities to test behavior with tables between
16TB and 32TB in size, I'd be interested to hear the results ...)

>> If you all have any other size benchmarks or such please include it. Is
>> PG ready for huge db's? How big is the biggest known?

The 4TB database mentioned in the FAQ belongs to the American Chemical
Society (something about scanning all their journals back to the
mid-1800s...); you can dig in the PG list archives for details.

Another large database I'm aware of is the 2-micron sky survey:
http://pegasus.astro.umass.edu/
which covers a bit under half a billion stars; it is reportedly about
150GB when loaded into Postgres. The UMass people seem to be happy with
the performance they get...

regards, tom lane