Re: WORM and Read Only Tables (v0.1)

From: Josh Berkus <fuzzy(at)agliodbs(dot)com>
To: pgsql-hackers(at)postgresql(dot)org
Cc: Simon Riggs <simon(at)2ndquadrant(dot)com>
Subject: Re: WORM and Read Only Tables (v0.1)
Date: 2007-12-12 04:30:03
Message-ID: 200712112030.03473.fuzzy@agliodbs.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Simon,

> Use Case: VLDB with tons of (now) read only data, some not. Data needs
> to be accessible, but data itself is rarely touched, allowing storage
> costs to be minimised via a "storage hierarchy" of progressively cheaper
> storage.

There's actually 2 cases to optimize for:
1) write-once-read-many (WORM)
2) write-once-read-seldom (WORS)

The 2nd case is becoming extremely popular due to the presence of
government-mandated records databases. For example, I'm currently working on
one call completion records database which will hold 75TB of data, of which
we expect less than 1% to *ever* be queried.

One of the other things I'd like to note is that for WORM, conventional
storage is never going to approach column-store DBs for general performance.
So, should we be working on incremental improvements like the ones you
propose, or should we be working on integrating a c-store into PostgreSQL on
a per-table basis?

--
Josh "the Fuzzy" Berkus
San Francisco

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Josh Berkus 2007-12-12 04:36:06 Re: VLDB Features
Previous Message Tom Lane 2007-12-12 04:25:58 Re: Why my C function is called twice?