Re: Are there plans to add data compression feature to postgresql?

Yes, we are in a data warehouse like environments, where the database server is used to hold very large volumn of read only historical data, CPU, memory, I/O and network are all OK now except storage space, the only goal of compression is to reduce storage consumption.

> Date: Thu, 30 Oct 2008 10:53:27 +1100> From: gxallen(at)gmail(dot)com> To: pgsql-general(at)postgresql(dot)org> Subject: Re: [GENERAL] Are there plans to add data compression feature to postgresql?> > Tom Lane wrote:> > =?utf-8?Q?=E5=B0=8F=E6=B3=A2_=E9=A1=BE?= <guxiaobo1982(at)hotmail(dot)com> writes:> > > >> [ snip a lot of marketing for SQL Server ]> >> > >> > I think the part of this you need to pay attention to is> >> > > >> Of course, nothing is entirely free, and this reduction in space and> >> time come at the expense of using CPU cycles.> >> > >> > We already have the portions of this behavior that seem to me to be> > likely to be worthwhile (such as NULL elimination and compression of> > large field values). Shaving a couple bytes from a bigint doesn't> > strike me as interesting.> > Think about it on a fact table for a warehouse. A few bytes per bigint > multiplied by several billions/trillions of bigints (not an exaggeration > in a DW) and you're talking some significant storage savi
ng on the main > storage hog in a DW. Not to mention the performance _improvements_ you > can get, even with some CPU overhead for dynamic decompression, if the > planner/optimiser understands how to work with the compression index/map > to perform things like range/partition elimination etc. Admittedly this > depends heavily on the storage mechanics and optimisation techniques of > the DB, but there is value to be had there ... IBM is seeing typical > storage savings in the 40-60% range, mostly based on boring, > bog-standard int, char and varchar data.> > The IDUG (so DB2 users themselves, not IBM's marketing) had a > competition to see what was happening in the real world, take a look if > interested: http://www.idug.org/wps/portal/idug/compressionchallenge> > Other big benefits come with XML ... but that is even more dependent on > the starting point. Oracle and SQL Server will see big benefits in > compression with this, because their XML technology is so > mind-bogglin
gly broken in the first place.> > So there's certainly utility in this kind of feature ... but whether it > rates above some of the other great stuff in the PostgreSQL pipeline is > questionable.> > Ciao> Fuzzy> :-)> > ------------------------------------------------> Dazed and confused about technology for 20 years> http://fuzzydata.wordpress.com/> > > -- > Sent via pgsql-general mailing list (pgsql-general(at)postgresql(dot)org)> To make changes to your subscription:> http://www.postgresql.org/mailpref/pgsql-general
_________________________________________________________________
News, entertainment and everything you care about at Live.com. Get it now!
http://www.live.com/getstarted.aspx