Re: Best way to handle multi-billion row read-only table?

From: Justin Graf <justin(at)magwerks(dot)com>
To: Asher Hoskins <asher(at)piceur(dot)co(dot)uk>
Cc: pgsql-general(at)postgresql(dot)org
Subject: Re: Best way to handle multi-billion row read-only table?
Date: 2010-02-09 23:05:21
Message-ID: 4B71EA31.4060903@magwerks.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

On 2/9/2010 4:41 PM, Asher Hoskins wrote:
>
> Thanks for that, it looks like partitioning is the way to go. I'm
> assuming that I should try and keep my total_relation_sizes less than
> the memory size of the machine?
This depends on what the quires look like. As other have stated when
partitioning you have to consider how the data is quired.

>
>
> If I partition so that each partition holds data for a single channel
> (and set a CHECK constraint for this) then I can presumably remove the
> channel from the index since constraint exclusion will mean that only
> partitions holding the channel I'm interested in will be searched in a
> query. Given that within a partition all of my sample_time's will be
> different do you know if there's a more efficient way to index these?
Given the timestamp will most likely be the where clause, NO on the
plus side its only 8 bytes

All legitimate Magwerks Corporation quotations are sent in a .PDF file attachment with a unique ID number generated by our proprietary quotation system. Quotations received via any other form of communication will not be honored.

CONFIDENTIALITY NOTICE: This e-mail, including attachments, may contain legally privileged, confidential or other information proprietary to Magwerks Corporation and is intended solely for the use of the individual to whom it addresses. If the reader of this e-mail is not the intended recipient or authorized agent, the reader is hereby notified that any unauthorized viewing, dissemination, distribution or copying of this e-mail is strictly prohibited. If you have received this e-mail in error, please notify the sender by replying to this message and destroy all occurrences of this e-mail immediately.
Thank you.

In response to

Browse pgsql-general by date

  From Date Subject
Next Message Erik Jones 2010-02-09 23:28:14 Recovering data from table show corruption with "invalid page header in block X"
Previous Message Guillaume Lelarge 2010-02-09 22:39:10 Re: viewing large queries in pg_stat_activity