| From: | Mark Dilger <mark(dot)dilger(at)enterprisedb(dot)com> |
|---|---|
| To: | Joe Carlson <jwcarlson(at)lbl(dot)gov> |
| Cc: | Rob Sargent <robjsargent(at)gmail(dot)com>, pgsql-general(at)lists(dot)postgresql(dot)org |
| Subject: | Re: TEXT column > 1Gb |
| Date: | 2023-04-12 17:04:32 |
| Message-ID: | E18E02B1-39DE-4464-9C8C-F0C06BBF771B@enterprisedb.com |
| Views: | Raw Message | Whole Thread | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-general |
> On Apr 12, 2023, at 7:59 AM, Joe Carlson <jwcarlson(at)lbl(dot)gov> wrote:
>
> The use case is genomics. Extracting substrings is common. So going to chunked storage makes sense.
Are you storing nucleotide sequences as text strings? If using the simple 4-character (A,C,G,T) alphabet, you can store four bases per byte. If using a nucleotide code 16-character alphabet you can still get two bases per byte. An amino acid 20-character alphabet can be stored 8 bases per 5 bytes, and so forth. Such a representation might allow you to store sequences two or four times longer than the limit you currently hit, but then you are still at an impasse. Would a factor or 2x or 4x be enough for your needs?
—
Mark Dilger
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Joe Carlson | 2023-04-12 17:19:57 | Re: TEXT column > 1Gb |
| Previous Message | Rob Sargent | 2023-04-12 16:36:25 | Re: TEXT column > 1Gb |