From: | Seb <spluque(at)gmail(dot)com> |
---|---|
To: | pgsql-general(at)postgresql(dot)org |
Subject: | break table into portions for writing to separate files |
Date: | 2014-05-01 17:50:57 |
Message-ID: | 87oazhpev2.fsf@net82.ceos.umanitoba.ca |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-general |
Hello,
I've been looking for a way to write a table into multiple files, and am
wondering if there are some clever suggestions. Say we have a table
that is too large (several Gb) to write to a file that can be used for
further analyses in other languages. The table consists of a timestamp
field and several numeric fields, with records every 10th of a second.
It could be meaningfully broken down into subsets of say 20 minutes
worth of records. One option is to write a shell script that loops
through the timestamp, selects the corresponding subset of the table,
and writes it as a unique file. However, this would be extremely slow
because each select takes several hours, and there can be hundreds of
subsets. Is there a better way?
Cheers,
--
Seb
From | Date | Subject | |
---|---|---|---|
Next Message | dinesh kumar | 2014-05-01 18:20:17 | Re: break table into portions for writing to separate files |
Previous Message | Shaun Thomas | 2014-05-01 15:54:46 | Re: Backups over slave instead master? |