Re: Large table load (40 millon rows) - WAL hassles

> Version: Postgres 7.1.2
>
> A product we are developing requires frequent loading
> of a large number of rows into a table. We are using
> the "copy file" command, but with WAL we are effectively
> doubling the amount of disk writing we are doing.
>
> After the rows are loaded we do a "create index".
>
> Is there a way to turn off WAL or otherwise speeding up
> the table loading process? Would db_restore be faster?
>
> The rows are loaded in sorted order. Does this impact
> index creation negatively or positively?
>
> We are currently working with test data but we estimate
> production data to be 6 - 9 billion rows. Is anyone
> else running with these volumes?
>

We have a problem with 7.1.X where the WAL logs stay around longer than
needed. Maybe someone has a patch with that fix.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026