Re: Importing Large Amounts of Data

From: Gavin Sherry <swm(at)linuxworld(dot)com(dot)au>
To: Curt Sampson <cjs(at)cynic(dot)net>
Cc: pgsql-hackers(at)postgresql(dot)org
Subject: Re: Importing Large Amounts of Data
Date: 2002-04-16 05:17:25
Message-ID: Pine.LNX.4.21.0204161511130.8569-100000@linuxworld.com.au
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Tue, 16 Apr 2002, Curt Sampson wrote:

> > Given the very low parsing and 'planning' overhead, the real cost would be
> > WAL (the bootstrapper could fail and render the database unusable) and the
> > subsequent updating of on-disk relations.
>
> MS SQL Server, when doing a BULK INSERT or BCP, can do it as a fully or
> "minimally" logged operation. When minimally logged, there's no ability
> to roll-forward or recover inserted data, just the ability to go back
> to the state at the beginning of the operation. This technique can work
> even though an on-line database. A bit more information is available at

The other reason I say that this bootstrap tool would still use WAL is
that bypassing WAL would require writing a fairly significant amount of
code (unless the pre-WAL heap_insert() code could be used, with relevant
modification).

On the other hand, I would imagine it to be very difficult to implement
an 'interactive' roll back facility with the kind of tool I am
describing.

Gavin

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Hiroshi Inoue 2002-04-16 05:23:48 Re: ANSI Compliant Inserts
Previous Message Bruce Momjian 2002-04-16 05:15:11 Re: [PATCHES] YADP - Yet another Dependency Patch