Re: pg_dump large-file support > 16GB

On Thu, 2005-03-17 at 15:09, Lonni J Friedman wrote:
> On Thu, 17 Mar 2005 14:05:35 +0100, Rafael Martinez Guerrero
> <r(dot)m(dot)guerrero(at)usit(dot)uio(dot)no> wrote:
> > Hello
> >
> > We are having problems with pg_dump.
> >
> > We are trying to dump a 30GB+ database using pg_dump with the --file
> > option. In the beginning everything works fine, pg_dump runs and we get
> > a dumpfile. But when this file becomes 16GB it disappears from the
> > filesystem, pg_dump continues working without giving an error until it
> > finnish (even when the file does not exist)(The filesystem has free
> > space).
> >
> > I can generate without problems files bigger than 16GB with other
> > programs.
> >
> > Some information:
> > ---------------------------
> > OS: Red Hat Enterprise Linux WS release 3 (Taroon Update 4)
> > Kernel: 2.4.21-27.0.2.ELsmp #1 SMP i686
> > PG: 7.4.7
> >
> > LVM version 1.0.8-2(26/05/2004)
> > EXT3 FS 2.4-0.9.19, 19 August 2002 on lvm(58,6), internal journal
> > EXT3-fs: mounted filesystem with ordered data mode.
> > ----------------------------
> >
> > Any ideas? It looks like pg_dump has a limit of 16GB? How can we solve
> > this?
> >
>
> Have you tried piping the dump through split so that each file's size
> is limited?

It is a possibility. But when you have several hundred databases, the
amount of backup files you have to take care of, grow in a exponencial
form ;).

We already have several well tested backup/maintenace/restore scripts
that we would like to continue using.

My question is why is this limit (16GB) there, when my OS does not have
that limit? Is it possible to take it away in a easy way? It looks like
pg_dump is compiled with large-file support because it can work with
files bigger than 4GB.

More ideas? :)

Regards.
--
Rafael Martinez, <r(dot)m(dot)guerrero(at)usit(dot)uio(dot)no>
Center for Information Technology Services
University of Oslo, Norway

PGP Public Key: http://folk.uio.no/rafael/