From: | momjian(at)postgresql(dot)org (Bruce Momjian) |
---|---|
To: | pgsql-committers(at)postgresql(dot)org |
Subject: | pgsql: Add ideas for concurrent pg_dump and pg_restore: < * pg_dump > * |
Date: | 2008-03-04 01:33:32 |
Message-ID: | 20080304013332.DC7D1754108@cvs.postgresql.org |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-committers |
Log Message:
-----------
Add ideas for concurrent pg_dump and pg_restore:
< * pg_dump
> * pg_dump / pg_restore
> o Allow pg_dump to utilize multiple CPUs and I/O channels by dumping
> multiple objects simultaneously
>
> The difficulty with this is getting multiple dump processes to
> produce a single dump output file.
> http://archives.postgresql.org/pgsql-hackers/2008-02/msg00205.php
>
> o Allow pg_restore to utilize multiple CPUs and I/O channels by
> restoring multiple objects simultaneously
>
> This might require a pg_restore flag to indicate how many
> simultaneous operations should be performed. Only pg_dump's
> -Fc format has the necessary dependency information.
>
> o To better utilize resources, restore data, primary keys, and
> indexes for a single table before restoring the next table
>
> Hopefully this will allow the CPU-I/O load to be more uniform
> for simultaneous restores. The idea is to start data restores
> for several objects, and once the first object is done, to move
> on to its primary keys and indexes. Over time, simultaneous
> data loads and index builds will be running.
>
> o To better utilize resources, allow pg_restore to check foreign
> keys simultaneously, where possible
> o Allow pg_restore to create all indexes of a table
> concurrently, via a single heap scan
>
> This requires a pg_dump -Fc file because that format contains
> the required dependency information.
> http://archives.postgresql.org/pgsql-general/2007-05/msg01274.php
>
> o Allow pg_restore to load different parts of the COPY data
> simultaneously
< single heap scan, and have a restore of a pg_dump somehow use it
> single heap scan, and have pg_restore use it
< http://archives.postgresql.org/pgsql-general/2007-05/msg01274.php
Modified Files:
--------------
pgsql/doc:
TODO (r1.2251 -> r1.2252)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/doc/TODO?r1=1.2251&r2=1.2252)
pgsql/doc/src/FAQ:
TODO.html (r1.760 -> r1.761)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/doc/src/FAQ/TODO.html?r1=1.760&r2=1.761)
From | Date | Subject | |
---|---|---|---|
Next Message | Bruce Momjian | 2008-03-04 01:38:37 | pgsql: Add URL for: o Allow pg_restore to utilize multiple CPUs |
Previous Message | User Nanardon | 2008-03-04 00:32:45 | pgrpm - pgrpm: - no promote epoch in version comparison (otherwise 4.4.8 |