From: | Andrew Dunstan <andrew(at)dunslane(dot)net> |
---|---|
To: | pgsql-committers(at)postgresql(dot)org |
Subject: | pgsql: Add parallel pg_dump option. |
Date: | 2013-03-24 15:39:44 |
Message-ID: | E1UJn1M-0001uh-Nw@gemulon.postgresql.org |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-committers pgsql-hackers |
Add parallel pg_dump option.
New infrastructure is added which creates a set number of workers
(threads on Windows, forked processes on Unix). Jobs are then
handed out to these workers by the master process as needed.
pg_restore is adjusted to use this new infrastructure in place of the
old setup which created a new worker for each step on the fly. Parallel
dumps acquire a snapshot clone in order to stay consistent, if
available.
The parallel option is selected by the -j / --jobs command line
parameter of pg_dump.
Joachim Wieland, lightly editorialized by Andrew Dunstan.
Branch
------
master
Details
-------
http://git.postgresql.org/pg/commitdiff/9e257a181cc1dc5e19eb5d770ce09cc98f470f5f
Modified Files
--------------
doc/src/sgml/backup.sgml | 18 +
doc/src/sgml/perform.sgml | 9 +
doc/src/sgml/ref/pg_dump.sgml | 89 +++-
src/bin/pg_dump/Makefile | 2 +-
src/bin/pg_dump/compress_io.c | 10 +
src/bin/pg_dump/dumputils.c | 86 ++-
src/bin/pg_dump/dumputils.h | 13 +-
src/bin/pg_dump/parallel.c | 1293 +++++++++++++++++++++++++++++++++
src/bin/pg_dump/parallel.h | 85 +++
src/bin/pg_dump/pg_backup.h | 11 +-
src/bin/pg_dump/pg_backup_archiver.c | 735 +++++++------------
src/bin/pg_dump/pg_backup_archiver.h | 35 +-
src/bin/pg_dump/pg_backup_custom.c | 88 +++-
src/bin/pg_dump/pg_backup_db.c | 20 +-
src/bin/pg_dump/pg_backup_directory.c | 264 +++++++-
src/bin/pg_dump/pg_backup_tar.c | 8 +-
src/bin/pg_dump/pg_dump.c | 681 ++++++++++--------
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/pg_dump_sort.c | 92 +++-
src/bin/pg_dump/pg_dumpall.c | 20 +-
src/bin/pg_dump/pg_restore.c | 17 +-
src/tools/msvc/Mkvcbuild.pm | 5 +
22 files changed, 2765 insertions(+), 819 deletions(-)
From | Date | Subject | |
---|---|---|---|
Next Message | Tom Lane | 2013-03-24 19:35:46 | pgsql: Fix some unportable constructs in parallel pg_dump code. |
Previous Message | Tom Lane | 2013-03-23 23:18:10 | pgsql: Update time zone abbreviation lists for changes missed since 200 |
From | Date | Subject | |
---|---|---|---|
Next Message | Andrew Dunstan | 2013-03-24 15:44:34 | parallel pg_dump |
Previous Message | Greg Smith | 2013-03-24 14:26:12 | Re: Let's invent a function to report lock-wait-blocking PIDs |