small parallel restore optimization

From: Andrew Dunstan <andrew(at)dunslane(dot)net>
To: PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: small parallel restore optimization
Date: 2009-03-06 17:01:36
Message-ID: 49B156F0.7060704@dunslane.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers


Here's a little optimization for the parallel restore code, that
inhibits reopening the archive file unless the worker will actually need
to read from the file (i.e. a data member). It seems to work OK on both
Linux and Windows, and I propose to apply it in a day or two.

I've seen a recent error that suggests we are clobbering memory
somewhere in the parallel code, as well as Olivier Prennant's reported
error that suggests the same thing, although I'm blessed if I can see
where it might be. Maybe some more eyeballs on the code would help.

cheers

andrew

Index: pg_backup_archiver.c
===================================================================
RCS file: /cvsroot/pgsql/src/bin/pg_dump/pg_backup_archiver.c,v
retrieving revision 1.165
diff -c -u -r1.165 pg_backup_archiver.c
cvs diff: conflicting specifications of output style
--- pg_backup_archiver.c 5 Mar 2009 14:51:10 -0000 1.165
+++ pg_backup_archiver.c 6 Mar 2009 16:51:41 -0000
@@ -3471,8 +3471,11 @@
*
* Note: on Windows, since we are using threads not processes, this
* *doesn't* close the original file pointer but just open a new
one.
+ *
+ * Only do this if we actually need to read the file for data.
*/
- (AH->ReopenPtr) (AH);
+ if ( te->section == SECTION_DATA )
+ (AH->ReopenPtr) (AH);

/*
* We need our own database connection, too
@@ -3490,7 +3493,9 @@
PQfinish(AH->connection);
AH->connection = NULL;

- (AH->ClosePtr) (AH);
+ /* close the file if we reopened it */
+ if ( te->section == SECTION_DATA )
+ (AH->ClosePtr) (AH);

if (retval == 0 && AH->public.n_errors)
retval = WORKER_IGNORED_ERRORS;

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Tom Lane 2009-03-06 17:20:12 Re: small parallel restore optimization
Previous Message Zdenek Kotala 2009-03-06 15:28:30 Re: [PATCH] Regression test fix for Czech locale