Re: Speed up build on Windows by generating symbol definition in batch

Thanks for reviewing!
I've updated the patch according to your comments.

Best regards,
Peifeng Qiu

On Sun, Apr 7, 2019 at 2:31 PM Noah Misch <noah(at)leadboat(dot)com> wrote:

> On Sat, Mar 30, 2019 at 03:42:39PM +0900, Peifeng Qiu wrote:
> > When I watched the whole build process with a task manager, I discovered
> > that a lot of time was spent on generating export symbol definitions,
> > without consuming much CPU or IO.
> > The script that doing this is src/tools/msvc/gendef.pl, it enumerates
> the
> > whole directory for ".obj" files and call dumpbin utility to generate
> > ".sym" files one by one like this:
> >
> > dumpbin /symbols /out:a.sym a.obj >NUL
> >
> > Actually the dumpbin utility accepts a wildcard file name, so we can
> > generate the export symbols of all ".obj" files in batch.
> >
> > dumpbin /symbols /out:all.sym *.obj >NUL
> >
> > This will avoid wasting time by creating and destroying dumpbin process
> > repeatedly and can speed up the build process considerably.
> > I've tested on my 4-core 8-thread Intel i7 CPU. I've set MSBFLAGS=/m to
> > ensure it can utilize all CPU cores.
> > Building without this patch takes about 370 seconds. Building with this
> > patch takes about 200 seconds. That's almost 2x speed up.
>
> I, too, get a strong improvement, from 201s to 149s. I can confirm it
> yields
> identical *.def files. Thanks for identifying this improvement.
>
> > - my ($objfile, $symfile) = @_;
> > - my ($symvol, $symdirs, $symbase) = splitpath($symfile);
> > - my $tmpfile = catpath($symvol, $symdirs, "symbols.out");
>
> You removed the last use of File::Spec::Functions, so remove its "use"
> statement.
>
> > - system("dumpbin /symbols /out:$tmpfile $_ >NUL")
> > - && die "Could not call dumpbin";
>
> This error handling was crude, but don't replace it with zero error
> handling.
>
> > - rename($tmpfile, $symfile);
>
> Keep the use of a temporary file, too.
>
> > +system("dumpbin /symbols /out:$symfile $ARGV[0]/*obj >NUL");
>
> That should be *.obj, not *obj.
>