Why does the WAL writer sit on completed segments (on Windows)?

From: Christian Ullrich <chris(at)chrullrich(dot)net>
To: pgsql-admin(at)postgresql(dot)org
Subject: Why does the WAL writer sit on completed segments (on Windows)?
Date: 2017-04-17 15:00:35
Message-ID: od2lad$df0$1@blaine.gmane.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-admin

Hello,

I just encountered a problem with my archive_command when using 7-Zip to
compress the WAL file. PostgreSQL is 9.6.2, the EDB x64 build.

archive_command is set to a batch file that effectively runs:

7z a -tgzip M:\y\archive\%f.gz %p

Whenever a segment is ready to be archived, I get three log entries like
this:

LOG: archive command failed with exit code 2
DETAIL: The failed archive command was: G:\data-9.6\archive.cmd
pg_xlog\000000010000000000000011 000000010000000000000011

WARNING: The process cannot access the file because it is being
used by another process.

The reason, as Process Monitor shows, is that the WAL writer process
keeps the file open for about one second after the
archive_status\*.ready file has been created. The archiver runs the
archive_command "too early", resulting in the sharing [1] violation seen
above.

It gives up after three attempts and tries again a minute later. That
attempt then succeeds (except for when the checkpointer is _still_
sitting on the file; I've seen that twice now, too).

When 7-Zip starts, it tries to open the file with FILE_SHARE_READ only,
which is sensible for an archiver that wants to prevent other people
changing the input file while it is being read. Since the WAL writer has
it open with write access, the attempt fails.

Is this a known, er, feature, or should I consider it a bug in PostgreSQL?

There is a workaround; it works when I let 7z read the file through an
input redirection:

7z a -tgzip -si M:\y\archive\%f.gz < %p

This always works, presumably because cmd opens the file with a more
generous share mode.

[1] Nothing to do with file shares; pg_xlog and archive are on the same
system. Share modes are the reason open files usually cannot be
deleted on Windows.

Thanks,

--
Christian

Responses

Browse pgsql-admin by date

  From Date Subject
Next Message Tom Lane 2017-04-17 15:28:31 Re: Why does the WAL writer sit on completed segments (on Windows)?
Previous Message Gone, Sajan 2017-04-17 14:24:43 Re: Postgres replication