== PostgreSQL Weekly News - July 14 2013 ==

== PostgreSQL Weekly News - July 14 2013 ==

== PostgreSQL Product News ==

PostgreSQL Maestro 13.7, an ODBC-based management tool which works
with PostgreSQL, released.
http://www.sqlmaestro.com/products/postgresql/maestro/

== PostgreSQL Jobs for July ==

http://archives.postgresql.org/pgsql-jobs/2013-07/threads.php

== PostgreSQL Local ==

We're going to have a booth at OSCON this year. Please sign up and
hang out at the booth for an hour or two:
https://wiki.postgresql.org/wiki/Oscon_2013_signup

PGDay UK 2013 is July 12, 2013. Registration is closing soon. Book now!
http://postgresqlusergroup.org.uk

PostgreSQL Brazil will be held August 15-17, 2013 in Porto Velho, RO,
Brazil.
http://pgbr.postgresql.org.br/2013/chamada.en.php

Postgres Open 2013 will be in Chicago, IL, USA, September 16-18.
http://postgresopen.org/

The Italian PGDay (PGDay.IT) will be held on October the 25th in
Prato, Tuscany, Italy, at Monash University Center.
The International Call For Papers is open:
http://2013.pgday.it/index.php/programma/call-for-papers-english/

pgconf.EU 2013 will be held on Oct 29-Nov 1, 2013 at the Conrad Hotel
in downtown Dublin, Ireland. The CfP is open.
http://2013.pgconf.eu/

PGConf.DE 2013 will be held November 8th, 2013, at the Rhineland
Industrial Museum in Oberhausen. The CfP is open through September
15, 2013.
http://2013.pgconf.de/

== PostgreSQL in the News ==

Planet PostgreSQL: http://planet.postgresql.org/

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm Pacific time.
Please send English language ones to david(at)fetter(dot)org, German language
to pwn(at)pgug(dot)de, Italian language to pwn(at)itpug(dot)org(dot) Spanish language
to pwn(at)arpug(dot)com(dot)ar(dot)

== Applied Patches ==

Heikki Linnakangas pushed:

- Improve scalability of WAL insertions. This patch replaces
WALInsertLock with a number of WAL insertion slots, allowing
multiple backends to insert WAL records to the WAL buffers
concurrently. This is particularly useful for parallel loading large
amounts of data on a system with many CPUs. This has one
user-visible change: switching to a new WAL segment with
pg_switch_xlog() now fills the remaining unused portion of the
segment with zeros. This potentially adds some overhead, but it has
been a very common practice by DBA's to clear the "tail" of the
segment with an external pg_clearxlogtail utility anyway, to make
the WAL files compress better. With this patch, it's no longer
necessary to do that. This patch adds a new GUC, xloginsert_slots,
to tune the number of WAL insertion slots. Performance testing
suggests that the default, 8, works pretty well for all kinds of
worklods, but I left the GUC in place to allow others with different
hardware to test that easily. We might want to remove that before
release. Reviewed by Andres Freund.
http://git.postgresql.org/pg/commitdiff/9a20a9b21baa819df1760b36f3c36f25d11fc27b

- Fix pg_test_fsync, broken by xloginsert scaling patch. I didn't
realize that ALIGNOF_XLOG_BUFFER was used in pg_test_fsync.
http://git.postgresql.org/pg/commitdiff/b5ed21998c13f09fd733c87a224e3fe27e41d34d

- Fix Windows build. Was broken by my xloginsert scaling patch.
XLogCtl global variable needs to be initialized in each process, as
it's not inherited by fork() on Windows.
http://git.postgresql.org/pg/commitdiff/f489470f8abee19ec0788afad92cf192c132271e

- Add #include needed for _mm_mfence() intrinsic on ia64. Hopefully
this fixes the build failure on buildfarm member dugong.
http://git.postgresql.org/pg/commitdiff/6052bceba5b4cb3d4ae814185d5035307ecabb65

- Fix memory barrier support on icc on ia64, 2nd attempt. Itanium
doesn't have the mfence instruction - that's a 386 thing. Use the
"mf" instruction instead. This reverts the previous commit to add
"#include <emmintrinsic.h>"; the problem was not with a missing
#include.
http://git.postgresql.org/pg/commitdiff/e5592c61adb0766eaee53ec07d2f05783d1c6548

Alvaro Herrera pushed:

- Fix mention of htup.h in pageinspect docs. It's htup_details.h now.
Jeff Janes
http://git.postgresql.org/pg/commitdiff/12fbe2b3dd9c712532541c6a89cd26b1974c66f2

Peter Eisentraut pushed:

- Fix bool abuse. path_encode's "closed" argument used to take three
values: TRUE, FALSE, or -1, while being of type bool. Replace that
with a three-valued enum for more clarity.
http://git.postgresql.org/pg/commitdiff/7888c61238ca082404cf8348a69e2b66bf9f4e96

- Fix lack of message pluralization
http://git.postgresql.org/pg/commitdiff/8dead08c54826e5927bf7a07caa94a86e6981aa0

- Add coverage/ to .gitignore
http://git.postgresql.org/pg/commitdiff/a87416db3b3d76d3b7095c23b08e1899ec2b203d

- doc: Replace link to pgFouine with pgBadger. From: Ian Lawrence
Barwick <barwick(at)gmail(dot)com>
http://git.postgresql.org/pg/commitdiff/f02b14f787980ac5b2f25407e5cd771e6b1a1dfc

- pg_dump: Formatting cleanup of new messages
http://git.postgresql.org/pg/commitdiff/e852c5df2ded8b076693d0c0632dcbda107be89e

- Add session_preload_libraries configuration parameter. This is like
shared_preload_libraries except that it takes effect at backend
start and can be changed without a full postmaster restart. It is
like local_preload_libraries except that it is still only settable
by a superuser. This can be a better way to load modules such as
auto_explain. Since there are now three preload parameters, regroup
the documentation a bit. Put all parameters into one section,
explain common functionality only once, update the descriptions to
reflect current and future realities. Reviewed-by: Dimitri Fontaine
<dimitri(at)2ndQuadrant(dot)fr>
http://git.postgresql.org/pg/commitdiff/070518ddab2c94afea119f2b1944c05d16792b07

- pg_isready: Message improvement
http://git.postgresql.org/pg/commitdiff/1f75a5f9c1307257c7682f2968a835435fd2b4b2

Bruce Momjian pushed:

- pg_upgrade: document possible pg_hba.conf options. Previously,
pg_upgrade docs recommended using .pgpass if using MD5
authentication to avoid being prompted for a password. Turns out
pg_ctl never prompts for a password, so MD5 requires .pgpass ---
document that. Also recommend 'peer' for authentication too.
Backpatch back to 9.1.
http://git.postgresql.org/pg/commitdiff/886c05d8e8642572af25fee23f414bd31f2e7f3e

Noah Misch pushed:

- Switch user ID to the object owner when populating a materialized
view. This makes superuser-issued REFRESH MATERIALIZED VIEW safe
regardless of the object's provenance. REINDEX is an earlier
example of this pattern. As a downside, functions called from
materialized views must tolerate running in a security-restricted
operation. CREATE MATERIALIZED VIEW need not change user ID.
Nonetheless, avoid creation of materialized views that will
invariably fail REFRESH by making it, too, start a
security-restricted operation. Back-patch to 9.3 so materialized
views have this from the beginning. Reviewed by Kevin Grittner.
http://git.postgresql.org/pg/commitdiff/f3ab5d46960023cf8a9df3751ab9748ce01a46a0

- Make comments reflect that omission of SPI_gettypmod() is
intentional.
http://git.postgresql.org/pg/commitdiff/448fee2e238ae4797e68d7d15b49f2fc52691547

- Document the OVER keyword as being unreserved. It became so in
commit 5893ffa79c03824f34ae3d37f211381fd1c17283. David Fetter,
extracted from a larger patch.
http://git.postgresql.org/pg/commitdiff/01337d46eb34503b81b989885d0aa3e25da5a96d

Stephen Frost pushed:

- During parallel pg_dump, free commands from master. The command
strings read by the child processes during parallel pg_dump, after
being read and handled, were not being free'd. This patch corrects
this relatively minor memory leak. Leak found by the Coverity
scanner. Back patch to 9.3 where parallel pg_dump was introduced.
http://git.postgresql.org/pg/commitdiff/234e4cf6e1eac2f0e514379a2a533ffb71b33732

- pg_receivexlog - Exit on failure to parse. In
streamutil.c:GetConnection(), upgrade failure to parse the
connection string to an exit(1) instead of simply returning NULL.
Most callers already immediately exited, but pg_receivexlog would
loop on this case, continually trying to re-parse the connection
string (which can't be changed after pg_receivexlog has started).
GetConnection() was already expected to exit(1) in some cases (eg:
failure to allocate memory or if unable to determine the
integer_datetimes flag), so this change shouldn't surprise anyone.
Began looking at this due to the Coverity scanner complaining that
we were leaking err_msg in this case- no longer an issue since we
just exit(1) immediately.
http://git.postgresql.org/pg/commitdiff/d368a301b3a4bf5fec17e81c630adddeac80a7fc

- Ensure 64bit arithmetic when calculating tapeSpace. In
tuplesort.c:inittapes(), we calculate tapeSpace by first figuring
out how many 'tapes' we can use (maxTapes) and then multiplying the
result by the tape buffer overhead for each. Unfortunately, when we
are on a system with an 8-byte long, we allow work_mem to be larger
than 2GB and that allows maxTapes to be large enough that the 32bit
arithmetic can overflow when multiplied against the buffer overhead.
When this overflow happens, we end up adding the overflow to the
amount of space available, causing the amount of memory allocated to
be larger than work_mem. Note that to reach this point, you have to
set work mem to at least 24GB and be sorting a set which is at least
that size. Given that a user who can set work_mem to 24GB could
also set it even higher, if they were looking to run the system out
of memory, this isn't considered a security issue. This overflow
risk was found by the Coverity scanner. Back-patch to all supported
branches, as this issue has existed since before 8.4.
http://git.postgresql.org/pg/commitdiff/273dcd16282c8014a14a9ecbf467459b8702e745

- Be sure to close() file descriptor on error case. In
receivelog.c:writeTimeLineHistoryFile(), we were not properly
closing the open'd file descriptor in error cases. While this
wouldn't matter much if we were about to exit due to such an error,
that's not the case with pg_receivexlog as it can be a long-running
process and these errors are non-fatal. This resource leak was
found by the Coverity scanner. Back-patch to 9.3 where this issue
first appeared.
http://git.postgresql.org/pg/commitdiff/cec62efd0e551a56635b47ea4185ec27a6840de7

- Fix resource leak in initdb -X option. When creating the symlink
for the xlog directory, free the string which stores the link
location. Not really an issue but it doesn't hurt to be good about
this- prior cleanups have fixed similar issues. Leak found by the
Coverity scanner. Not back-patching as I don't see it being worth
the code churn.
http://git.postgresql.org/pg/commitdiff/5461d36b5b7d99e237b2f63a7975e6430727cb0b

- Check version before allocating PQExpBuffer. In
pg_dump.c:getEventTriggers, check what major version we are on
before calling createPQExpBuffer() to avoid leaking that bit of
memory. Leak discovered by the Coverity scanner. Back-patch to 9.3
where support for dumping event triggers was added.
http://git.postgresql.org/pg/commitdiff/3355443fb188b86d59ca90912d5456b427c29116

== Rejected Patches (for now) ==

No one was disappointed this week :-)

== Pending Patches ==

KONDO Mitsumasa sent in two more revisions of a patch to improve the
checkpoint IO scheduler for stable transaction responses.

Hari Babu sent in two more revisions of a patch to compute Max LSN of
Data Pages.

Pavel Stehule sent in another revision of a patch to improve
performance of calculations with NUMERIC.

KaiGai Kohei sent in two more revisions of a patch to implement
row-level access control.

MauMau sent a patch intended to silence what could potentially be many
redundant error messages when statistics file is inaccessible.

James Sewell sent in another revision of a patch to add an ldapoption
to disable chasing LDAP referrals.

Bernd Helmle and Dean Rasheed traded patches to add tab completion to
write operations on both foreign tables and writeable views.

Dimitri Fontaine sent in two more revisions of a patch to implement
extension templates.

Pavel Stehule sent in two more revisions of a patch to add a simple
date constructor using integers.

Sawada Masahiko sent in two more revisions of a patch to enable
fail-back without creating a new backup on the previously failed node.

Kevin Grittner sent in another revision of a patch to implement
REFRESH MATERIALIZED VIEWS CONCURRENTLY.

Josh Kupershmidt sent in a patch to implement tab completion for
\lo_import.

ivan babrou sent in another revision of a patch to implement
millisecond precision for connect_timeout in libpq.

Fabien COELHO sent in another revision of a patch to implement
a progress indicator for pgbench.

Jeevan Chalke sent in a patch to fix a misbehavior in greedy regexes.

Dean Rasheed sent in a patch to let psql see any updateable relations,
whether they're domestic tables, foreign tables, or views.

Nicholas White sent in another revision of a patch to allow LEAD and
LAG window functions to ignore NULLs.

Gibheer sent in a patch to add a new GUC, replication_reserved_connections,
to reserve connection slots for replication in the same way
superuser_reserved_connections works for superusers.

Fabien COELHO sent in another revision of a patch to add a "big" test
to make check.

Michael Paquier sent in another revision of a patch to implement
REINDEX CONCURRENTLY.

Andres Freund sent in a patch to add an Assert to catcache.c.

Stas Kelvich sent in a patch to optimize storage and lookups in
contrib/cube.

Cedric Villemain sent in another revision of a patch to to document
some changes in VPATH builds.

Karol Trzcionka sent in another revision of a patch to implement
UPDATE ... RETURNING OLD/NEW.

Abhijit Menon-Sen sent in a patch to instrument an issue where an
INSERT wrecks plans on a table.

Peter Eisentraut sent in a patch to fix an issue where ECPG has two
slightly different prototypes.

Fabrízio de Royes Mello sent in another revision of a patch to add
support for "IF NOT EXISTS" to the remaining "CREATE" statements which
don't yet have it.

Greg Smith and Fabien COELHO traded patches to add a --throttle option
to pgbench.

Noah Misch sent in another revision of a patch to add FILTER for
aggregates.