== PostgreSQL Weekly News - December 12 2010 ==

== PostgreSQL Weekly News - December 12 2010 ==

The Call for Papers is still open for the PostgreSQL room at FOSDEM,
which will be held in Brussels, Belgium, on February 5-6, 2011. Send
your proposals to fosdem AT postgresql DOT eu.

Devrim GUNDUZ sent in a fix for the RHEL6 RPMs. See below for
details.
http://people.planetpostgresql.org/devrim/index.php?/archives/54-Attention-9.0.1-OR-RHEL-6-RPM-users.html

== PostgreSQL Product News ==

PostgreSQL Maestro 10.12, a GUI admin tool for PostgreSQL, released.
http://www.sqlmaestro.com/products/postgresql/maestro/

psycopg2 2.3.1, a Python connector for PostgreSQL, released.
http://initd.org/psycopg/articles/2010/12/04/psycopg-231-released/

Slony-I, a master-to-multiple-cascading-replica system for PostgreSQL,
released updated revisions 1.2.22 and 2.0.6.
http://www.slony.info/

== PostgreSQL Jobs for December ==

http://archives.postgresql.org/pgsql-jobs/2010-12/threads.php

== PostgreSQL Local ==

The PostgreSQLFr Call for Projects has begun. Projects must target
PostgreSQL and the French-speaking community. Mail appel-projets-2010
AT postgresql DOT fr.
http://www.postgresql.fr/appel_a_projets_2010:call_for_projects

FOSDEM is one of the biggest Free and Open Source event held annually
in Brussels, Belgium, on February 5-6, 2011, and attended by around
4000 people. Send in your proposal to fosdem AT postgresql DOT eu.
http://andreas.scherbaum.la/blog/archives/749-PostgreSQLFOSDEM-2011-Call-for-talks.html

PGDay is scheduled at this year's Southern California Linux Exposition
(SCALE) held in the LAX Hilton Hotel in the city of Los Angeles,
California, on Friday February 25th, 2011. Post your talk submission
to pgday-submissions AT googlegroups DOT com.

PGCon will be May 19-20, 2011 at the University of Ottawa, preceded by
two days of tutorials on May 17-18.
http://www.pgcon.org/2011/

== PostgreSQL in the News ==

Planet PostgreSQL: http://planet.postgresql.org/

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm Pacific time.
Please send English language ones to david(at)fetter(dot)org, German language
to pwn(at)pgug(dot)de, Italian language to pwn(at)itpug(dot)org(dot) Spanish language
to pwn(at)arpug(dot)com(dot)ar(dot)

== Reviews ==

ITAGAKI Takahiro reviewed the per-column collation patch, bringing up
issues with pg_dump, along with a couple of naming issues in the
source code and a request for clearer global state.
== Applied Patches ==

Peter Eisentraut pushed:

- Put only single space after "Sort Method:", for consistency.
http://git.postgresql.org/gitweb?p=postgresql.git;a=commitdiff;h=951d7861212120571a601d946194892db6f2bde1

Heikki Linnakangas pushed:

- Fix two typos, by Fujii Masao.
http://git.postgresql.org/gitweb?p=postgresql.git;a=commitdiff;h=95e42a2c2980003880dee622fc3bed0b03322542

- Fix bugs in the hot standby known-assigned-xids tracking logic. If
there's an old transaction running in the master, and a lot of
transactions have started and finished since, and a WAL-record is
written in the gap between the creating the running-xacts snapshot
and WAL-logging it, recovery will fail with "too many
KnownAssignedXids" error. This bug was reported by Joachim Wieland
on Nov 19th. In the same scenario, when fewer transactions have
started so that all the xids fit in KnownAssignedXids despite the
first bug, a more serious bug arises. We incorrectly initialize the
clog code with the oldest still running transaction, and when we see
the WAL record belonging to a transaction with an XID larger than
one that committed already before the checkpoint we're recovering
from, we zero the clog page containing the already committed
transaction, leading to data loss. In hindsight, trying to track
xids in the known-assigned-xids array before seeing the
running-xacts record was too complicated. To fix that, hold
XidGenLock while the running-xacts snapshot is taken and WAL-logged.
That ensures that no transaction can begin or end in that gap, so
that in recvoery we know that the snapshot contains all transactions
running at that point in WAL.
http://git.postgresql.org/gitweb?p=postgresql.git;a=commitdiff;h=5a031a5556ff83b8a9646892715d7fef415b83c3

Andrew Dunstan pushed:

- Allow the low level COPY routines to read arbitrary numbers of
fields. This doesn't involve any user-visible change in behavior,
but will be useful when the COPY routines are exposed to allow their
use by Foreign Data Wrapper routines, which will be able to use
these routines to read irregular CSV files, for example.
http://git.postgresql.org/gitweb?p=postgresql.git;a=commitdiff;h=af1a614ec6d074fdea46de2e1c462f23fc7ddc6f

Tom Lane pushed:

- Add a stack overflow check to copyObject(). There are some code
paths, such as SPI_execute(), where we invoke copyObject() on raw
parse trees before doing parse analysis on them. Since the bison
grammar is capable of building heavily nested parsetrees while
itself using only minimal stack depth, this means that copyObject()
can be the front-line function that hits stack overflow before
anything else does. Accordingly, it had better have a
check_stack_depth() call. I did a bit of performance testing and
found that this slows down copyObject() by only a few percent, so
the hit ought to be negligible in the context of complete processing
of a query. Per off-list report from Toshihide Katayama.
Back-patch to all supported branches.
http://git.postgresql.org/gitweb?p=postgresql.git;a=commitdiff;h=8b5692809707c0e15d04c530a3fed9347350ea01

- Force default wal_sync_method to be fdatasync on Linux. Recent
versions of the Linux system header files cause xlogdefs.h to
believe that open_datasync should be the default sync method,
whereas formerly fdatasync was the default on Linux. open_datasync
is a bad choice, first because it doesn't actually outperform
fdatasync (in fact the reverse), and second because we try to use
O_DIRECT with it, causing failures on certain filesystems (e.g.,
ext4 with data=journal option). This part of the patch is largely
per a proposal from Marti Raudsepp. More extensive changes are
likely to follow in HEAD, but this is as much change as we want to
back-patch. Also clean up confusing code and incorrect
documentation surrounding the fsync_writethrough option. Those
changes shouldn't result in any actual behavioral change, but I
chose to back-patch them anyway to keep the branches looking similar
in this area. In 9.0 and HEAD, also do some copy-editing on the WAL
Reliability documentation section. Back-patch to all supported
branches, since any of them might get used on modern Linux versions.
http://git.postgresql.org/gitweb?p=postgresql.git;a=commitdiff;h=576477e73c4a9cb32c7471c1cec354fd5328140e

- Clean up some copied-and-pasted code in pg_upgrade. 1. Don't
reimplement S_ISDIR() and S_ISREG() badly. 2. Don't reimplement
access() badly. This code appears to have been copied from ancient
versions of the corresponding backend routines, and not patched to
incorporate subsequent fixes (see my commits of 2008-03-31 and
2010-01-14 respectively). It might be a good idea to change it to
just *call* those routines, but for now I'll just transpose these
fixes over.
http://git.postgresql.org/gitweb?p=postgresql.git;a=commitdiff;h=663fc32e26e8df41434d751e2203c1aa410d1916

- Make S_IRGRP etc available in mingw builds as well as MSVC. (Hm, I
wonder whether BCC defines them either...) Also label dangling
endifs a bit better in this area.
http://git.postgresql.org/gitweb?p=postgresql.git;a=commitdiff;h=5132ad8bdf8e85ea0054217ef7ee38c51ace4824

- Reset all database-level stats in pgstat_recv_resetcounter(). We
were failing to zero out some pg_stat_database counters that have
been added since the initial pgstats coding. This is a bug, but not
back-patching the fix since changing this behavior in a minor
release seems a cure worse than the disease. Report and patch by
Tomas Vondra.
http://git.postgresql.org/gitweb?p=postgresql.git;a=commitdiff;h=0c90442355fbbe785740669f63141f24674c8958

Simon Riggs pushed:

- Optimize commit_siblings in two ways to improve group commit.
First, avoid scanning the whole ProcArray once we know there are at
least commit_siblings active; second, skip the check altogether if
commit_siblings = 0. Greg Smith.
http://git.postgresql.org/gitweb?p=postgresql.git;a=commitdiff;h=e620ee35b249b0af255ef788003d1c9edb815a35

- Reduce spurious Hot Standby conflicts from never-visible records.
Hot Standby conflicts only with tuples that were visible at some
point. So ignore tuples from aborted transactions or for tuples
updated/deleted during the inserting transaction when generating the
conflict transaction ids. Following detailed analysis and test case
by Noah Misch. Original report covered btree delete records,
correctly observed by Heikki Linnakangas that this applies to other
cases also. Fix covers all sources of cleanup records via common
code.
http://git.postgresql.org/gitweb?p=postgresql.git;a=commitdiff;h=b9075a6d2f9b07a00262a670dd60272904c79dce

- Self review of previous patch. Fix assumption that xmax >= xmin.
http://git.postgresql.org/gitweb?p=postgresql.git;a=commitdiff;h=9975c683b102d06ed5d5ab799eaba0d00a9ff38c

- Reduce spurious Hot Standby conflicts from never-visible records.
Hot Standby conflicts only with tuples that were visible at some
point. So ignore tuples from aborted transactions or for tuples
updated/deleted during the inserting transaction when generating the
conflict transaction ids. Following detailed analysis and test case
by Noah Misch. Original report covered btree delete records,
correctly observed by Heikki Linnakangas that this applies to other
cases also. Fix covers all sources of cleanup records via common
code. Includes additional fix compared to commit on HEAD
http://git.postgresql.org/gitweb?p=postgresql.git;a=commitdiff;h=a804a23e7af0e075b88e7b03bfd9b0f22d2657b2

Robert Haas pushed:

- Minor documentation cleanup. Fujii Masao
http://git.postgresql.org/gitweb?p=postgresql.git;a=commitdiff;h=1490946c57f44d3bf8601e9a404aaa80b58a715a

- Allow bidirectional copy messages in streaming replication mode.
Fujii Masao. Review by Alvaro Herrera, Tom Lane, and myself.
http://git.postgresql.org/gitweb?p=postgresql.git;a=commitdiff;h=d3d414696f39e2b57072fab3dd4fa11e465be4ed

Magnus Hagander pushed:

- Add required new port files to MSVC builds.
http://git.postgresql.org/gitweb?p=postgresql.git;a=commitdiff;h=20f396429186a35a81fc7ef3ad34c3134ead2992

== Rejected Patches (for now) ==

No one was disappointed this week :-)

== Pending Patches ==

Joachim Wieland sent in another revision of the patch to clone
snapshots and make them available.

Craig Ringer sent in a patch to add a crash dump, similar to a core
dump, only for Windows.

Josh Berkus sent in two revisions of a patch to make wal_sync_method
default to fdatasync on Linux kernels 2.6.33 and higher.

Fujii Masao sent in another revision of the patch to change libpq to
help with synchronous replication.

Fujii Masao sent in a patch to set a replication timeout. This, too,
is intended to help with synchronous replication.

Peter Eisentraut sent in a patch to add a pg_sequence_parameters()
function.

KaiGai Kohei sent in another revision of the access control label
switcher function.

Joachim Wieland sent in another revision of the patch to add a
directory-based format for pg_dump/pg_restore.

Fujii Masao sent in a patch to remove wal_sender_delay.

Tim Bunce sent in a patch to optimize argument-passing in PL/Perl[U].

Kevin Grittner sent in four revisions of a patch to add a optimize and
correct SERIALIZABLE so that it doesn't switch from READ ONLY
DEFERRABLE mode to READ WRITE.

Tom Lane sent in a patch to fix some infelicities in the speed of
calculating percent_rank() vs. rank().

Fujii Masao sent in a patch to fix the way ps(1) displays standby
queries.

Dimitri Fontaine sent in two more revisions to add extensions.

Robert Haas sent in another revision of the patch to add unlogged
tables.

Thomas (tv AT fuzzy DOT cz) sent in two revisions of a patch to keep a
timestamp of the most recent stats reset per-database, per-table, and
per-function.

Dimitri Fontaine sent in another revision of the patch to add a
pg_execute_from_file() function. This is part of the EXTENSIONS work.

Alexander Korotkov sent in a patch to add wildcard support for
pg_trgm.

Simon Riggs sent in a patch to add a NOT ENFORCED clause to foreign
key constraints.

Peter Eisentraut sent in another revision of the per-column collation
patch.

ITAGAKI Takahiro sent in another revision of the patch to add
file_fdw(), part of the SQL/MED standard dealing with reading and
writing external files.