== PostgreSQL Weekly News - April 05 2009 ==

== PostgreSQL Weekly News - April 05 2009 ==

Registration is now open for pgCon in Ottawa:
http://www.pgcon.org/2009/registration.php

Registration is now open for OSCON. There will be a pgDay the Sunday
before OSCON: https://en.oreilly.com/oscon2009/public/register

PgDay Florianopolis will be May 22. Contact Dickson S. Guedes (guedes
AT guedesoft DOT net) to participate or submit a paper.
http://www.postgresql.org.br/eventos/pgday/sc

== PostgreSQL Product News ==

Archiveopteryx 3.0.7 and 3.1.0, a PostgreSQL-based email handler, released.
http://www.archiveopteryx.org/3.0.7
http://www.archiveopteryx.org/3.1.0

PGORM 0.09, a .NET object-relational mapper for PostgreSQL, released.
http://pgorm.googlecode.com

PostgreDAC ver. 2.5.1 released.
http://microolap.com/products/connectivity/postgresdac/download/

== PostgreSQL 8.4 Feature of the Week ==

- CIText: David Wheeler added a new contrib module which implements a
full-featured Case Insensitive Text datatype.

== PostgreSQL Tip of the Week ==

- \h <COMMAND> in psql will give you the full documentation on a large
variety of SQL commands.

== PostgreSQL Jobs for April ==

http://archives.postgresql.org/pgsql-jobs/2009-04/threads.php

== PostgreSQL Local ==

Kevin Kempter will be speaking about backup and recover at Denver
PUG's first meeting on April 8, 2009.
http://www.diapug.org/Meeting.html

PgDay Brasilia will be April 17. Contact fernando.ike AT gmail DOT
com or eduardo DOT santos AT planejamento DOT gov DOT br to
participate.
http://www.postgresql.org.br/drupal6/eventos/pgday/df

Percona Performance Conference will be taking place at the Santa Clara
Convention Center, Santa Clara, California USA
http://conferences.percona.com/

CfP is open for PgDay Sao Paulo, which will be April 24. Contact
marins DOT consultoria AT gmail DOT com or marcelojscosta AT gmail DOT
com to participate.

PostgreSQL Conference, U.S. (JDCon) will be holding a PgDay at
LinuxFest Northwest (April 25/26th). The call for papers is out at
http://www.postgresqlconference.org/

There will also be PgDays on April 29 in Porto Velho, RO and on April
30 in Ji-Parana, RO. Contact Luis Fernando Bueno: proflfbueno AT
gmail DOT com to participate.

Michael Renner will be giving a PostgreSQL replication workshop at
Netways OSDC 2009 on April 29 and 30 in Nuremberg, Germany.
http://www.netways.de/english/osdc/y2009/programm/w/michael_renner_postgresql_repliziert_ein_ueberblick/

PGCon 2009 will be held 21-22 May 2009, in Ottawa at the University of
Ottawa. It will be preceded by two days of tutorials on 19-20 May
2009.
http://www.pgcon.org/2009/

Save The Date: pgDay San Jose. Sunday, July 19th 2009 immediately
before OSCON. CfP, more info TBA!

PGCon Brazil will be take place October 23-24 2009 at Unicamp in
Campinas, Sao Paulo state.

== PostgreSQL in the News ==

Planet PostgreSQL: http://planet.postgresql.org/

PostgreSQL Weekly News is brought to you this week by David Fetter
and Josh Berkus.

Submit news and announcements by Sunday at 3:00pm Pacific time.
Please send English language ones to david(at)fetter(dot)org, German language
to pwn(at)pgug(dot)de, Italian language to pwn(at)itpug(dot)org(dot)

== Applied Patches ==

Tom Lane committed:

- Fix an oversight in the support for storing/retrieving "minimal
tuples" in TupleTableSlots. We have functions for retrieving a
minimal tuple from a slot after storing a regular tuple in it, or
vice versa; but these were implemented by converting the internal
storage from one format to the other. The problem with that is it
invalidates any pass-by-reference Datums that were already fetched
from the slot, since they'll be pointing into the just-freed version
of the tuple. The known problem cases involve fetching both a
whole-row variable and a pass-by-reference value from a slot that is
fed from a tuplestore or tuplesort object. The added regression
tests illustrate some simple cases, but there may be other failure
scenarios traceable to the same bug. Note that the added tests
probably only fail on unpatched code if it's built with
--enable-cassert; otherwise the bug leads to fetching from freed
memory, which will not have been overwritten without additional
conditions. Fix by allowing a slot to contain both formats
simultaneously; which turns out not to complicate the logic much at
all, if anything it seems less contorted than before. Back-patch to
8.2, where minimal tuples were introduced.

- In pgsql/src/backend/optimizer/plan/planner.c, fix window function
plan generation to cope with volatile sort expressions. (Not clear
how useful these really are, but failing is no good...) Per report
from David Fetter and Robert Treat.

- Add PQinitOpenSSL() function to support applications that use
libcrypto but not OpenSSL (or perhaps vice versa, if that's
possible). Andrew Chernow, with minor editorialization by me.

- Add a "relistemp" boolean column to pg_class, which is true for
temporary relations (including a temp table's indexes and toast
table/index), and false for normal relations. For ease of checking,
this commit just adds the column and fills it correctly --- revising
the relation access machinery to use it will come separately.

- Modify the relcache to record the temp status of both local and
nonlocal temp relations; this is no more expensive than before, now
that we have pg_class.relistemp. Insert tests into bufmgr.c to
prevent attempting to fetch pages from nonlocal temp relations.
This provides a low-level defense against bugs-of-omission allowing
temp pages to be loaded into shared buffers, as in the
contrib/pgstattuple problem reported by Stuart Bishop. While at it,
tweak a bunch of places to use new relcache tests (instead of
expensive probes into pg_namespace) to detect local or nonlocal temp
tables.

- Fix contrib/pgstattuple and contrib/pageinspect to prevent attempts
to read temporary tables of other sessions; that is unsafe because
of the way our buffer management works. Per report from Stuart
Bishop. This is redundant with the bufmgr.c checks in HEAD, but not
at all redundant in the back branches.

- Remove last references to the crypt auth method, per Andreas 'ads'
Scherbaum.

- In pgsql/doc/src/sgml/func.sgml, index some array functions, per
Mario Splivalo.

- In pgsql/src/bin/pg_dump/pg_dump.c, improve pg_dump's query for
retrieving BLOB comments to be more efficient when there are many
blobs and not so many comments. Tamas Vincze.

- In pgsql/src/pl/plpgsql/src/pl_exec.c, plpgsql's exec_simple_cast_value()
mistakenly supposed that it could bypass casting effort whenever the
input value was NULL. However this prevents application of not-null
domain constraints in the cases that use this function, as
illustrated in bug #4741. Since this function isn't meant for use
in performance-critical paths anyway, this certainly seems like
another case of "premature optimization is the root of all evil".
Back-patch as far as 8.2; older versions made no effort to enforce
domain constraints here anyway.

- In pgsql/src/backend/utils/misc/guc.c, fix GUC's reports of
assign_hook failure to always include the parameter value we failed
to assign, even in "can't happen" cases. Motivated by wondering
what's going on in a recent trouble report where "failed to commit"
did happen.

- Fix SetClientEncoding() to maintain a cache of previously selected
encoding conversion functions. This allows transaction rollback to
revert to a previous client_encoding setting without doing fresh
catalog lookups. I believe that this explains and fixes the recent
report of "failed to commit client_encoding" failures. This bug is
present in 8.3.x, but it doesn't seem prudent to back-patch the fix,
at least not till it's had some time for field testing in HEAD. In
passing, remove SetDefaultClientEncoding(), which was used nowhere.

- In pgsql/src/pl/plpgsql/src/pl_exec.c, minor code
beautification/consolidation.

- Refactor ExecProject and associated routines so that fast-path code
is used for simple Var targetlist entries all the time, even when
there are other entries that are not simple Vars. Also, ensure that
we prefetch attributes (with slot_getsomeattrs) for all Vars in the
targetlist, even those buried within expressions. In combination
these changes seem to significantly reduce the runtime for cases
where tlists are mostly but not exclusively Vars. Per my proposal
of yesterday.

- Defend against possible crash if a plpython function does not
specify names for its arguments. Also add a regression test, since
someone apparently changed every single plpython test case to use
only named parameters; else we'd have noticed this sooner. Euler
Taveira de Oliveira, per a report from Alvaro Herrera.

- In pgsql/src/backend/storage/buffer/bufmgr.c, add a comment
documenting the question of whether PrefetchBuffer should try to
protect an already-existing buffer from being evicted. This was
left as an open issue when the posix_fadvise patch was committed.
I'm not sure there's any evidence to justify more work in this area,
but we should have some record about it in the source code.

- In pgsql/src/port/path.c, use (unsigned char) cast in argument of
pg_tolower(). Maybe it works on Windows without that, but we
shouldn't put bad examples where people might copy them. Also,
reformat slightly to improve the odds that pgindent won't go nuts on
this.

- In pgsql/src/bin/psql/help.c, add missing help output for \ef
option. Andrew (RhodiumToad) Gierth.

- In pgsql/doc/src/sgml/ref/psql-ref.sgml, minor wordsmithing on
descriptions of some \d commands.

- In pgsql/src/bin/psql/describe.c, make \dt \di and friends more
consistent about the treatment of TOAST tables and indexes; to wit,
never show either. (You can examine them with plain \d if you're
really so inclined.)

- In pgsql/src/bin/psql/describe.c, improve obsolete comment.

- Rewrite interval_hash() so that the hashcodes are equal for values
that interval_eq() considers equal. I'm not sure how that
fundamental requirement escaped us through multiple revisions of
this hash function, but there it is; it's been wrong since
interval_hash was first written for PG 7.1. Per bug #4748 from
Roman Kononov. Backpatch to all supported releases. This patch
changes the contents of hash indexes for interval columns. That's
no particular problem for PG 8.4, since we've broken on-disk
compatibility of hash indexes already; but it will require a
migration warning note in the next minor releases of all existing
branches: "if you have any hash indexes on columns of type interval,
REINDEX them after updating".

- A session that does not have any live snapshots does not have to be
waited for when we are waiting for old snapshots to go away during a
concurrent index build. In particular, this rule lets us avoid
waiting for idle-in-transaction sessions. This logic could be
improved further if we had some way to wake up when the session we
are currently waiting for goes idle-in-transaction. However that
would be a significantly more complex/invasive patch, so it'll have
to wait for some other day. Simon Riggs, with some improvements by
Tom.

- Remove the recently added node types ReloptElem and OptionDefElem in
favor of adding optional namespace and action fields to DefElem.
Having three node types that do essentially the same thing bloats
the code and leads to errors of confusion, such as in yesterday's
bug report from Khee Chin.

- Make an attempt at fixing our current Solaris 11 breakage: add a
configure probe for opterr (exactly like the one for optreset) and
have getopt.c define the variables only if configure doesn't find
them in libc.

- In pgsql/src/port/getopt.c, hmm, baiji thinks we need explicit
'extern' here.

- Remove contrib/intarray's definitions of the <@ and @> operators, so
that they don't cause confusion with the built-in anyarray versions
of those operators. Adjust the module's index opclasses to support
the built-in operators in place of the private ones. The private
implementations are still available under their historical names @
and ~, so no functionality is lost. Some quick testing suggests
that they offer no real benefit over the core operators, however.
Per a complaint from Rusty Conover.

- In pgsql/src/include/pg_config.h.win32, I had always wondered why
pg_config.h.win32 claimed that Windows provides optreset. Current
mastodon results prove that in fact it does not; it was only because
getopt.c defined the variable anyway that things failed to fall
over.

- Remove a boatload of useless definitions of 'int optreset'. If we
are using our own ports of getopt or getopt_long, those will define
the variable for themselves; and if not, we don't need these,
because we never touch the variable anyway.

- Change EXPLAIN output so that subplans and initplans (particularly
CTEs) are individually labeled, rather than just grouped under an
"InitPlan" or "SubPlan" heading. This in turn makes it possible for
decompilation of a subplan reference to usefully identify which
subplan it's referencing. I also made InitPlans identify which
parameter symbol(s) they compute, so that references to those
parameters elsewhere in the plan tree can be connected to the
initplan that will be executed. Per a gripe from Robert Haas about
EXPLAIN output of a WITH query being inadequate, plus some
longstanding pet peeves of my own.

- In pgsql/src/backend/executor/execQual.c, make ExecInitExpr build
the list of SubPlans found in a plan tree in order of discovery,
rather than reverse order. This doesn't matter functionally (I
suppose the previous coding dates from the time when lcons was
markedly cheaper than lappend). However now that EXPLAIN is
labeling subplans with IDs that are based on order of creation, this
may help produce a slightly less surprising printout.

- Change cardinality() into a C-code function, instead of a
SQL-language alias for array_length(v,1). The efficiency gain here
is doubtless negligible --- what I'm interested in is making sure
that if we have second thoughts about the definition, we will not
have to force a post-beta initdb to change the implementation.

Alvaro Herrera committed:

- In pgsql/doc/src/sgml/plpython.sgml, update URL to Python bug
tracker. Backpatch to 8.3; doesn't seem worthy of further
backpatch.

- Disallow setting fillfactor for TOAST tables. To implement this
without almost duplicating the reloption table, treat relopt_kind as
a bitmask instead of an integer value. This decreases the range of
allowed values, but it's not clear that there's need for that much
values anyway. This patch also makes heap_reloptions explicitly a
no-op for relation kinds other than heap and TOAST tables. Patch by
ITAGAKI Takahiro with minor edits from me. (In particular I removed
the bit about adding relation kind to an error message, which I
intend to commit separately.)

Bruce Momjian committed:

- In pgsql/doc/src/sgml/release.sgml, reorder release note sections.

- In pgsql/doc/src/sgml/release.sgml, more release note wording
improvements; section order adjustments.

- In pgsql/doc/src/sgml/release.sgml, more release note adjustments,
reordering.

- In pgsql/doc/src/sgml/release.sgml, more release note changes,
including a lower level of subsections.

- In pgsql/doc/src/sgml/release.sgml, more new subsections in release
notes.

- In pgsql/doc/src/sgml/release.sgml, update release note introductory
description.

- In pgsql/doc/src/sgml/release.sgml, remove some "Other" sections in
the release notes by putting the items at the top of their sections.

- Change psql \d* display so 'S' _or_ a pattern include system
objects.

- In pgsql/src/bin/psql/describe.c, do not show information_schema in
\d* commands, unless 'S' or pattern is specified. Martin Pihlak.

- Add support for additional DTrace probes. Robert Lor.

- Have PL/pgSQL FETCH set DIAGNOSTICS ROW_COUNT. Andrew (RhodiumToad)
Gierth.

- In pgsql/src/backend/utils/misc/guc.c, give a better error message
when trying to change "effective_io_concurrency" on systems without
posix_fadvise().

- Revert DTrace patch from Robert Lor.

- In pgsql/doc/src/sgml/config.sgml, document that Solaris can't use
effective_io_concurrency because of an ineffective posix_fadvise().

Heikki Linnakangas committed:

- In pgsql/src/backend/storage/ipc/procarray.c, fix a rare race
condition when commit_siblings > 0 and a transaction commits at the
same instant as a new backend is spawned. Since CountActiveBackends()
doesn't hold ProcArrayLock, it needs to be prepared for the case
that a pointer at the end of the proc array is still NULL even
though numProcs says it should be valid, since it doesn't hold
ProcArrayLock. Backpatch to 8.1. 8.0 and earlier had this right,
but it was broken in the split of PGPROC and sinval shared memory
arrays. Per report and proposal by Marko Kreen.

- In pgsql/src/backend/utils/adt/pg_locale.c, update comment to
reflect that LC_COLLATE and LC_CTYPE are now per-database settings.

Magnus Hagander committed:

- In pgsql/src/bin/initdb/initdb.c, don't crash initdb when we fail to
get the current username. Give an error message and exit instead,
like we do elsewhere... Per report from Wez Furlong and Robert
Treat.

- In pgsql/src/port/path.c, make directory name comparisons on Win32
case insensitive. This method will not catch all different ways
since the locale handling in NTFS doesn't provide an easy way to do
that, but it will hopefully solve the most common cases causing
startup problems when the backend is found in the system PATH.
Attempts to fix bug #4694.

Teodor Sigaev committed:

- In pgsql/contrib/hstore/hstore_io.c, fix memory allocation for
output of hstore type. Per report from Zhang Maosen.

- Fix infinite loop while checking of partial match in pending list.
Improve comments. Now GIN-indexable operators should be strict.
Per Tom Lane's questions/suggestions.

== Rejected Patches (for now) ==

Hitoshi Harada's doc patch which changed HeapTupleSatisfiesNow to
HeapTupleSatisfiesVisibility in doc/src/sgml/pgstattuple.sgml.

== Pending Patches ==

Abhijit Menon-Sen sent in a patch to implement has_sequence_privilege().

Andrew (RhodiumToad) Gierth sent in a patch to psql which adds a
mention of \ef to the \? command's output.

K. Srinath sent in a patch which allows an index to cover all tables
in an inheritance hierarchy.

Pavel Stehule and Tom Lane sent in a couple of versions of a patch to
allow raising an exception from _PG_init per previous discussion.

Heikki Linnakangas sent in a patch to help with message encoding.

Pavel Stehule sent in a patch to transform array inputs in variadic
functions into standard parameters.

Teodor Sigaev sent in two patches intended to fix a bug in GiST
reported by Andrew (RhodiumToad) Gierth.

Fujii Masao sent in another revision of his pg_standby trigger patch.

Robert Haas sent in a patch to add SET NDISTINCT to ALTER COLUMN.

Bruce Momjian sent in another revision of his patch to handle
platforms which don't support effective_io_concurrency.

Martin Pihlak sent in a patch which adds a U (user) modifier to psql's
\d commands in order to be able to match patterns only on non-system
objects.

Alvaro Herrera sent in a patch to fix a reloptions bug reported by
Khee Chin.

Teodor Sigaev sent in a patch to fix a crash in GiST insertion of
pathological box data.

Zdenek Kotala sent in another patch to fix the misbehavior of
Solaris's getopt.