== PostgreSQL Weekly News - March 27 2016 ==

== PostgreSQL Weekly News - March 27 2016 ==

== PostgreSQL Product News ==

Ora2PG 17.2 a migration system for moving from Oracle and MySQL to
PostgreSQL, released.
https://github.com/darold/ora2pg/blob/master/changelog

PostGIS 2.2.2, the industry standard geographic information
system for PostgreSQL, released.
http://postgis.net/2016/03/22/postgis-2.2.2/

PyGreSQL 5.0, a Python driver for PostgreSQL, released.
http://www.pygresql.org/contents/changelog.html

Veil 9.5.1, a fine-grained access control system for PostgreSQL,
released. Project page and download link below:
https://github.com/marcmunro/veil
http://pgfoundry.org/frs/shownotes.php?release_id=2119

== PostgreSQL Jobs for March ==

http://archives.postgresql.org/pgsql-jobs/2016-03/

== PostgreSQL Local ==

Nordic PGDay 2016 is a one day one track conference which will be held in
Helsinki, Finland, on March 17, 2016. Registration is still open.
http://2016.nordicpgday.org/

PGDay Paris 2016, on March 31 in Paris, has opened registration.
http://www.pgday.paris/registration/ (scroll down for English)

The 8th PostgreSQL Session will be held on April 6th, 2016, in Lyon,
France.
<AT> postgresql-sessions <DOT> org.

The first meeting of PostgreSQL Israel will be taking place in Tel
Aviv on April 7, 2016.
http://www.meetup.com/PostgreSQL-Israel/events/229430729/

PGConf US 2016 will take place April 18-20, 2016 in NYC. Registration
is open.
http://www.pgconf.us/2016/

LinuxFest Northwest will take place April 23-24, 2016 at Bellingham
Technical College in Bellingham, Washington, USA. The CfP is now
open.
http://www.linuxfestnorthwest.org/2016/present

FOSS4G NA, will be held May 2-5, 2016 in Raleigh, North Carolina.
The CfP is still open.
https://2016.foss4g-na.org/cfp

PGCon 2016 will be held May 17-21, 2016 in Ottawa.
http://www.pgcon.org/

This year's Swiss PGDay will be held on June 24, 2016 at the
University of Applied Sciences in Rapperswil (Switzerland).
The CfP is open.
http://www.pgday.ch/

"5432 ... Meet us!", will take place in Milan, Italy on June 28-29, 2016.
The CfP is open until March 14, 2016.
http://5432meet.us/

PG Day UK 2016 - Call for Papers is open until 5th April 2016. The
conference will be 5th July 2016. Please submit proposals to
Simon(dot)Riggs(at)2ndquadrant(dot)com <Simon AT 2ndquadrant DOT com> or via the
website at:
http://www.pgconf.uk/papers

PgConf Silicon Valley 2016 will be held on November 14-16, 2016.
http://www.pgconfsv.com/

== PostgreSQL in the News ==

Planet PostgreSQL: http://planet.postgresql.org/

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm Pacific time.
Please send English language ones to david(at)fetter(dot)org, German language
to pwn(at)pgug(dot)de, Italian language to pwn(at)itpug(dot)org(dot) Spanish language
to pwn(at)arpug(dot)com(dot)ar(dot)

== Applied Patches ==

Tom Lane pushed:

- Teach the configure script to validate its --with-pgport argument.
Previously, configure would take any string, including an empty
string, leading to obscure compile failures in guc.c. It seems
worth expending a few lines of code to ensure that the argument is a
decimal number between 1 and 65535. Report and patch by Jim Nasby;
reviews by Alex Shulgin, Peter Eisentraut, Ivan Kartyshov
http://git.postgresql.org/pg/commitdiff/bf53d5c208a3bdce243a38666fc50f5418c78c3b

- pg_stat_get_progress_info() should be marked STRICT. I didn't
bother with a catversion bump. Report and patch by Thomas Munro
http://git.postgresql.org/pg/commitdiff/2da75499879032d8d2f233ca42cc2efe48fd76ef

- Use repalloc_huge() to enlarge a SPITupleTable's tuple pointer
array. Commit 23a27b039d94ba35 widened the rows-stored counters to
uint64, but that's academic unless we allow the tuple pointer array
to exceed 1GB. (It might be a good idea to provide some other limit
on how much storage a SPITupleTable can eat. On the other hand,
there are plenty of other ways to drive a backend into swap hell.)
Dagfinn Ilmari Mannsåker
http://git.postgresql.org/pg/commitdiff/74a379b984d4df91acec2436a16c51caee3526af

- Improve conversions from uint64 to Perl types. Perl's integers are
pointer-sized, so can hold more than INT_MAX on LP64 platforms, and
come in both signed (IV) and unsigned (UV). Floating point values
(NV) may also be larger than double. Since Perl 5.19.4 array
indices are SSize_t instead of I32, so allow up to SSize_t_max on
those versions. The limit is not imposed just by av_extend's
argument type, but all the array handling code, so remove the
speculative comment. Dagfinn Ilmari Mannsåker
http://git.postgresql.org/pg/commitdiff/f3f3aae4b7841f4dc51129691a7404a03eb55449

- Update PL/Perl's comment about hv_store(). Negative klen is
documented since Perl 5.16, and 5.6 is no longer supported so no
need to comment about it. Dagfinn Ilmari Mannsåker
http://git.postgresql.org/pg/commitdiff/07341a2980a37ccbb3a51af2bd2f3c87953d8ea4

- Rethink representation of PathTargets. In commit 19a541143a09c067 I
did not make PathTarget a subtype of Node, and embedded a
RelOptInfo's reltarget directly into it rather than having a
separately-allocated Node. In hindsight that was misguided
micro-optimization, enabled by the fact that at that point we didn't
have any Paths with custom PathTargets. Now that PathTarget
processing has been fleshed out some more, it's easier to see that
it's better to have PathTarget as an indepedent Node type, even if
it does cost us one more palloc to create a RelOptInfo. So change
it while we still can. This commit just changes the representation,
without doing anything more interesting than that.
http://git.postgresql.org/pg/commitdiff/307c78852f516042cebacaed411a0391bfeb2129

- Allow callers of create_foreignscan_path to specify nondefault
PathTarget. Although the default choice of rel->reltarget should
typically be sufficient for scan or join paths, it's not at all
sufficient for the purposes PathTargets were invented for; in
particular not for upper-relation Paths. So break API compatibility
by adding a PathTarget argument to create_foreignscan_path(). To
ease updating of existing code, accept a NULL value of the argument
as selecting rel->reltarget.
http://git.postgresql.org/pg/commitdiff/28048cbaa285b8ac46940e4b39f985d9885fc698

- Provide a planner hook at a suitable place for creating upper-rel
Paths. In the initial revision of the upper-planner pathification
work, the only available way for an FDW or custom-scan provider to
inject Paths representing post-scan-join processing was to insert
them during scan-level GetForeignPaths or similar processing. While
that's not impossible, it'd require quite a lot of duplicative
processing to look forward and see if the extension would be capable
of implementing the whole query. To improve matters for custom-scan
providers, provide a hook function at the point where the core code
is about to start filling in upperrel Paths. At this point Paths
are available for the whole scan/join tree, which should reduce the
amount of redundant effort considerably. (An alternative design
that was suggested was to provide a separate hook for each
post-scan-join processing step, but that seems messy and not clearly
more useful.) Following our time-honored tradition, there's no
documentation for this hook outside the source code. As-is, this
hook is only meant for custom scan providers, which we can't assume
very much about. A followon patch will implement an FDW callback to
let FDWs do the same thing in a somewhat more structured fashion.
http://git.postgresql.org/pg/commitdiff/5864d6a4b62ada2ad60a8c456b4ee62972a9c10d

- Add a GetForeignUpperPaths callback function for FDWs. This is
basically like the just-added create_upper_paths_hook, but control
is funneled only to the FDW responsible for all the baserels of the
current query; so providing such a callback is much less likely to
add useless overhead than using the hook function is. The
documentation is a bit sketchy. We'll likely want to improve it,
and/or adjust the call conventions, when we get some experience with
actually using this callback. Hopefully somebody will find time to
experiment with it before 9.6 feature freeze.
http://git.postgresql.org/pg/commitdiff/101fd9349eddb7e9ed84a239145d5230a9bc7336

- Cope if platform declares mbstowcs_l(), but not locale_t, in
<xlocale.h>. Previously, we included <xlocale.h> only if necessary
to get the definition of type locale_t. According to notes in
PGAC_TYPE_LOCALE_T, this is important because on some versions of
glibc that file supplies an incompatible declaration of locale_t.
(This info may be obsolete, because on my RHEL6 box that seems to be
the *only* definition of locale_t; but there may still be glibc's in
the wild for which it's a live concern.) It turns out though that on
FreeBSD and maybe other BSDen, you can get locale_t from stdlib.h or
locale.h but mbstowcs_l() and friends only from <xlocale.h>. This
was leaving us compiling calls to mbstowcs_l() and friends with no
visible prototype, which causes a warning and could possibly cause
actual trouble, since it's not declared to return int. Hence,
adjust the configure checks so that we'll include <xlocale.h> either
if it's necessary to get type locale_t or if it's necessary to get a
declaration of mbstowcs_l(). Report and patch by Aleksander
Alekseev, somewhat whacked around by me. Back-patch to all
supported branches, since we have been using mbstowcs_l() since 9.1.
http://git.postgresql.org/pg/commitdiff/0e9b89986b7ced6daffdf14638a25a35c45423ff

- Be more careful about out-of-range dates and timestamps. Tighten
the semantics of boundary-case timestamptz so that we allow
timestamps >= '4714-11-24 00:00+00 BC' and < 'ENDYEAR-01-01 00:00+00
AD' exactly, no more and no less, but it is allowed to enter
timestamps within that range using non-GMT timezone offsets (which
could make the nominal date 4714-11-23 BC or ENDYEAR-01-01 AD).
This eliminates dump/reload failure conditions for timestamps near
the endpoints. To do this, separate checking of the inputs for
date2j() from the final range check, and allow the Julian date code
to handle a range slightly wider than the nominal range of the
datatypes. Also add a bunch of checks to detect out-of-range dates
and timestamps that formerly could be returned by operations such as
date-plus-integer. All C-level functions that return date,
timestamp, or timestamptz should now be proof against returning a
value that doesn't pass IS_VALID_DATE() or IS_VALID_TIMESTAMP().
Vitaly Burovoy, reviewed by Anastasia Lubennikova, and substantially
whacked around by me
http://git.postgresql.org/pg/commitdiff/a70e13a39eccf5fc944c66e0029004b6abcb3cae

- Fix j2day() to behave sanely for negative Julian dates. Somebody
had apparently once figured that casting to unsigned int would
produce the right output for negative inputs, but that would only be
true if 2^32 were a multiple of 7, which of course it ain't. We
need to use a signed division and then correct the sign of the
remainder. AFAICT, the only case where this would arise currently
is when doing ISO-week calculations for dates in 4714BC, where we'd
compute a negative Julian date representing 4714-01-04BC and then do
some arithmetic with it. Since we don't even really document
support for such dates, this is not of much consequence. But we may
as well get it right. Per report from Vitaly Burovoy.
http://git.postgresql.org/pg/commitdiff/5db51464311eb7fe4e90030c6a514ff61e9f1c00

- Fix "pg_bench -C -M prepared". This didn't work because when we
dropped and re-established a database connection, we did not bother
to reset session-specific state such as the statements-are-prepared
flags. The st->prepared[] array certainly needs to be flushed, and
I cleared a couple of other fields as well that couldn't possibly
retain meaningful state for a new connection. In passing, fix some
bogus comments and strange field order choices. Per report from
Robins Tharakan.
http://git.postgresql.org/pg/commitdiff/47211af17a2dbee38b53b2ea6de81499dbb2c7f5

- Fix assorted breakage in to_char()'s OF format option. In HEAD, fix
incorrect field width for hours part of OF when tm_gmtoff is
negative. This was introduced by commit 2d87eedc1d4468d3 as a
result of falsely applying a pattern that's correct when + signs are
omitted, which is not the case for OF. In 9.4, fix missing abs()
call that allowed a sign to be attached to the minutes part of OF.
This was fixed in 9.5 by 9b43d73b3f9bef27, but for inscrutable
reasons not back-patched. In all three versions, ensure that the
sign of tm_gmtoff is correctly reported even when the GMT offset is
less than 1 hour. Add regression tests, which evidently we
desperately need here. Thomas Munro and Tom Lane, per report from
David Fetter
http://git.postgresql.org/pg/commitdiff/55c3a04d60ccea9e999088fb847ceeb9fd4dd927

- Remove useless double calls of make_parsestate(). Aleksander
Alekseev
http://git.postgresql.org/pg/commitdiff/bd0ab28912d7502b237b8aeb95d052abe4ff6bc6

- Clean up some misplaced #includes. Random .h files have no business
including postgres-fe.h (or postgres.h). If that wasn't the first
#include done by the calling .c file, it's the .c file that's
broken. Noted while prepping Kyotaro Horiguchi's psql lexer
refactoring patch.
http://git.postgresql.org/pg/commitdiff/3422fecccadb021b7b4cdbc73b2c29f66f031761

- Decouple psqlscan.l from surrounding program. Remove assorted
external references from psqlscan.l in preparation for making it
usable by other frontend programs. This mostly involves getting rid
of direct calls to psql_error() and GetVariable() in favor of
introducing a callback-functions struct to encapsulate variable
fetching and error printing. In addition, pass the current encoding
and standard-strings status as additional parameters to
psql_scan_setup instead of looking directly at "pset" or calling
additional functions. I did not bother to change some references to
psql_error that are in functions that will soon migrate to a
psql-specific backslash-command lexer. Other than that, this
version of psqlscan.l is capable of compiling standalone. It still
depends on assorted src/common functions as well as some
encoding-related libpq functions, but we expect that all programs
using it will be happy with those dependencies. Kyotaro Horiguchi,
somewhat editorialized on by me
http://git.postgresql.org/pg/commitdiff/4e1d2a170836028370675922ea9a690648d3c18d

- Convert psql's flex lexer to be re-entrant, and make it compile
standalone. Change psqlscan.l to specify '%option reentrant',
adjust internal APIs to match, and get rid of its internal static
variables. While this is good cleanup in an abstract sense, the
reason to do it right now is that it seems the only practical way to
support use of separate flex lexers with common PsqlScanState
infrastructure. If we build two non-reentrant lexers then we are
going to have problems with dangling buffer pointers in whichever
lexer isn't active when we transition from one buffer to another, as
well as curious side-effects if we try to share any code between the
files. (Horiguchi-san had a different solution to that in his
pending patch, but I find it ugly and probably broken for corner
cases.) Depending on which version of flex you're using, this may
result in getting a "warning: unused variable 'yyg'" warning from
psqlscan, similar to the one you'd have seen for a long time in
backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the
file, for the convenience of those who compile with -Werror. Also,
stop compiling psqlscan as part of mainloop.c, and make it a
standalone build target instead. This is a lot cleaner than before,
though it doesn't really change much in practice as of this commit.
(I'm not sure whether the MSVC build scripts will need some help
with this part, but the buildfarm will soon tell us.)
http://git.postgresql.org/pg/commitdiff/27199058d98ef7ff2f468af44654bc35bb70fe4a

- Split psql's lexer into two separate .l files for SQL and backslash
cases. This gets us to a point where psqlscan.l can be used by
other frontend programs for the same purpose psql uses it for, ie to
detect when it's collected a complete SQL command from input that is
divided across line boundaries. Moreover, other programs can supply
their own lexers for backslash commands of their own choosing. A
follow-on patch will use this in pgbench. The end result here is
roughly the same as in Kyotaro Horiguchi's
0001-Make-SQL-parser-part-of-psqlscan-independent-from-ps.patch,
although the details of the method for switching between lexers are
quite different. Basically, in this patch we share the entire
PsqlScanState, YY_BUFFER_STATE stack, *and* yyscan_t between
different lexers. The only thing we need to do to switch to a
different lexer is to make sure the start_state is valid for the new
lexer. This works because flex doesn't keep any other persistent
state that depends on the specific lexing tables generated for a
particular .l file. (We are assuming that both lexers are built
with the same flex version, or at least versions that are compatible
with respect to the contents of yyscan_t; but that doesn't seem
likely to be a big problem in practice, considering how slowly flex
changes.) Aside from being more efficient than Horiguchi-san's
original solution, this avoids possible corner-case changes in
semantics: the original code was capable of popping the input buffer
stack while still staying in backslash-related parsing states. I'm
not sure that that equates to any useful user-visible behaviors, but
I'm not sure it doesn't either, so I'm loath to assume that we only
need to consider the topmost buffer when parsing a backslash
command. I've attempted to update the MSVC build scripts for the
added .l file, but will rely on the buildfarm to see if I missed
anything. Kyotaro Horiguchi and Tom Lane
http://git.postgresql.org/pg/commitdiff/0ea9efbe9ec1bf07cc6ae070bdd54700af08e44d

- Suppress FLEX_NO_BACKUP check for psqlscanslash.l. The existing
infrastructure for FLEX_NO_BACKUP doesn't work reliably when two
lexers are built in parallel in the same directory. We can probably
fix that, but as a short-term workaround, just don't make the check
for psqlscanslash.l. Per buildfarm.
http://git.postgresql.org/pg/commitdiff/a3e39f83632935911bc159154a33e89495f4a676

- Use yylex_init not yylex_init_extra(). Older versions of flex don't
have the latter. Per buildfarm.
http://git.postgresql.org/pg/commitdiff/ff0a7e6167f475672d82d1cd7cd0d5e735154c4d

- Fix missed update in _readForeignScan(). Blatant fail in
0bf3ae88af330496517722e391e7c975e6bad219. Caught by buildfarm
member mandrill.
http://git.postgresql.org/pg/commitdiff/07aed46a6b3994508e5674301c85ebf5807905ea

- With ancient gcc, skip pg_attribute_printf() on function pointer.
Buildfarm results show that the ability to attach
pg_attribute_printf decoration to a function pointer appeared
somewhere between gcc 2.95.3 and gcc 4.0.1. Guess that it was there
in 4.0.
http://git.postgresql.org/pg/commitdiff/b46d9beb658af7eb4e2a08dfa34206a117c9654f

- Build backend/parser/scan.l and interfaces/ecpg/preproc/pgc.l
standalone. Now that we know about the %top{} trick, we can revert
to building flex lexers as separate .o files. This is worth doing
for a couple of reasons besides sheer cleanliness. We can narrow
the scope of the -Wno-error flag that's forced on scan.c. Also,
since these grammar and lexer files are so large, splitting them
into separate build targets should have some advantages in build
speed, particularly in parallel or ccache'd builds. We have quite a
few other .l files that could be changed likewise, but the above
arguments don't apply to them, so the benefit of fixing them seems
pretty minimal. Leave the rest for some other day.
http://git.postgresql.org/pg/commitdiff/72b1e3a21f0540ffa5c1f8f474b6c52097a368bb

- Typo fix.
http://git.postgresql.org/pg/commitdiff/78e7c4439917b01afd645a2ec657008ba6c33d37

- Sync backend/parser/scan.l with bin/psql/psqlscan.l. Make some
minor formatting adjustments to make it easier to diff these files
and see that they indeed implement the same flex rules (at least to
the extent that we want them to be the same). (Someday it'd be nice
to make ecpg's pgc.l more easily diff'able too, but today is not
that day.) Also run relevant parts of these files and
psqlscanslash.l through pgindent. No actual behavioral changes
here, just obsessive neatnik-ism.
http://git.postgresql.org/pg/commitdiff/21c8ee79464a180ab0257abdfceae89274a46632

- Fix phony .PHONY. A couple makefiles had misspelled the magic
.PHONY target as PHONY.
http://git.postgresql.org/pg/commitdiff/d5351fcb03fc8e20651d5863b88b397a8be68d74

- SQL commands in pgbench scripts are now ended by semicolons, not
newlines. To allow multiline SQL commands in scripts, adopt the
same rules psql uses to decide what is the end of a SQL command, to
wit, an unquoted semicolon not encased in parentheses. Do this by
importing the same flex lexer that psql uses, since coping with
stuff like dollar-quoted literals is hard to get right without going
the full nine yards. This makes use of the infrastructure added in
commit 0ea9efbe9ec1bf07 to support independently-written flex lexers
scanning the same PsqlScanState input-buffer data structure. Since
that infrastructure isn't very friendly to ad-hoc parsing code such
as strtok(), improve exprscan.l so that it can parse either
whitespace-separated words or expression tokens, on demand, and
rewrite pgbench.c's backslash-command parsing code to always use the
lexer to fetch tokens. It's still the case that pgbench backslash
commands extend to the end of the line, no more and no less. That
could be changed in a fairly localized way now, and there was some
interest in doing so, but it seems like material for a separate
patch. In passing, make some marginal cleanups in syntax error
reporting, const-ify a few data structures that could use it, and
run some of this code through pgindent. I can't tell whether the
MSVC build scripts need to be taught explicitly about the changes
here or not, but the buildfarm will soon tell us. Kyotaro Horiguchi
and Tom Lane
http://git.postgresql.org/pg/commitdiff/68ab8e8ba4a471d91b69f2f89782ba10a0fbef0c

- Make pgbench's expression lexer reentrant. This is a necessary
preliminary step for making it play with psqlscan.l given the way I
set up the lexer input-buffer sharing mechanism in commit
0ea9efbe9ec1bf07. I've not tried to make it *actually* reentrant;
there's still some static variables laying about. But flex thinks
it's reentrant, and that's what counts. In support of that, fix
exprparse.y to pass through the yyscan_t from the caller. Also do
some minor code beautification, like not casting away const.
http://git.postgresql.org/pg/commitdiff/429ee5a822db0e8faf669d77c810f1eeaaff1ab4

- Best-guess attempt at fixing MSVC build for 68ab8e8ba4a471d9.
pgbench now needs to use src/bin/psql/psqlscan.l, but it's not very
clear how to fit that into the MSVC build system. If this doesn't
work I'm going to need some help from somebody who actually
understands those scripts ...
http://git.postgresql.org/pg/commitdiff/6f1f34c92b11593ec62ff3e12781eb96dc911821

- Use %option bison-bridge in psql/pgbench lexers. The point of this
change is to use %pure-parser in pgbench's exprparse.y. The
immediate reason is that it turns out very ancient versions of bison
have a bug with the combination of a reentrant lexer and
non-reentrant parser. We could consider dropping support for such
ancient bisons; but considering that we might well need exprparse.y
to be reentrant some day, it seems better to make it so right now
than to move the portability goalposts. (AFAICT there's no
particular performance consequence to this change, either, so
there's no good reason not to do it.) Now, %pure-parser assumes that
the called lexer is built with %option bison-bridge. Because we're
assuming bitwise compatibility of yyscan_t (yyguts_t) data
structures among all the psql/pgbench lexers, that requirement
propagates back to psql's lexers as well. But it's just a few lines
of change on that side too; and if psqlscan.l is to set the baseline
for a possibly-large family of lexers, it should err on the side of
including not omitting useful features.
http://git.postgresql.org/pg/commitdiff/b6afae71aaf6d2df76d0a0a77c8b630220a01ec1

- lean up some Coverity complaints about commit 0bf3ae88af330496. The
two get_tle_by_resno() calls introduced by this commit lacked any
check for a NULL return, unlike any other calls of that function
anywhere in our tree. Coverity quite properly complained about it.
Also fix a misindented line in process_query_params(), which
Coverity also complained about on the grounds that the bad
indentation suggested possible programmer misinterpretation.
http://git.postgresql.org/pg/commitdiff/92b7902deb3155f6975f33e8b6c8be4d9d066172

- Allow the delay in psql's \watch command to be a fractional second.
Instead of just "2" seconds, allow eg. "2.5" seconds. Per request
from Alvaro Herrera. No docs change since the docs didn't say you
couldn't do this already.
http://git.postgresql.org/pg/commitdiff/b283096534b9c514a92a70c98c033015b6792ba7

- Improve header output from psql's \watch command. Include the \pset
title string if there is one, and shorten the prefab part of the
header to be "timestamp (every Ns)". Per suggestion by David
Johnston. Michael Paquier and Tom Lane
http://git.postgresql.org/pg/commitdiff/dea2b5960a9460c02896ed361d35e92bce02801a

- Fix EvalPlanQual bug when query contains both locked and not-locked
rels. In commit afb9249d06f47d7a, we (probably I) made ExecLockRows
assign null test tuples to all relations of the query while setting
up to do an EvalPlanQual recheck for a newly-updated locked row.
This was sheerest brain fade: we should only set test tuples for
relations that are lockable by the LockRows node, and in particular
empty test tuples are only sensible for inheritance child relations
that weren't the source of the current tuple from their inheritance
tree. Setting a null test tuple for an unrelated table causes it to
return NULLs when it should not, as exhibited in bug #14034 from
Bronislav Houdek. To add insult to injury, doing it the wrong way
required two loops where one would suffice; so the corrected code is
even a bit shorter and faster. Add a regression test case based on
his example, and back-patch to 9.5 where the bug was introduced.
http://git.postgresql.org/pg/commitdiff/71404af2a29ce4a3a5907cdc8b893ec2bc0285b4

- Fix unsafe use of strtol() on a non-null-terminated Text datum.
jsonb_set() could produce wrong answers or incorrect error reports,
or in the worst case even crash, when trying to convert a path-array
element into an integer for use as an array subscript. Per report
from Vitaly Burovoy. Back-patch to 9.5 where the faulty code was
introduced (in commit c6947010ceb42143). Michael Paquier
http://git.postgresql.org/pg/commitdiff/384dfbde19330541f7fb487f9352949aa06c812e

- Code review for error reports in jsonb_set(). User-facing (even
tested by regression tests) error conditions were thrown with
elog(), hence had wrong SQLSTATE and were untranslatable. And the
error message texts weren't up to project style, either.
http://git.postgresql.org/pg/commitdiff/ea4b8bd6188ecb17ba37d93f57b8b020a964e66c

- Move keywords.c/kwlookup.c into src/common/. Now that we have
src/common/ for code shared between frontend and backend, we can get
rid of (most of) the klugy ways that the keyword table and keyword
lookup code were formerly shared between different uses. This is a
first step towards a more general plan of getting rid of
special-purpose kluges for sharing code in src/bin/. I chose to
merge kwlookup.c back into keywords.c, as it once was, and always
has been so far as keywords.h is concerned. We could have kept them
separate, but there is noplace that uses ScanKeywordLookup without
also wanting access to the backend's keyword list, so there seems
little point. ecpg is still a bit weird, but at least now the
trickiness is documented. I think that the MSVC build script should
require no adjustments beyond what's done here ... but we'll soon
find out.
http://git.postgresql.org/pg/commitdiff/2c6af4f44228d76d3351fe26f68b00b55cdd239a

- Avoid PGDLLIMPORT for simple local references in frontend programs.
I was wondering if this would be an issue, and buildfarm member
frogmouth says it is.
http://git.postgresql.org/pg/commitdiff/c2d1eea9e750edb267e3f071a129e03d79ad198b

- Create src/fe_utils/, and move stuff into there from pg_dump's
dumputils. Per discussion, we want to create a static library and
put the stuff into it that until now has been shared across src/bin/
directories by ad-hoc methods like symlinking a source file. This
commit creates the library and populates it with a couple of files
that contain the widely-useful portions of pg_dump's dumputils.c
file. dumputils.c survives, because it has some stuff that didn't
seem appropriate for fe_utils, but it's significantly smaller and is
no longer referenced from any other directory. Follow-on patches
will move more stuff into fe_utils. The Mkvcbuild.pm hacking here
is just a best guess; we'll see how the buildfarm likes it.
http://git.postgresql.org/pg/commitdiff/588d963b00e5e4385b6425418e3faa726f63f72e

- Add missed inclusion requirement in Mkvcbuild.pm. Per buildfarm.
http://git.postgresql.org/pg/commitdiff/0ecd3fedfcf3427ebeb73cc61b2fcf6ed67c43a2

- Suppress compiler warning for get_am_type_string(). Compilers that
don't know that elog(ERROR) doesn't return complained that this
function might fail to return a value. Per buildfarm. While at it,
const-ify the function's declaration, since the intent is evidently
to always return a constant string.
http://git.postgresql.org/pg/commitdiff/a376960c8f8ec08783e1c529f36fbeb60236b378

- Move psql's print.c and mbprint.c into src/fe_utils. Just turning
the crank ...
http://git.postgresql.org/pg/commitdiff/d65bea26a867e3bbd053bf87b985b0e113256414

- Move psql's psqlscan.l into src/fe_utils. This completes (at least
for now) the project of getting rid of ad-hoc linkages among the
src/bin/ subdirectories. Everything they share is now in
src/fe_utils/ and is included from a static library at link time. A
side benefit is that we can restore the FLEX_NO_BACKUP check for
psqlscanslash.l. We might need to think of another way to do that
check if we ever need to build two lexers with that property in the
same source directory, but there's no foreseeable reason to need
that.
http://git.postgresql.org/pg/commitdiff/c1156411ad0879a71956b64aa487babe7572685b

- Link libpq after libpgfeutils to satisfy Windows linker. Some of
the non-MSVC Windows buildfarm members seem to need this to avoid
getting "undefined symbol" errors on libpgfeutils' references to
libpq. I could understand that if libpq were a static library, but
surely it is not? Oh well, at least the extra reference is no more
harmful than it is for libpgcommon or libpgport.
http://git.postgresql.org/pg/commitdiff/7caaeaf3607fae91318f24debce3dc017ca299a3

- Don't split up SRFs when choosing to postpone SELECT output
expressions. In commit 9118d03a8cca3d97 we taught the planner to
postpone evaluation of set-returning functions in a SELECT's
targetlist until after any sort done to satisfy ORDER BY. However,
if we postpone some SRFs this way while others do not get postponed
(because they're sort or group key columns) we will break the
traditional behavior by which all SRFs in the tlist run in-step
during ExecTargetList(), so that you get the least common multiple
of their periods not the product. Fix make_sort_input_target() so
it will not split up SRF evaluation in such cases. There is still a
hazard of similar odd behavior if there's a SRF in a grouping column
and another one that isn't, but that was true before and we're just
trying to preserve bug-compatibility with the traditional behavior.
This whole area is overdue to be rethought and reimplemented, but
we'll try to avoid changing behavior until then. Per report from
Regina Obe.
http://git.postgresql.org/pg/commitdiff/d543170f2fdd6d9845aaf91dc0f6be7a2bf0d9e7

- Fix DROP OPERATOR to reset oprcom/oprnegate links to the dropped
operator. This avoids leaving dangling links in pg_operator; which
while fairly harmless are also unsightly. While we're at it,
simplify OperatorUpd, which went through heap_modify_tuple for no
very good reason considering it had already made a tuple copy it
could just scribble on. Roma Sokolov, reviewed by Tomas Vondra,
additional hacking by Robert Haas and myself.
http://git.postgresql.org/pg/commitdiff/c94959d4110a1965472956cfd631082a96f64a84

- In PL/Tcl, make database errors return additional info in the
errorCode. Tcl has a convention for returning additional info about
an error in a global variable named errorCode. Up to now PL/Tcl has
ignored that, but this patch causes database errors caught by PL/Tcl
to fill in errorCode with useful information from the ErrorData
struct. Jim Nasby, reviewed by Pavel Stehule and myself
http://git.postgresql.org/pg/commitdiff/fb8d2a7f57d87102f0a95025fbf1cad9c341739b

- Improve PL/Tcl errorCode facility by providing decoded name for
SQLSTATE. We don't really want to encourage people to write numeric
SQLSTATEs in programs; that's unreadable and error-prone. Copy
plpgsql's infrastructure for converting between SQLSTATEs and
exception names shown in Appendix A, and modify examples in tests
and documentation to do it that way.
http://git.postgresql.org/pg/commitdiff/cd37bb78599dcf24cd22a124ce9174b5e2a76880

- Fix PL/Tcl for vpath builds. Commit cd37bb78599dcf24 works for
in-tree builds, but not so much for VPATH. Per buildfarm.
http://git.postgresql.org/pg/commitdiff/9f73a2f6d1c1305cf0dc749dbf631cffe26beda0

- Update time zone data files to tzdata release 2016c. DST law
changes in Azerbaijan, Chile, Haiti, Palestine, and Russia (Altai,
Astrakhan, Kirov, Sakhalin, Ulyanovsk regions). Historical
corrections for Lithuania, Moldova, Russia (Kaliningrad, Samara,
Volgograd). As of 2015b, the keepers of the IANA timezone database
started to use numeric time zone abbreviations (e.g., "+04") instead
of inventing abbreviations not found in the wild like "ASTT". This
causes our rather old copy of zic to whine "warning: time zone
abbreviation differs from POSIX standard" several times during "make
install". This warning is harmless according to the IANA folk, and
I don't see any problems with these abbreviations in some simple
tests; but it seems like now would be a good time to update our copy
of the tzcode stuff. I'll look into that soon.
http://git.postgresql.org/pg/commitdiff/676265eb7b57ba5bfae859630b909e6045893b68

- Avoid a couple of zero-divide scenarios in the planner.
cost_subplan() supposed that the given subplan must have plan_rows >
0, which as far as I can tell was true until recent refactoring of
the code in createplan.c; but now that code allows the Result for a
provably empty subquery to have plan_rows = 0. Rather than undo
that change, put in a clamp to prevent zero divide.
get_cheapest_fractional_path() likewise supposed that
best_path->rows > 0. This assumption has been wrong for longer.
It's actually harmless given IEEE float math, because a positive
value divided by zero gives +Infinity and
compare_fractional_path_costs() will do the right thing with that.
Still, best not to assume that. final_cost_nestloop() also seems to
have some risks in this area, so borrow the clamping logic already
present in the mergejoin cost functions. Lastly, remove unnecessary
clamp_row_est() in planner.c's calls to get_number_of_groups(). The
only thing that function does with path_rows is pass it to
estimate_num_groups() which already has an internal clamp, so we
don't need the extra call; and if we did, the callers are arguably
the wrong place for it anyway. First two items reported by Piotr
Stefaniak, the others are products of my nosing around for similar
problems. No back-patch since there's no evidence that problems
arise in the back branches.
http://git.postgresql.org/pg/commitdiff/76281aa9647e6a5dfc646514554d0f519e3b8a58

- Modernize zic's test for valid timezone abbreviations. We really
need to sync all of our IANA-derived timezone code with upstream,
but that's going to be a large patch and I certainly don't care to
shove such a thing into stable branches immediately before a
release. As a stopgap, copy just the tzcode2016c logic that checks
validity of timezone abbreviations. This prevents getting multiple
"time zone abbreviation differs from POSIX standard" bleats with
tzdata 2014b and later.
http://git.postgresql.org/pg/commitdiff/221619ad69b7e060041796a1974fbb0eeb9542d7

- First-draft release notes for 9.5.2. As usual, the release notes
for other branches will be made by cutting these down, but put them
up for community review first.
http://git.postgresql.org/pg/commitdiff/29b6123ecb4113e366325245cec5a5c221dae691

Robert Haas pushed:

- Improve check for overly-long extensible node name. The old code is
bad for two reasons. First, it has an off-by-one error. Second, it
won't help if you aren't running with assertions enabled. Per
discussion, we want a check here in that case too. Author: KaiGai
Kohei, adjusted by me. Reviewed-by: Petr Jelinek Discussion:
56E0D547(dot)1030101(at)2ndquadrant(dot)com
http://git.postgresql.org/pg/commitdiff/3adf9ced17dfa84faa209127b4499e5a5b995806

- Update more comments for 96198d94cb7adc664bda341842dc8db671d8be72.
Etsuro Fujita, reviewed (though not completely endorsed) by Ashutosh
Bapat, and slightly expanded by me.
http://git.postgresql.org/pg/commitdiff/6be84eeb8d5a72014b1b6e44d0646f69531a1ccf

- Fix EXPLAIN ANALYZE SELECT INTO not to choose a parallel plan. We
don't support any parallel write operations at present, so choosing
a parallel plan causes us to error out. Also, add a new regression
test that uses EXPLAIN ANALYZE SELECT INTO; if we'd had this
previously, force_parallel_mode testing would have caught this
issue. Mithun Cy and Robert Haas
http://git.postgresql.org/pg/commitdiff/270b7daf5cb1e955f8771ec819eb810057d1779b

- Add simple VACUUM progress reporting. There's a lot more that could
be done here yet - in particular, this reports only very
coarse-grained information about the index vacuuming phase - but
even as it stands, the new pg_stat_progress_vacuum can tell you
quite a bit about what a long-running vacuum is actually doing.
Amit Langote and Robert Haas, based on earlier work by Vinayak
Pokale and Rahila Syed.
http://git.postgresql.org/pg/commitdiff/c16dc1aca5e01e6acaadfcf38f5fc964a381dc62

- Fix typos. Thomas Reiss
http://git.postgresql.org/pg/commitdiff/2a90cb69e18e0c0637f3f93702e7b2f7808ff1b2

- postgres_fdw: make_tuple_from_result_row should set cur_attno for
ctid. There's no reason for this function to do this for every
other attribute number and omit it for CTID, especially since
conversion_error_callback has code to handle that case. This seems
to be an oversight in commit
e690b9515072fd7767fdeca5c54166f6a77733bc. Etsuro Fujita
http://git.postgresql.org/pg/commitdiff/4a46a99d8936328ab00e54bf8a2900832c4687e7

- Fix typos. Oskari Saarenmaa
http://git.postgresql.org/pg/commitdiff/3aff33aa687e47d52f453892498b30ac98a296af

- Add idle_in_transaction_session_timeout. Vik Fearing, reviewed by
Stéphane Schildknecht and me, and revised slightly by me.
http://git.postgresql.org/pg/commitdiff/c6dda1f48e573cc60fe25d8d470a05bfabed0252

- Fix typo. Amit Langote
http://git.postgresql.org/pg/commitdiff/1c4f001b79878deb9475e5b32ff4ef87773c9f23

- Fix problems in commit c16dc1aca5e01e6acaadfcf38f5fc964a381dc62.
Vinayak Pokale provided a patch for a copy-and-paste error in a
comment. I noticed that I'd use the word "automatically" nearby
where I meant to talk about things being "atomic". Rahila Syed
spotted a misplaced counter update. Fix all that stuff.
http://git.postgresql.org/pg/commitdiff/bc55cc0b6a4af048f46c7e46c9e9c1f24b7ee9b8

- Another comment update. I thought this was in my last commit, but I
goofed.
http://git.postgresql.org/pg/commitdiff/f2b74b01d4a18241bd7560e74d527f3ba42d4738

- Fix typos. Jim Nasby
http://git.postgresql.org/pg/commitdiff/0218e8b3fa7ec72af441ac6f80927bff0d497334

- Update tuplesort.c comments for memory mangement improvements. I'm
committing these changes separately so that it's clear what is
Peter's original work versus what I changed. This is a followup to
commit 0011c0091e886b874e485a46ff2c94222ffbf550, and these changes
are all by me.
http://git.postgresql.org/pg/commitdiff/c27033ff7c17b5100d02c454a0eebb95ec7b91cc

- Various minor corrections of and improvements to comments.
Aleksander Alekseev
http://git.postgresql.org/pg/commitdiff/2d8a1e22b109680204cb015a30e5a733a233ed64

- Push scan/join target list beneath Gather when possible. This means
that, for example, "SELECT expensive_func(a) FROM bigtab WHERE
something" can compute expensive_func(a) in the workers rather than
the leader if it happens to be parallel-safe, which figures to be a
big win in some practical cases. Currently, we can only do this if
the entire target list is parallel-safe. If we worked harder, we
might be able to evaluate parallel-safe targets in the worker and
any parallel-restricted targets in the leader, but that would be
more complicated, and there aren't that many parallel-restricted
functions that people are likely to use in queries anyway. I think.
So just do the simple thing for the moment. Robert Haas, Amit
Kapila, and Tom Lane
http://git.postgresql.org/pg/commitdiff/992b5ba30dcafdc222341505b072a6b009b248a7

- Improve memory management for external sorts. Introduce a new
memory context which stores tuple data, and reset it at the end of
each merge pass; this helps avoid memory fragmentation and,
consequently, overallocation. Also, for the final merge patch,
eliminate memory context chunk header overhead entirely by
allocating all of the memory used for buffering tuples during the
merge in a single chunk. Since this modestly increases the number
of tuples we can store, grow the memtuples array a bit so that we're
less likely to run short of slots there. Peter Geoghegan. Review
and testing of patches in this series by Jeff Janes, Greg Stark,
Mithun Cy, and me.
http://git.postgresql.org/pg/commitdiff/0011c0091e886b874e485a46ff2c94222ffbf550

- Directly modify foreign tables. postgres_fdw can now sent an UPDATE
or DELETE statement directly to the foreign server in simple cases,
rather than sending a SELECT FOR UPDATE statement and then updating
or deleting rows one-by-one. Etsuro Fujita, reviewed by Rushabh
Lathia, Shigeru Hanada, Kyotaro Horiguchi, Albe Laurenz, Thom Brown,
and me.
http://git.postgresql.org/pg/commitdiff/0bf3ae88af330496517722e391e7c975e6bad219

- Use INT64_FORMAT instead of %ld for int64. Commit
0011c0091e886b874e485a46ff2c94222ffbf550 introduced this mistake.
Patch by me. Reported by Andres Freund, who also reviewed the
patch.
http://git.postgresql.org/pg/commitdiff/08a6d36dcbe4ebdcdf9d554e5b571748fe23001b

- Support parallel aggregation. Parallel workers can now partially
aggregate the data and pass the transition values back to the
leader, which can combine the partial results to produce the final
answer. David Rowley, based on earlier work by Haribabu Kommi.
Reviewed by Álvaro Herrera, Tomas Vondra, Amit Kapila, James Sewell,
and me.
http://git.postgresql.org/pg/commitdiff/e06a38965b3bcdaa881e7e06892d4d8ab6c2c980

- Make max_parallel_degree PGC_USERSET. It was intended to be this
way all along, just like other planner GUCs such as work_mem. But I
goofed.
http://git.postgresql.org/pg/commitdiff/ae507d9222989f2fd8455cf32d4dcb337ccf32bb

- Partition the freelist for shared dynahash tables. Without this,
contention on the freelist can become a pretty serious problem on
large servers. Aleksander Alekseev, reviewed by Anastasia
Lubennikova, Dilip Kumar, and me.
http://git.postgresql.org/pg/commitdiff/44ca4022f3f9297bab5cbffdd97973dbba1879ed

- postgres_fdw: Fix crash when pushing down multiple joins. A join
clause might mention multiple relations on either side, so it need
not be the case that a given joinrel's constituent relations are all
on one side of the join clause or all on the other. Report by
Rajkumar Raghuwanshi. Analysis and fix by Michael Paquier and
Ashutosh Bapat.
http://git.postgresql.org/pg/commitdiff/3151f16e1874db82ed85a005dac15368903ca9fb

- Disable abbreviated keys for string-sorting in non-C locales.
Unfortunately, every version of glibc thus far tested has bugs
whereby strcoll() ordering does not match strxfrm() ordering as
required by the standard. This can result in, for example,
corrupted indexes. Disabling abbreviated keys in these cases slows
down non-C-collation string sorting considerably, but there seems to
be no practical alternative. Users who are confident that their
libc implementations are solid in this regard can re-enable the
optimization by compiling with TRUST_STRXFRM. Users who have built
indexes using PostgreSQL 9.5 or PostgreSQL 9.5.1 should REINDEX if
there is a possibility that they may have been affected by this
problem. Report by Marc-Olaf Jaschke. Investigation mostly by Tom
Lane, with help from Peter Geoghegan, Noah Misch, Stephen Frost, and
me. Patch by me, reviewed by Peter Geoghegan and Tom Lane.
http://git.postgresql.org/pg/commitdiff/3df9c374e279db37b00cd9c86219471d0cdaa97c

- Use correct GetDatum function. Oops.
http://git.postgresql.org/pg/commitdiff/59a02815e2d11e5a7b4848897c763175b6b42f8a

- Improve documentation for combine functions. David Rowley
http://git.postgresql.org/pg/commitdiff/a596db332b8c7f593a82af86f69353ba08f6214c

Peter Eisentraut pushed:

- Add missing include for self-containment
http://git.postgresql.org/pg/commitdiff/be6de4c1215a8ad5607b1fcc7e9e6da1de780877

- UCS_to_EUC_JIS_2004.pl: Turn off "test" mode by default. It
produces debugging output files that are of no further use, so we
don't need that by default.
http://git.postgresql.org/pg/commitdiff/f9e5ed61ed1e27390051e8eb10bd9ec4cb40d7b2

- Make spacing and punctuation consistent.
http://git.postgresql.org/pg/commitdiff/9dbcb500cad06f2dd1c699e8dbcfbe23145e61fa

- Add syslog_sequence_numbers parameter. Reviewed-by: Andreas
Karlsson <andreas(at)proxel(dot)se>
http://git.postgresql.org/pg/commitdiff/f4c454e9ba5262c5614c7637983c83b8c327e231

- Add syslog_split_messages parameter. Reviewed-by: Andreas Karlsson
<andreas(at)proxel(dot)se>
http://git.postgresql.org/pg/commitdiff/fc201dfd95059fd2fef9862a2fd09cfab42c9bf7

- Merge wal_level "archive" and "hot_standby" into new name "replica".
The distinction between "archive" and "hot_standby" existed only
because at the time "hot_standby" was added, there was some
uncertainty about stability. This is now a long time ago. We would
like to move forward with simplifying the replication configuration,
but this distinction is in the way, because a primary server cannot
tell (without asking a standby or predicting the future) which one
of these would be the appropriate level. Pick a new name for the
combined setting to make it clearer that it covers all (non-logical)
backup and replication uses. The old values are still accepted but
are converted internally. Reviewed-by: Michael Paquier
<michael(dot)paquier(at)gmail(dot)com> Reviewed-by: David Steele
<david(at)pgmasters(dot)net>
http://git.postgresql.org/pg/commitdiff/b555ed8102672cfedb06559952b8341756386d69

- Allow SSL server key file to have group read access if owned by
root. We used to require the server key file to have permissions
0600 or less for best security. But some systems (such as Debian)
have certificate and key files managed by the operating system that
can be shared with other services. In those cases, the "postgres"
user is made a member of a special group that has access to those
files, and the server key file has permissions 0640. To accommodate
that kind of setup, also allow the key file to have permissions 0640
but only if owned by root. From: Christoph Berg <myon(at)debian(dot)org>
Reviewed-by: Alvaro Herrera <alvherre(at)alvh(dot)no-ip(dot)org>
http://git.postgresql.org/pg/commitdiff/9a83564c58b7f6363141a8f1d0c87c89a5ebab5d

Álvaro Herrera pushed:

- Fix typos in comments.
http://git.postgresql.org/pg/commitdiff/5bcc413f80de4e0bc8f0df15d7d0ac2e8272dec5

- docs: Fix typo'd brin_summarize_new_values. I wrote
"brin_summarize_new_pages" instead, in docs as well as in the commit
message of commit ac443d1034d9. Bug: #14030 Reported-By: Chris
Pacejo
http://git.postgresql.org/pg/commitdiff/696684d878747a1a5cfae01887404629c40c09dd

- pgbench: Allow changing weights for scripts. Previously, all
scripts had the same probability of being chosen when multiple of
them were specified via -b, -f, -N, -S. With this commit, -b and -f
now search for an "@" in the script name and use the integer found
after it as the drawing probability for that script. (One
disadvantage is that if you have script whose names contain @, you
are now forced to specify "@1" at the end; otherwise the name's @ is
confused with a weight separator. We don't expect many pgbench
script with @ in their names in the wild, so this shouldn't be too
serious a problem.) While at it, rework the interface between
addScript, process_file, process_builtin, and findBuiltin. It had
gotten a bit out of hand with recent commits. Author: Fabien Coelho
Reviewed-By: Andres Freund, Robert Haas, Álvaro Herrera, Michaël
Paquier Discussion:
http://www.postgresql.org/message-id/alpine.DEB.2.10.1603160721240.1666@sto
http://git.postgresql.org/pg/commitdiff/7bafffea647e21864cb857ab5b6fe266f2d0976a

- pgbench: Silence new compiler warnings. The original coding in
7bafffea647 and previous wasn't all that great anyway. Reported by
Jeff Janes and Tom Lane
http://git.postgresql.org/pg/commitdiff/1038bc91ca98865bd60bf63db46fc331f3099998

Stephen Frost pushed:

- Avoid incorrectly indicating exclusion constraint wait. INSERT ...
ON CONFLICT's precheck may have to wait on the outcome of another
insertion, which may or may not itself be a speculative insertion.
This wait is not necessarily associated with an exclusion
constraint, but was always reported that way in log messages if the
wait happened to involve a tuple that had no speculative token.
Initially discovered through use of ON CONFLICT DO NOTHING, where
spurious references to exclusion constraints in log messages were
more likely. Patch by Peter Geoghegan. Reviewed by Julien Rouhaud.
Back-patch to 9.5 where INSERT ... ON CONFLICT was added.
http://git.postgresql.org/pg/commitdiff/fd658dbb300456b393536802d1145a9cea7b25d6

Teodor Sigaev pushed:

- Improve script generating unaccent rules. Script now use the
standard Unicode transliterator Latin-ASCII. Author: Leonard
Benedetti
http://git.postgresql.org/pg/commitdiff/9a206d063c410df7cd5da01b169b23bff413fef5

- fix typo in comment
http://git.postgresql.org/pg/commitdiff/ce91b9209f221f9555d340b40e02aaf8cdc777e4

- Add files forgotten in f576b17cd6ba653bdace1f0da9a3b57f4984e460
http://git.postgresql.org/pg/commitdiff/e4b523e5b52a84010a34fc0796904ce39116e56c

- Add word_similarity to pg_trgm contrib module. Patch introduces a
concept of similarity over string and just a word from another
string. Version of extension is not changed because 1.2 was already
introduced in 9.6 release cycle, so, there wasn't a public version.
Author: Alexander Korotkov, Artur Zakirov
http://git.postgresql.org/pg/commitdiff/f576b17cd6ba653bdace1f0da9a3b57f4984e460

- Improve support of Hunspell: Allow to use non-ascii characters as
affix flag. Non-numeric affix flags now are stored as string instead
of numeric value of character. Allow to use 0 as affix flag in
numeric encoded affixes That adds support for arabian, hungarian,
turkish and brazilian portuguese languages. Author: Artur Zakirov
with heavy editorization by me
http://git.postgresql.org/pg/commitdiff/f4ceed6ceba31a72ed7a726fef05d211641f283c

- GUC variable pg_trgm.similarity_threshold insead of set_limit(). Use
GUC variable pg_trgm.similarity_threshold insead of
set_limit()/show_limit() which was introduced when defining GUC
varuables by modules was absent. Author: Artur Zakirov
http://git.postgresql.org/pg/commitdiff/5871b88487cfd07966e2ce08609a4d6d5ee9718e

- pg_trgm's set_limit() now uses SetConfigOption(). Deprecated
set_limit() is modified to use SetConfigOption() to set
similarity_threshold which is actually an instance of
pg_trgm.similarity_threshold GUC variable. Previous coding directly
sets similarity_threshold what could cause an inconsistency between
states of actual variable and GUC representation. Per gripe from
Tom Lane
http://git.postgresql.org/pg/commitdiff/aa698d753566f68bdd54881d30b1a515b0327b0e

- Fix a typo. Erik Rijkers
http://git.postgresql.org/pg/commitdiff/61d2ebdbf9155d80f83281d8c5729c8f8f0949bc

- Introduce parse_ident(). SQL-layer function to split qualified
identifier into array parts. Author: Pavel Stehule with minor
editorization by me and Jim Nasby
http://git.postgresql.org/pg/commitdiff/3187d6de0e5a9e805b27c48437897e8c39071d45

- Improve docs of pg_trgm changes Artur Zakirov, per gripe from Jeff
Janes
http://git.postgresql.org/pg/commitdiff/f6bd0da63bf40bc324eec6fd47f3d3d301f44dab

Andres Freund pushed:

- Error out if waiting on socket readiness without a specified socket.
Previously we just ignored such an attempt, but that seems to serve
no purpose but making things harder to debug. Discussion:
20160114143931(dot)GG10941(at)awork2(dot)anarazel(dot)de
20151230173734(dot)hx7jj2fnwyljfqek(at)alap3(dot)anarazel(dot)de Reviewed-By:
Robert Haas
http://git.postgresql.org/pg/commitdiff/6bc4d95fcc2a432fc202cba03d5393be096f0422

- Only clear latch self-pipe/event if there is a pending notification.
This avoids a good number of, individually quite fast, system calls
in scenarios with many quick queries. Besides the aesthetic benefit
of seing fewer superflous system calls with strace, it also improves
performance by ~2% measured by pgbench -M prepared -c 96 -j 8 -S
(scale 100). Without having benchmarked it, this patch also adjust
the windows code, as that makes it easier to unify the unix/windows
codepaths in a later patch. There's little reason to diverge in
behaviour between the platforms. Discussion:
CA+TgmoYc1Zm+Szoc_Qbzi92z2c1vRHZmjhfPn5uC=w8bXv6Avg(at)mail(dot)gmail(dot)com
Reviewed-By: Robert Haas
http://git.postgresql.org/pg/commitdiff/c4901a1e03a7730e4471fd1143f1caf79695493d

- Remove unused, and dangerous, TestLatch() macro. The macro has not
seen any in-tree use since latches had been introduced in 2746e5f,
in 2010.
http://git.postgresql.org/pg/commitdiff/fad0f9d8c9f6a8e99156b8f01cba54be39f31761

- Make it easier to choose the used waiting primitive in unix_latch.c.
This allows for easier testing of the different primitives; in
preparation for adding a new primitive. Discussion:
20160114143931(dot)GG10941(at)awork2(dot)anarazel(dot)de Reviewed-By: Robert Haas
http://git.postgresql.org/pg/commitdiff/c17966201c7de2a4c437bed6d83c6a7f2e7108f4

- Fix stupid omission in c4901a1e. Reported-By: Jeff Janes
Discussion:
CAMkU=1zGxREwoyaCrp_CHadEB+dPgpVyKBysCJ+6xP9gCOvAuw(at)mail(dot)gmail(dot)com
http://git.postgresql.org/pg/commitdiff/6eb2be15b5d24b98d334a9dd637f0edb37e2eb7e

- Second attempt at fixing MSVC build for 68ab8e8ba4a471d9. After the
previous fix in 6f1f34c9 msvc ended up looking for psqlscan.c in the
wrong directory. David's fix just forces the path to be adjusted.
That's not a particularly pretty fix, but it hopefully will make the
buildfarm green again. Author: David Rowley Discussion:
CAKJS1f_9CCi_t+LEgV5GWoCj3wjavcMoDc5qfcf_A0UwpQoPoA(at)mail(dot)gmail(dot)com
http://git.postgresql.org/pg/commitdiff/326d73c86fda407a810675c3b5a48e0a0cc992f5

- Introduce WaitEventSet API. Commit ac1d794 ("Make idle backends
exit if the postmaster dies.") introduced a regression on, at least,
large linux systems. Constantly adding the same postmaster_alive_fds
to the OSs internal datastructures for implementing poll/select can
cause significant contention; leading to a performance regression of
nearly 3x in one example. This can be avoided by using e.g. linux'
epoll, which avoids having to add/remove file descriptors to the
wait datastructures at a high rate. Unfortunately the current latch
interface makes it hard to allocate any persistent per-backend
resources. Replace, with a backward compatibility layer,
WaitLatchOrSocket with a new WaitEventSet API. Users can allocate
such a Set across multiple calls, and add more than one
file-descriptor to wait on. The latter has been added because
there's upcoming postgres features where that will be helpful. In
addition to the previously existing poll(2), select(2),
WaitForMultipleObjects() implementations also provide an
epoll_wait(2) based implementation to address the aforementioned
performance problem. Epoll is only available on linux, but that is
the most likely OS for machines large enough (four sockets) to
reproduce the problem. To actually address the aforementioned
regression, create and use a long-lived WaitEventSet for FE/BE
communication. There are additional places that would benefit from
a long-lived set, but that's a task for another day. Thanks to Amit
Kapila, who helped make the windows code I blindly wrote actually
work. Reported-By: Dmitry Vasilyev Discussion:
CAB-SwXZh44_2ybvS5Z67p_CDz=XFn4hNAD=CnMEF+QqkXwFrGg(at)mail(dot)gmail(dot)com
20160114143931(dot)GG10941(at)awork2(dot)anarazel(dot)de
http://git.postgresql.org/pg/commitdiff/98a64d0bd713cb89e61bef6432befc4b7b5da59e

- Combine win32 and unix latch implementations. Previously latches
for windows and unix had been implemented in different files. A
later patch introduce an expanded wait infrastructure, keeping the
implementation separate would introduce too much duplication. This
basically just moves the functions, without too much change. The
reason to keep this separate is that it allows blame to continue
working a little less badly; and to make review a tiny bit easier.
Discussion: 20160114143931(dot)GG10941(at)awork2(dot)anarazel(dot)de
http://git.postgresql.org/pg/commitdiff/72e2d21c1249b674496f97cd6009c0bda62f6b4d

- Properly declare FeBeWaitSet. Surprising that this worked on a
number of systems. Reported by buildfarm member longfin.
http://git.postgresql.org/pg/commitdiff/7fa0064092e135415a558dc3c4d7393d14ab6d8e

- Change various Gin*Is* macros to return 0/1. Returning the direct
result of bit arithmetic, in a macro intended to be used in a
boolean manner, can be problematic if the return value is stored in
a variable of type 'bool'. If bool is implemented using C99's _Bool,
that can lead to comparison failures if the variable is then
compared again with the expression (see ginStepRight() for an
example that fails), as _Bool forces the result to be 0/1. That
happens in some configurations of newer MSVC compilers. It's also
problematic when storing the result of such an expression in a
narrower type. Several gin macros have been declared in that style
since gin's initial commit in 8a3631f8d86. There's a lot more
macros like this, but this is the only one causing regression test
failures; and I don't want to commit and backpatch a larger patch
with lots of conflicts just before the next set of minor releases.
Discussion: 20150811154237(dot)GD17575(at)awork2(dot)anarazel(dot)de Backpatch: All
supported branches
http://git.postgresql.org/pg/commitdiff/af4472bcb88ab36b9abbe7fd5858e570a65a2d1a

- Don't use !! but != 0/NULL to force boolean evaluation. I
introduced several uses of !! to force bit arithmetic to be boolean,
but per discussion the project prefers != 0/NULL. Discussion:
CA+TgmoZP5KakLGP6B4vUjgMBUW0woq_dJYi0paOz-My0Hwt_vQ(at)mail(dot)gmail(dot)com
http://git.postgresql.org/pg/commitdiff/1a7a43672bf2939dda93a27d498349e7a4aa3c14

Andrew Dunstan pushed:

- Remove dependency on psed for MSVC builds. Modern Perl has removed
psed from its core distribution, so it might not be readily
available on some build platforms. We therefore replace its use with
a Perl script generated by s2p, which is equivalent to the sed
script. The latter is retained for non-MSVC builds to avoid creating
a new hard dependency on Perl for non-Windows tarball builds.
Backpatch to all live branches. Michael Paquier and me.
http://git.postgresql.org/pg/commitdiff/5d0320105699c253fe19b8b42ae1bffb67785b02

Fujii Masao pushed:

- Fix typo in docs. Jeff Janes
http://git.postgresql.org/pg/commitdiff/112a2d0615bab27630829e486c8b0cd2fdd6980b

Simon Riggs pushed:

- Change comment to describe correct lock level used
http://git.postgresql.org/pg/commitdiff/8320c625d983f4cf4c8f453165502d7cf3aa7cf3

Álvaro Herrera pushed:

- Support CREATE ACCESS METHOD. This enables external code to create
access methods. This is useful so that extensions can add their own
access methods which can be formally tracked for dependencies, so
that DROP operates correctly. Also, having explicit support makes
pg_dump work correctly. Currently only index AMs are supported, but
we expect different types to be added in the future. Authors:
Alexander Korotkov, Petr Jelínek. Reviewed-By: Teodor Sigaev, Petr
Jelínek, Jim Nasby. Commitfest-URL:
https://commitfest.postgresql.org/9/353/ Discussion:
https://www.postgresql.org/message-id/CAPpHfdsXwZmojm6Dx+TJnpYk27kT4o7Ri6X_4OSWcByu1Rm+VA@mail.gmail.com
http://git.postgresql.org/pg/commitdiff/473b93287040b20017cc25a157cffdc5b978c254

== Rejected Patches (for now) ==

No one was disappointed this week :-)

== Pending Patches ==

Etsuro Fujita sent in a patch to expose umid in foreign tables.

Konstantin Knizhnik sent in another revision of a patch to extend
ALTER INDEX to allow changing the predicate of a partial index.

Stephen Frost sent in two more revisions of a patch to add default
roles.

Tomas Vondra sent in another revision of a patch to use foreign keys
to improve join estimates.

Peter Geoghegan sent in a patch to distrust external OpenSSL clients
and clear err queue.

Amit Kapila sent in another revision of a patch to prevent prepared
statements from inappriately enabling parallelism.

Álvaro Herrera sent in another revision of a patch to allow logical
slots to follow timeline switches.

Amit Kapila sent in another revision of a patch to group the status
update for transactions.

Kaigai Kouhei sent in another revision of a patch to rework custom
scan serialization.

Anastasia Lubennikova sent in another revision of a patch to implement
covering unique indexes.

Michaël Paquier sent in two more revisions of a patch to implement
SCRAM authentication with infrastructure for making auth more
pluggable.

Corey Huinker sent in another revision of a patch to implement \gexec
in psql.

Robbie Harwood sent in two more revisions of a patch to add GSSAPI
encryption support.

David Rowley and Julien Rouhaud traded patches to enable setting
parallel degree.

Alexander Shulgin sent in another revision of a patch to add
\errverbose psql command and support in libpq.

Michaël Paquier sent in another revision of a patch to fix hot standby
checkpoints.

Ashutosh Bapat sent in two revisions of a patch to require user
mappings for foreign joins if the tables involved would have required
same.

Oskari Saarenmaa sent in a patch to display backends for dropped roles
in pg_stat_activity.

Craig Ringer sent in two more revisions of a patch to implement
failover slots.

Michaël Paquier sent some follow-on patches to the ones now committed
that make sure filesystem moves are successful before moving on.

Constantin S. Pan sent in another revision of a patch to speed up GIN
build with parallel workers.

Thomas Reiss and Thom Brown traded patches to replace
pg_stat_activity.waiting with things more descriptive.

Vik Fearing sent in another revision of a patch to add an
idle-in-transaction timeout.

David Fetter sent in another revision of a patch to add weighted
statistics (mean, stddev).

David Rowley sent eight more revisions of a patch to enable parallel
aggregation.

Peter Geoghegan sent in a patch to refactor speculative insertion into
unique indexes.

Craig Ringer sent in two more revisions of a patch to fix an issue
where logical decoding slots were not documented to be able to go
backwards, but were able to from SQL.

Craig Ringer sent in a patch to fix TAP to initialize $timed_out
correctly if passed.

Aleksander Alekseev sent in a patch to fix double variable
initializations in policy.c.

Amit Kapila and Robert Haas traded patches to push down target lists
below gather nodes.

Fabien COELHO and Álvaro Herrera traded patches to add pgbench stats
per script, etc.

Andres Freund and Thomas Munro traded patches to fix a performance
regression.

Artur Zakirov sent in a patch to allow numbers in email addresses in
tsearch.

Dmitry Dolgov sent in two more revisions of a patch to add a function
which allows inserting into a JSONB object at an arbitrary point.

Dmitry Ivanov sent in two more revisions of a patch to add phrase
search to the tsearch extension.

David Rowley and Haribabu Kommi traded patches for combining
aggregates.

Tomas Vondra sent in four more revisions of a patch to add
multivariate statistics.

Dilip Kumar and Robert Haas traded patches to help scale relation
extension.

Alexander Kuleshov sent in a patch to reduce three macros
PG_CMD_PRINTF{1,2,3} to one.

Mark Dilger and Kevin Grittner traded patches to remove gratuitous
gender-specific wording from the code.

Marisa Emerson sent in two more revisions of a patch to implement BSD
Authentication.

Haribabu Kommi sent in three more revisions of a patch to add
pg_hba_lookup().

Stas Kelvich sent in four revisions of a patch to fix a bug in fd.c on
OS X.

Tomas Vondra sent in another revision of a patch to improve GROUP BY
estimation.

Kevin Grittner sent in another revision of a patch to implement
"snapshot too old" configured by time.

Teodor Sigaev sent in two more revisions of a patch to implement index
lookups for OR clauses.

Andrew Dunstan sent in two revisions of a patch to add enum support
for btree_gi(n|st) supplied extensions.

Tomas Vondra sent in another revision of a patch to fix an issue in
pgstat.

Robert Haas sent in a patch to fix an issue with the recently added
memory management improvement for external sorts.

Tomas Vondra and Fabien COELHO traded patches to fix outdated docs on
pgbench.

Petr Jelínek and Artur Zakirov traded patches to create generic
logical WAL messages.

Anastasia Lubennikova sent in another revision of a patch to compress
away duplicates in a B-Tree index.

Teodor Sigaev and Emre Hasegeli traded patches to add support for box
type in SP-GiST index.

Christoph Berg sent in a patch to create a pg_filedump for 9.5.

Jeff Janes sent in a patch to allow zero weights in pgbench tests.

Magnus Hagander sent in another revision of a patch to implement
non-exclusive backups.

Fabien COELHO sent in four more revisions of a patch to extend pgbench
expressions with functions.

Alexander Korotkov sent in three more revisions of a patch to move
PinBuffer and UnpinBuffer to atomics.

Tomas Vondra sent in another revision of a patch to enable index-only
scans with partial indexes.

Yuri Niyazov sent in a patch to improve the pg_upgrade documentation.

Pavel Stěhule sent in another revision of a patch to add a
\crosstabview to psql.

Michaël Paquier and Ashutosh Bapat traded patches to fix some issues
in the recently added FDW write pushdown patch.

Christian Ullrich sent in two revisions of a patch to fix realm
handling in SSPI auth.

Anastasia Lubennikova sent in another revision of a patch to add
covering + unique indexes.

Julien Rouhaud sent in two more revisions of a patch to choose
parallel degree.

Jeff Janes sent in a patch to fix a typo in the vacuum progress
documentation.

Aleksander Alekseev sent in two more revisions of a patch to make
PostgreSQL sanitizers-friendly.

Abhijit Menon-Sen sent in a patch to implement ALTER FUNCTION x
DEPENDS ON EXTENSION y.

Artur Zakirov sent in another revision of a patch to add fuzzy
substring searching with the pg_trgm extension.

Grzegorz Sampolski sent in another revision of a patch to add an rhost
entry to PAM auth.

Constantin S. Pan sent in another revision of a patch to speed up GIN
index builds with parallel workers.

Craig Ringer sent in another revision of a patch to add timeline
following for logical decoding.

Michaël Paquier sent in two revisions of a patch to ensure that all
messages from pg_rewind --debug get translated.

Pavan Deolasee sent in two revisions of a patch to fix an issue where
pg_xlogdump fails to handle WAL file with multi-page
XLP_FIRST_IS_CONTRECORD data.

Craig Ringer sent in two revisions of a patch to add a README for
src/backend/replication/logical.

Vitaly Burovoy and Michaël Paquier traded patches to fix an issue in
searching path in jsonb_set when walking through JSONB array.

Kyotaro HORIGUCHI sent in another revision of a patch to show dropped
users' backends in pg_stat_activity.

Fujii Masao and SAWADA Masahiko traded patches for support for N
synchronous standby servers, N > 1.

YUriy Zhuravlev sent in four revisions of a patch to add CINE to
PREPARE.

Roma Sokolov and Robert Haas traded patches to fix DROP OPERATOR to
reset links to itself on commutator and negator.

Amit Kapila sent in another revision of a patch to speed up CLOG
access by increasing CLOG buffers.

Stephen Frost sent in another revision of a patch to cause pg_dump to
dump catalog ACLs.

Aleksander Alekseev sent in another revision of a patch to fix lock
contention for HASHHDR.mutex.

Daniel Verité sent in another revision of a patch to fix pg_dump and
COPY's handling of gigantic lines.

Petr Jediný sent in another revision of a patch to add BRIN to the
multicolumn indexes documentation.

Gilles Darold sent in another revision of a patch to add
pg_current_logfile().

Kaigai Kouhei sent in another revision of a patch to rework CustomScan
serialization/deserialization.

Tom Lane sent in a patch to fix SRF behavior in target lists.

Michaël Paquier sent in another revision of a patch to add VS 2015
support in src/tools/msvc.

Abhijit Menon-Sen sent in three more revisions of a patch to deal with
extension dependencies that aren't quite 'e'.

Thomas Munro sent in three more revisions of a patch to add "causal
reads."

Petr Jelínek sent in another revision of a patch to add a sequence
access method and a gapless sequence implementation using same.

Aleksander Alekseev sent in two revisions of a patch to fix some code
duplication in heapam.c.

Dagfinn Ilmari Mannsåker sent in three revisions of a patch to
implement ALTER TYPE ... ALTER VALUE .. TO .. for enums.

Robert Haas sent in a patch to measure waste memory for cache
alignment correctness.

Thomas Munro sent in a patch to fix a typo in a comment.