== PostgreSQL Weekly News - November 25, 2018 ==

== PostgreSQL Weekly News - November 25, 2018 ==

PGConf India 2019 will be on February 13-15, 2019 in Bengaluru, Karnataka.
The CfP at the site below is open through December 15, 2018.
http://pgconf.in/

== PostgreSQL Product News ==

pg_backtrace, and extenstion which makes it possible to get Postgres errors and
exceptions when no debugger is available, released.
https://github.com/postgrespro/pg_backtrace

pgpool-II 4.0.2, 3.7.7, 4.6.14, 3.5.18, and 3.4.21 released.
http://www.pgpool.net/docs/latest/en/html/release.html

PostGIS 2.5.1, the industry standard geographic information
system package for PostgreSQL, released.
https://postgis.net/2018/11/18/postgis-2.5.1/

PostGIS 2.2.8 is the end of the 2.2 series.
https://postgis.net/2018/11/22/postgis-2.2.8/

PostGIS 2.3.8 and 2.4.6 released.
https://postgis.net/2018/11/24/postgis-patches/

pgFormatter 3.3, a formatter/beautifier for SQL code, released.
http://sqlformat.darold.net/

griddb_fdw 1.0 released.
https://github.com/pgspider/griddb_fdw

== PostgreSQL Jobs for November ==

http://archives.postgresql.org/pgsql-jobs/2018-11/

== PostgreSQL Local ==

2Q PGConf will be on December 4-5, 2018 in Chicago, IL.
http://www.2qpgconf.com/

PGConf.ASIA 2018 will take place on December 10-12, 2018 in Akihabara, Tokyo,
Japan.
http://www.pgconf.asia/EN/2018/

FOSDEM PGDay 2019, a one day conference held before the main FOSDEM event will
be held in Brussels, Belgium, on Feb 1st, 2019.
https://2019.fosdempgday.org/

Prague PostgreSQL Developer Day 2019 (P2D2 2019) is a two-day
conference that will be held on February 13-14, 2019 in Prague, Czech Republic.
The CfP is open until January 4, 2018 at https://p2d2.cz/callforpapers
http://www.p2d2.cz/

PGConf India 2019 will be on February 13-15, 2019 in Bengaluru, Karnataka.
The CfP at the site below is open through December 15, 2018.
http://pgconf.in/

pgDay Paris 2019 will be held in Paris, France on March 12, 2019
at 199bis rue Saint-Martin. The CfP is open until November 30, 2018.
http://2019.pgday.paris/callforpapers/

PGConf APAC 2019 will be held in Singapore March 19-21, 2019. The CfP is open
at http://2019.pgconfapac.org/cfp through November 30, 2018.
http://2019.pgconfapac.org/

PGDay.IT 2019 will take place May 16th and May 17th in Bologna, Italy. The CfP
is open at https://2019.pgday.it/en/blog/cfp and the Call for Workshops is at
https://2019.pgday.it/en/blog/cfw until January 15, 2019.
https://2019.pgday.it/en/

== PostgreSQL in the News ==

Planet PostgreSQL: http://planet.postgresql.org/

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm PST8PDT to david(at)fetter(dot)org(dot)

== Applied Patches ==

Michaël Paquier pushed:

- Remove unnecessary memcpy when reading WAL record fitting on page. When
reading a WAL record, its contents are copied into an intermediate buffer.
However, doing so is not necessary if the record fits fully into the current
page, saving one memcpy for each such record. The allocation handling of the
intermediate buffer is also now done only when a record crosses a page
boundary, shaving some extra cycles when reading a WAL record. Author: Andrey
Lepikhov Reviewed-by: Kyotaro Horiguchi, Heikki Linnakangas Discussion:
https://postgr.es/m/c2ea54dd-a1d3-80eb-ddbf-7e6f258e615e@postgrespro.ru
https://git.postgresql.org/pg/commitdiff/285bd0ac4a7c0538d544c40aa725682e11cb71a9

- Rework documentation of pg_promote. This clarifies the behavior of how the
"wait" flag works, which is something that the previous version of the
documentation failed to do. Author: Ian Barwick Discussion:
https://postgr.es/m/cbd38450-2295-10a1-1f73-591a692ae0b0@2ndquadrant.com
https://git.postgresql.org/pg/commitdiff/74171f8c12e68408f1fa895fdee36fdc8b0a9f79

- Fix issues with TAP tests of pg_verify_checksums Two issues have been spotted
and get fixed here: - When checking for corrupted files, make sure that
pg_verify_checksums complains about the correct file. In order to make the
logic more robust, all files created are immediately removed once checks on
them are done. The error message generated by pg_verify_checksums also now
includes the file name it sees as corrupted. - Before running
corruption-related tests, empty files are generated. which used names mapping
with the corrupted files, potentially leading to conflicts. So use different
set of names for both. Author: Michael Banck Discussion:
https://postgr.es/m/20181119181119.GC23740@nighthawk.caipicrew.dd-dns.de
https://git.postgresql.org/pg/commitdiff/9685d7383ab48ebe6a6213530f64e4dc67776583

- Improve description of buffer used to store records in WAL reader. The
dedicated private buffer to store records is used only for these crossing a
page boundary since 285bd0ac, but its description did not match completely the
reality. Reported-by: Andrey Lepikhov Author: Michael Paquier Discussion:
https://postgr.es/m/49518b48-2036-5e43-1818-0f594e375e76@postgrespro.ru
https://git.postgresql.org/pg/commitdiff/0999ac479292c12a7c373e612b15e1ff47077990

- Fix typo in description of ExecFindPartition. Author: Amit Langote
Discussion:
https://postgr.es/m/CA+HiwqHg0=UL+Dhh3gpiwYNA=ufk9Lb7GQ2c=5rs=ZmVTP7xAw@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/25c026c284de5ffb1ea5b007afc10158bfab64cb

- Clarify documentation about PASSWORD in CREATE/ALTER ROLE The documentation of
CREATE/ALTER ROLE has been missing two things related to PASSWORD: - The
password value provided needs to be quoted, some places of the documentation
marked the field with quotes, but not others, which led to confusion. -
PASSWORD NULL was not provided consistently, with ENCRYPTED being not.
compatible with it. Reported-by: Steven Winfield Author: Michael Paquier
Reviewed-by: David G. Johnston Discussion:
https://postgr.es/m/154282901979.1316.7418475422120496802@wrigleys.postgresql.org
https://git.postgresql.org/pg/commitdiff/d392e9bdea957964e1fa6a5481e5adb5904d759a

- Add PGXS options to control TAP and isolation tests The following options are
added for extensions: - TAP_TESTS, to allow an extention to run TAP tests
which are the ones present in t/*.pl. A subset of tests can always be run
with the existing PROVE_TESTS for developers. - ISOLATION, to define a list
of isolation tests. - ISOLATION_OPTS, to pass custom options to
isolation_tester. A couple of custom Makefile targets have been accumulated
across the tree to cover the lack of facility in PGXS for a couple of releases
when using those test suites, which are all now replaced with the new flags,
without reducing the test coverage. This also fixes an issue with
contrib/bloom/, which had a custom target to trigger its TAP tests of its own
not part of the main check runs. Author: Michael Paquier Reviewed-by: Adam
Berlin, Álvaro Herrera, Tom Lane, Nikolay Shaplov, Arthur Zakirov Discussion:
https://postgr.es/m/20180906014849.GG2726@paquier.xyz
https://git.postgresql.org/pg/commitdiff/03faa4a8ddf04334850bf9631f305bda7a02f260

- Disable temporarily TAP tests for contrib/bloom/. The recent commit 03faa4a8
has enabled those tests, however several buildfarm members are complaining
about their stability on Windows and macOS. This will keep the buildfarm
green, while investigating the root problem. Discussion:
https://postgr.es/m/20181126003351.GE1776@paquier.xyz
https://git.postgresql.org/pg/commitdiff/b0b1f4183aa416d30afe35c5bef83d3e21f9b574

Thomas Munro pushed:

- PANIC on fsync() failure. On some operating systems, it doesn't make sense to
retry fsync(), because dirty data cached by the kernel may have been dropped
on write-back failure. In that case the only remaining copy of the data is in
the WAL. A subsequent fsync() could appear to succeed, but not have flushed
the data. That means that a future checkpoint could apparently complete
successfully but have lost data. Therefore, violently prevent any future
checkpoint attempts by panicking on the first fsync() failure. Note that we
already did the same for WAL data; this change extends that behavior to
non-temporary data files. Provide a GUC data_sync_retry to control this new
behavior, for users of operating systems that don't eject dirty data, and
possibly forensic/testing uses. If it is set to on and the write-back error
was transient, a later checkpoint might genuinely succeed (on a system that
does not throw away buffers on failure); if the error is permanent, later
checkpoints will continue to fail. The GUC defaults to off, meaning that we
panic. Back-patch to all supported releases. There is still a narrow window
for error-loss on some operating systems: if the file is closed and later
reopened and a write-back error occurs in the intervening time, but the inode
has the bad luck to be evicted due to memory pressure before we reopen, we
could miss the error. A later patch will address that with a scheme for
keeping files with dirty data open at all times, but we judge that to be too
complicated to back-patch. Author: Craig Ringer, with some adjustments by
Thomas Munro Reported-by: Craig Ringer Reviewed-by: Robert Haas, Thomas Munro,
Andres Freund Discussion:
https://postgr.es/m/20180427222842.in2e4mibx45zdth5%40alap3.anarazel.de
https://git.postgresql.org/pg/commitdiff/9ccdd7f66e3324d2b6d3dec282cfa9ff084083f1

- Don't forget about failed fsync() requests. If fsync() fails, md.c must keep
the request in its bitmap, so that future attempts will try again. Back-patch
to all supported releases. Author: Thomas Munro Reviewed-by: Amit Kapila
Reported-by: Andrew Gierth Discussion:
https://postgr.es/m/87y3i1ia4w.fsf%40news-spur.riddles.org.uk
https://git.postgresql.org/pg/commitdiff/1556cb2fc5c774c3f7390dd6fb19190ee0c73f8b

- Add WL_EXIT_ON_PM_DEATH pseudo-event. Users of the WaitEventSet and
WaitLatch() APIs can now choose between asking for WL_POSTMASTER_DEATH and
then handling it explicitly, or asking for WL_EXIT_ON_PM_DEATH to trigger
immediate exit on postmaster death. This reduces code duplication, since
almost all callers want the latter. Repair all code that was previously
ignoring postmaster death completely, or requesting the event but ignoring it,
or requesting the event but then doing an unconditional PostmasterIsAlive()
call every time through its event loop (which is an expensive syscall on
platforms for which we don't have USE_POSTMASTER_DEATH_SIGNAL support).
Assert that callers of WaitLatchXXX() under the postmaster remember to ask for
either WL_POSTMASTER_DEATH or WL_EXIT_ON_PM_DEATH, to prevent future bugs.
The only process that doesn't handle postmaster death is syslogger. It waits
until all backends holding the write end of the syslog pipe (including the
postmaster) have closed it by exiting, to be sure to capture any parting
messages. By using the WaitEventSet API directly it avoids the new assertion,
and as a by-product it may be slightly more efficient on platforms that have
epoll(). Author: Thomas Munro Reviewed-by: Kyotaro Horiguchi, Heikki
Linnakangas, Tom Lane Discussion:
https://postgr.es/m/CAEepm%3D1TCviRykkUb69ppWLr_V697rzd1j3eZsRMmbXvETfqbQ%40mail.gmail.com,
https://postgr.es/m/CAEepm=2LqHzizbe7muD7-2yHUbTOoF7Q+qkSD5Q41kuhttRTwA@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/cfdf4dc4fc9635ac8bf6eaaa5dbbcd364ab29f0c

- Fix assertion failure for SSL connections. Commit cfdf4dc4 added an assertion
that every WaitLatch() or similar handles postmaster death. One place did
not, but was missed in review and testing due to the need for an SSL
connection. Fix, by asking for WL_EXIT_ON_PM_DEATH. Reported-by: Christoph
Berg Discussion: https://postgr.es/m/20181124143845.GA15039%40msg.df7cb.de
https://git.postgresql.org/pg/commitdiff/ab69ea9feeb9a02c6299b5c1b786005552343f22

Peter Eisentraut pushed:

- pg_archivecleanup: Update file header comment a bit.
https://git.postgresql.org/pg/commitdiff/fc47e99a157a56a92dea679a74d546e28b253bb7

- Refine some guc.c help texts. These settings apply to communication with the
sending server, which is not necessarily a primary. Author: Sergei Kornilov
<sk(at)zsrv(dot)org>
https://git.postgresql.org/pg/commitdiff/a568cadaff2cfa0ec2a138ff4c5630050b03906b

- doc: Clarify CREATE TYPE ENUM documentation. The documentation claimed that
an enum type requires "one or more" labels, but since 1fd9883ff49, zero labels
are also allowed. Reported-by: Lukas Eder <lukas(dot)eder(at)gmail(dot)com> Bug: #15356
https://git.postgresql.org/pg/commitdiff/69bae23727253ca8b7616226020e241b8e923f4d

- Make WAL description output more consistent. The output for record types
XLOG_DBASE_CREATE and XLOG_DBASE_DROP used the order dbid/tablespaceid,
whereas elsewhere the order is tablespaceid/dbid[/relfilenodeid]. Flip the
order for those two types to make it consistent. Author: Jean-Christophe Arnu
<jcarnu(at)gmail(dot)com> Discussion:
https://www.postgresql.org/message-id/flat/CAHZmTm18Ln62KW-G8NYvO1wbBL3QU1E76Zep=DuHmg-zS2XFAg(at)mail(dot)gmail(dot)com/
https://git.postgresql.org/pg/commitdiff/2d9140ed266d702b305adeaf916086e1a586affe

- Add settings to control SSL/TLS protocol version. For example:
ssl_min_protocol_version = 'TLSv1.1' ssl_max_protocol_version = 'TLSv1.2'
Reviewed-by: Steve Singer <steve(at)ssinger(dot)info> Discussion:
https://www.postgresql.org/message-id/flat/1822da87-b862-041a-9fc2-d0310c3da173(at)2ndquadrant(dot)com
https://git.postgresql.org/pg/commitdiff/e73e67c719593c1c16139cc6c516d8379f22f182

- Make detection of SSL_CTX_set_min_proto_version more portable. As already
explained in configure.in, using the OpenSSL version number to detect presence
of functions doesn't work, because LibreSSL reports incompatible version
numbers. Fortunately, the functions we need here are actually macros, so we
can just test for them directly.
https://git.postgresql.org/pg/commitdiff/ea8bc349bd1d92a882d636b535723b36366d9fbe

- doc: Fix typo.
https://git.postgresql.org/pg/commitdiff/a80dcac60cdc4b96a296d33288afa74a80f42cca

- Integrate recovery.conf into postgresql.conf. recovery.conf settings are now
set in postgresql.conf (or other GUC sources). Currently, all the affected
settings are PGC_POSTMASTER; this could be refined in the future case by case.
Recovery is now initiated by a file recovery.signal. Standby mode is
initiated by a file standby.signal. The standby_mode setting is gone. If a
recovery.conf file is found, an error is issued. The trigger_file setting has
been renamed to promote_trigger_file as part of the move. The documentation
chapter "Recovery Configuration" has been integrated into "Server
Configuration". pg_basebackup -R now appends settings to postgresql.auto.conf
and creates a standby.signal file. Author: Fujii Masao
<masao(dot)fujii(at)gmail(dot)com> Author: Simon Riggs <simon(at)2ndquadrant(dot)com> Author:
Abhijit Menon-Sen <ams(at)2ndquadrant(dot)com> Author: Sergei Kornilov <sk(at)zsrv(dot)org>
Discussion:
https://www.postgresql.org/message-id/flat/607741529606767(at)web3g(dot)yandex(dot)ru/
https://git.postgresql.org/pg/commitdiff/2dedf4d9a899b36d1a8ed29be5efbd1b31a8fe85

Álvaro Herrera pushed:

- Disallow COPY FREEZE on partitioned tables. This didn't actually work: COPY
would fail to flush the right files, and instead would try to flush a
non-existing file, causing the whole transaction to fail. Cope by raising an
error as soon as the command is sent instead, to avoid a nasty later surprise.
Of course, it would be much better to make it work, but we don't have a patch
for that yet, and we don't know if we'll want to backpatch one when we do.
Reported-by: Tomas Vondra Author: David Rowley Reviewed-by: Amit Langote,
Steve Singer, Tomas Vondra
https://git.postgresql.org/pg/commitdiff/5c9a5513a3967367e5fcc9237bc4146f4b24b408

- psql: Show IP address in \conninfo. When hostaddr is given, the actual IP
address that psql is connected to can be totally unexpected for the given
host. The more verbose output we now generate makes things clearer. Since
the "host" and "hostaddr" parts of the conninfo could come from different
sources (say, one of them is in the service specification or a URI-style
conninfo and the other is not), this is not as silly as it may first appear.
This is also definitely useful if the hostname resolves to multiple addresses.
Author: Fabien Coelho Reviewed-by: Pavel Stehule, Arthur Zakirov Discussion:
https://postgr.es/m/alpine.DEB.2.21.1810261532380.27686@lancre
https://postgr.es/m/alpine.DEB.2.21.1808201323020.13832@lancre
https://git.postgresql.org/pg/commitdiff/6e5f8d489acccdc50a35a1b7db8e72b5ad579253

- psql: Describe partitioned tables/indexes as such. In \d and \z, instead of
conflating partitioned tables and indexes with plain ones, set the "type"
column and table title differently to make the distinction obvious. A simple
ease-of-use improvement. Author: Pavel Stehule, Michaël Paquier, Álvaro
Herrera Reviewed-by: Amit Langote Discussion:
https://postgr.es/m/CAFj8pRDMWPgijpt_vPj1t702PgLG4Ls2NCf+rEcb+qGPpossmg@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/d56e0fde828b95497ce9acfb4ce18bab64803c82

- Fix PartitionDispatchData vertical whitespace. Per David Rowley Discussion:
https://postgr.es/m/CAKJS1f-MstvBWdkOzACsOHyBgj2oXcBM8kfv+NhVe-Ux-wq9Sg@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/ee07e38c14a06a8479d2605406345c4a868465ae

- instr_time.h: add INSTR_TIME_SET_CURRENT_LAZY. Sets the timestamp to current
if not already set. Will acquire more callers momentarily. Author: Fabien
Coelho Discussion:
https://postgr.es/m/alpine.DEB.2.21.1808111104320.1705@lancre
https://git.postgresql.org/pg/commitdiff/6f7d02aa60b711c2a61d12b2f2f7ff4c5d5d6df4

- Fix typo in commit 6f7d02aa60b7. Per pink buildfarm.
https://git.postgresql.org/pg/commitdiff/03e10b962f3ada1953cf5bfc93b04459502e0993

- Rework the pgbench state machine code for clarity. This commit continues the
code improvements started by commit 12788ae49e19. With this commit, state
machine transitions are better contained in the routine that was called
doCustom() and is now called advanceConnectionState -- the resulting code is
easier to reason about, since there are no state changes occuring in the outer
layer. This change is prompted by future patches to add more features to
pgbench, which will need to effect some more surgery to this code. Fabien's
original had all the machine state changes inside one routine, but I (Álvaro)
thought that a subroutine to handle command execution is more straightforward
to review, so this commit does not match Fabien's submission closely. If
something is broken, it's probably my fault. Author: Fabien Coelho, Álvaro
Herrera Reviewed-by: Kirk Jamison Discussion:
https://postgr.es/m/alpine.DEB.2.21.1808111104320.1705@lancre
https://git.postgresql.org/pg/commitdiff/3bac77c48f166b9024a5ead984df73347466ae12

- Don't allow partitioned indexes in pg_global tablespace. Missing in
dfa608141982. Author: David Rowley Discussion:
https://postgr.es/m/CAKJS1f-M3NMTCpv=vDfkoqHbMPFf=3-Z1ud=+1DHH00tC+zLaQ@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/de38ce1b831eef0c1eddcb2d1439f134cd9cd02c

- Silence compiler warnings. Commit cfdf4dc4fc96 left a few unnecessary
assignments, one of which caused compiler warnings, as reported by Erik
Rijkers. Remove them all. Discussion:
https://postgr.es/m/df0dcca2025b3d90d946ecc508ca9678@xs4all.nl
https://git.postgresql.org/pg/commitdiff/3be5fe2b107fae24e03c9d29d7bd7c7ad5345787

Tom Lane pushed:

- Fix configure's AC_CHECK_DECLS tests to work correctly with clang. The test
case that Autoconf uses to discover whether a function has been declared
doesn't work reliably with clang, because clang reports a warning not an error
if the name is a known built-in function. On some platforms, this results in
a lot of compile-time warnings about strlcpy and related functions not having
been declared. There is a fix for this (by Noah Misch) in the upstream
Autoconf sources, but since they've not made a release in years and show no
indication of doing so anytime soon, let's just absorb their fix directly. We
can revert this when and if we update to a newer Autoconf release. Back-patch
to all supported branches. Discussion:
https://postgr.es/m/26819.1542515567@sss.pgh.pa.us
https://git.postgresql.org/pg/commitdiff/16fbac39ff68a737317f6fc7be78cc93a14856af

- Postpone LLVM-related uses of AC_CHECK_DECLS. Calling AC_CHECK_DECLS before
we've finished setting up the compiler's CFLAGS seems like a pretty risky
proposition, especially now that the first use of that macro will result in a
test to see whether the compiler gives warning or error for undeclared
built-in functions. That answer could very easily get changed later than
where PGAC_LLVM_SUPPORT is called; furthermore, it's hardly unlikely that
flags such as -D_GNU_SOURCE could change visibility of declarations. Hence,
be a little less cavalier about where to do LLVM-related tests. This results
in v11 and HEAD doing the warning-or-error check at the same place in the
script as older branches are doing it, which seems like a good thing. Per
further thought about commits 0b59b0e8b and 16fbac39f.
https://git.postgresql.org/pg/commitdiff/640a4ba052c2dd7aaf600ca13b6d3afca60adc05

- Update config/ax_pthread.m4 to latest upstream version. This change doesn't
fix any bugs that we've heard about, but it seems like a good idea on general
principles to track upstream occasionally. Discussion:
https://postgr.es/m/3320.1542647565@sss.pgh.pa.us
https://git.postgresql.org/pg/commitdiff/df303aff6601a893cf83c26ffd6c7eb5fb6b067e

- Handle EPIPE more sanely when we close a pipe reading from a program.
Previously, any program launched by COPY TO/FROM PROGRAM inherited the
server's setting of SIGPIPE handling, i.e. SIG_IGN. Hence, if we were doing
COPY FROM PROGRAM and closed the pipe early, the child process would see EPIPE
on its output file and typically would treat that as a fatal error, in turn
causing the COPY to report error. Similarly, one could get a failure report
from a query that didn't read all of the output from a contrib/file_fdw
foreign table that uses file_fdw's PROGRAM option. To fix, ensure that child
programs inherit SIG_DFL not SIG_IGN processing of SIGPIPE. This seems like
an all-around better situation since if the called program wants some
non-default treatment of SIGPIPE, it would expect to have to set that up for
itself. Then in COPY, if it's COPY FROM PROGRAM and we stop reading short of
detecting EOF, treat a SIGPIPE exit from the called program as a non-error
condition. This still allows us to report an error for any case where the
called program gets SIGPIPE on some other file descriptor. As coded, we won't
report a SIGPIPE if we stop reading as a result of seeing an in-band EOF
marker (e.g. COPY BINARY EOF marker). It's somewhat debatable whether we
should complain if the called program continues to transmit data after an EOF
marker. However, it seems like we should avoid throwing error in any
questionable cases, especially in a back-patched fix, and anyway it would take
additional code to make such an error get reported consistently. Back-patch
to v10. We could go further back, since COPY FROM PROGRAM has been around
awhile, but AFAICS the only way to reach this situation using core or contrib
is via file_fdw, which has only supported PROGRAM sources since v10. The COPY
statement per se has no feature whereby it'd stop reading without having hit
EOF or an error already. Therefore, I don't see any upside to back-patching
further that'd outweigh the risk of complaints about behavioral change. Per
bug #15449 from Eric Cyr. Patch by me, review by Etsuro Fujita and Kyotaro
Horiguchi Discussion:
https://postgr.es/m/15449-1cf737dd5929450e@postgresql.org
https://git.postgresql.org/pg/commitdiff/ffa4cbd623dd69f9fa99e5e92426928a5782cf1a

- Add needed #include. Per POSIX, WIFSIGNALED and related macros are provided
by <sys/wait.h>. Apparently on Linux they're also pulled in by some other
inclusions, but BSD-ish systems are pickier. Fixes portability issue in
ffa4cbd62. Per buildfarm.
https://git.postgresql.org/pg/commitdiff/cb09903fe63132a35e4b217bc394882b05c0c6f3

- Doc: rework introductory documentation about covering indexes. Documenting
INCLUDE in the section about unique indexes is confusing, as complained of by
Emilio Platzer. Furthermore, it entirely failed to explain why you might want
to use the feature. The section about index-only scans is really the right
place; it already talked about making such things the hard way. Rewrite that
text to describe INCLUDE as the normal way to make a covering index. Also,
move that section up a couple of places, as it now seems more important than
some of the stuff we had before it. It still has to be after expression and
partial indexes, since otherwise some of it would involve forward references.
Discussion:
https://postgr.es/m/154031939560.30897.14677735588262722042@wrigleys.postgresql.org
https://git.postgresql.org/pg/commitdiff/fe375d33a184b98510d08f45f55da1338cde974a

- Fix another crash in json{b}_populate_recordset and json{b}_to_recordset.
populate_recordset_worker() failed to consider the possibility that the
supplied JSON data contains no rows, so that update_cached_tupdesc never got
called. This led to a null-pointer dereference since commit 9a5e8ed28; before
that it led to a bogus "set-valued function called in context that cannot
accept a set" error. Fix by forcing the update to happen. Per bug #15514.
Back-patch to v11 as 9a5e8ed28 was. (If we were excited about the bogus
error, we could perhaps go back further, but it'd take more work to figure out
how to fix it in older branches. Given the lack of field complaints about
that aspect, I'm not excited.) Discussion:
https://postgr.es/m/15514-59d5b4c4065b178b@postgresql.org
https://git.postgresql.org/pg/commitdiff/eba2ce17121f198316d050e71d8bd049a43783ba

- Remove bogus file.
https://git.postgresql.org/pg/commitdiff/0f75a466ac41a19214d22de73cceeb5dc792411a

- Clamp semijoin selectivity to be not more than inner-join selectivity. We
should never estimate the output of a semijoin to be more rows than we
estimate for an inner join with the same input rels and join condition; it's
obviously impossible for that to happen. However, given the relatively poor
quality of our semijoin selectivity estimates --- particularly, but not only,
in cases where we punt and return a default estimate --- we did often deliver
such estimates. To improve matters, calculate both estimates inside
eqjoinsel() and take the smaller one. The bulk of this patch is just
mechanical refactoring to avoid repetitive information lookup when we call
both eqjoinsel_semi and eqjoinsel_inner. The actual new behavior is just
selec = Min(selec, inner_rel->rows * selec_inner); which looks a bit odd but
is correct because of our different definitions for inner and semi join
selectivity. There is one ensuing plan change in the regression tests, but it
looks reasonable enough (and checking the actual row counts shows that the
estimate moved closer to reality, not further away). Per bug #15160 from
Alexey Ermakov. Although this is arguably a bug fix, I won't risk
destabilizing plan choices in stable branches by back-patching. Tom Lane,
reviewed by Melanie Plageman Discussion:
https://postgr.es/m/152395805004.19366.3107109716821067806@wrigleys.postgresql.org
https://git.postgresql.org/pg/commitdiff/a314c34079cf06d05265623dd7c056f8fa9d577f

- Add a 64-bit hash function for type citext. Amul Sul, reviewed by Hironobu
Suzuki Discussion:
https://postgr.es/m/CAAJ_b947JjnNr9Cp45iNjSqKf6PA5mCTmKsRwPjows93YwQrmw@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/48c41fa97480756b70ad095fe8d59199df9b12c7

- Add a 64-bit hash function for type hstore. There's some question about the
correctness of the hash function, but if it's wrong, the 32-bit version is
also wrong. Amul Sul, reviewed by Hironobu Suzuki Discussion:
https://postgr.es/m/CAAJ_b947JjnNr9Cp45iNjSqKf6PA5mCTmKsRwPjows93YwQrmw@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/eb6f29141bed9dc95cb473614c30f470ef980705

- Fix float-to-integer coercions to handle edge cases correctly. ftoi4 and its
sibling coercion functions did their overflow checks in a way that looked
superficially plausible, but actually depended on an assumption that the MIN
and MAX comparison constants can be represented exactly in the float4 or
float8 domain. That fails in ftoi4, ftoi8, and dtoi8, resulting in a
possibility that values near the MAX limit will be wrongly converted (to
negative values) when they need to be rejected. Also, because we compared
before rounding off the fractional part, the other three functions threw
errors for values that really ought to get rounded to the min or max integer
value. Fix by doing rint() first (requiring an assumption that it handles NaN
and Inf correctly; but dtoi8 and ftoi8 were assuming that already), and by
comparing to values that should coerce to float exactly, namely INTxx_MIN and
-INTxx_MIN. Also remove some random cosmetic discrepancies between these six
functions. Per bug #15519 from Victor Petrovykh. This should get
back-patched, but first let's see what the buildfarm thinks of it --- I'm not
too sure about portability of some of the regression test cases. Patch by me;
thanks to Andrew Gierth for analysis and discussion. Discussion:
https://postgr.es/m/15519-4fc785b483201ff1@postgresql.org
https://git.postgresql.org/pg/commitdiff/cbdb8b4c0155b2d9679ecd79b886c29c2730b85d

- Adjust new test case for more portability. Early returns from the buildfarm
say that most critters are good with commit cbdb8b4c0, but gaur gives
unexpected results with the test case involving a float8 that's
one-ULP-less-than-2^63. It appears that that platform's version of rint()
rounds that value up to 2^63 instead of leaving it be. This is possibly a
bug, and it's also possible that no other platform anybody is using anywhere
behaves likewise. Still, the point of the test is not to insist that
everybody's rint() behaves exactly the same. Let's use
two-ULPs-less-than-2^63 instead, which I've tested to act the same on gaur as
on more modern hardware. (This is, more or less, exactly the portability
issue I'd feared might arise...) Discussion:
https://postgr.es/m/15519-4fc785b483201ff1@postgresql.org
https://git.postgresql.org/pg/commitdiff/452b637d4b02e4aa09d45a34d7a048d3f70116bb

Robert Haas pushed:

- Reduce unnecessary list construction in RelationBuildPartitionDesc. The
'partoids' list which was constructed by the previous version of this code was
necessarily identical to 'inhoids'. There's no point to duplicating the list,
so avoid that. Instead, construct the array representation directly from the
original 'inhoids' list. Also, use an array rather than a list for
'boundspecs'. We know exactly how many items we need to store, so there's
really no reason to use a list. Using an array instead reduces the number of
memory allocations we perform. Patch by me, reviewed by Michael Paquier and
Amit Langote, the latter of whom also helped with rebasing.
https://git.postgresql.org/pg/commitdiff/7ee5f88e650619ee619f7e2854ac50ff59d0b0c7

Andres Freund pushed:

- Remove WITH OIDS support, change oid catalog column visibility. Previously
tables declared WITH OIDS, including a significant fraction of the catalog
tables, stored the oid column not as a normal column, but as part of the tuple
header. This special column was not shown by default, which was somewhat odd,
as it's often (consider e.g. pg_class.oid) one of the more important parts of
a row. Neither pg_dump nor COPY included the contents of the oid column by
default. The fact that the oid column was not an ordinary column necessitated
a significant amount of special case code to support oid columns. That already
was painful for the existing, but upcoming work aiming to make table storage
pluggable, would have required expanding and duplicating that "specialness"
significantly. WITH OIDS has been deprecated since 2005 (commit
ff02d0a05280e0). Remove it. Removing includes: - CREATE TABLE and ALTER
TABLE syntax for declaring the table to be WITH OIDS has been removed (WITH
(oids[ = true]) will error out) - pg_dump does not support dumping tables
declared WITH OIDS and will issue a warning when dumping one (and ignore the
oid column). - restoring an pg_dump archive with pg_restore will warn when
restoring a table with oid contents (and ignore the oid column) - COPY will
refuse to load binary dump that includes oids. - pg_upgrade will error out
when encountering tables declared WITH OIDS, they have to be altered to remove
the oid column first. - Functionality to access the oid of the last inserted
row (like. plpgsql's RESULT_OID, spi's SPI_lastoid, ...) has been removed.
The syntax for declaring a table WITHOUT OIDS (or WITH (oids = false) for
CREATE TABLE) is still supported. While that requires a bit of support code,
it seems unnecessary to break applications / dumps that do not use oids, and
are explicit about not using them. The biggest user of WITH OID columns was
postgres' catalog. This commit changes all 'magic' oid columns to be columns
that are normally declared and stored. To reduce unnecessary query breakage
all the newly added columns are still named 'oid', even if a table's column
naming scheme would indicate 'reloid' or such. This obviously requires
adapting a lot code, mostly replacing oid access via HeapTupleGetOid() with
access to the underlying Form_pg_*->oid column. The bootstrap process now
assigns oids for all oid columns in genbki.pl that do not have an explicit
value (starting at the largest oid previously used), only oids assigned later
by oids will be above FirstBootstrapObjectId. As the oid column now is a
normal column the special bootstrap syntax for oids has been removed. Oids
are not automatically assigned during insertion anymore, all backend code
explicitly assigns oids with GetNewOidWithIndex(). For the rare case that
insertions into the catalog via SQL are called for the new pg_nextoid()
function can be used (which only works on catalog tables). The fact that oid
columns on system tables are now normal columns means that they will be
included in the set of columns expanded by * (i.e. SELECT * FROM pg_class will
now include the table's oid, previously it did not). It'd not technically be
hard to hide oid column by default, but that'd mean confusing behavior would
either have to be carried forward forever, or it'd cause breakage down the
line. While it's not unlikely that further adjustments are needed, the
scope/invasiveness of the patch makes it worthwhile to get merge this now.
It's painful to maintain externally, too complicated to commit after the code
code freeze, and a dependency of a number of other patches. Catversion bump,
for obvious reasons. Author: Andres Freund, with contributions by John Naylor
Discussion:
https://postgr.es/m/20180930034810.ywp2c7awz7opzcfr@alap3.anarazel.de
https://git.postgresql.org/pg/commitdiff/578b229718e8f15fa779e20f086c4b6bb3776106

- Fix sepgsql compile error caused by oid removal. Per buildfarm animal
rhinoceros. I (Andres) missed replacing a few uses of ObjectIdAttributeNumber
in sepgsql. It's quite probable that the sepgsql test output will need more
adapting than done in 578b22... Author: Thomas Munro Discussion:
https://postgr.es/m/CAEepm=2Sk+66HJV8FLDfm_sKTn22j7cWTY_Y1Rok3RxeWL_Y0w@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/937e4e509998de93d5e1c04d312f1b13ef153210

- Blind attempt at fixing sepgsql output for 578b22.
https://git.postgresql.org/pg/commitdiff/7306d5e920deb16f781797640715bf03f8d50d8c

Bruce Momjian pushed:

- doc: adjust time zone names text. Reported-by: Kevin <kcolagio(at)gmail(dot)com>
Discussion:
https://postgr.es/m/154082462281.30897.14043119084654378035@wrigleys.postgresql.org
Backpatch-through: 9.4
https://git.postgresql.org/pg/commitdiff/7906de847f229f391b9e6b5892b4b4a89f29edb4

- doc: adjust time zone names text, v2. Removed one too many words. Fix for
7906de847f229f391b9e6b5892b4b4a89f29edb4. Reported-by: Thomas Munro
Backpatch-through: 9.4
https://git.postgresql.org/pg/commitdiff/9cf5d3c4867aaa6804be8fdf5a29bf28d0096320

Andrew Gierth pushed:

- Avoid crashes in contrib/intarray gist__int_ops (bug #15518). 1. Integer
overflow in internal_size could result in memory corruption in decompression
since a zero-length array would be allocated and then written to. This leads
to crashes or corruption when traversing an index which has been populated
with sufficiently sparse values. Fix by using int64 for computations and
checking for overflow. 2. Integer overflow in g_int_compress could cause
pessimal merge choices, resulting in unnecessarily large ranges (which would
in turn trigger issue 1 above). Fix by using int64 again. 3. Even without
overflow, array sizes could become large enough to cause unexplained memory
allocation errors. Fix by capping the sizes to a safe limit and report actual
errors pointing at gist__intbig_ops as needed. 4. Large inputs to the
compression function always consist of large runs of consecutive integers, and
the compression loop was processing these one at a time in an O(N^2) manner
with a lot of overhead. The expected runtime of this function could easily
exceed 6 months for a single call as a result. Fix by performing a linear-time
first pass, which reduces the worst case to something on the order of seconds.
Backpatch all the way, since this has been wrong forever. Per bug #15518 from
report from irc user "dymk", analysis and patch by me. Discussion:
https://postgr.es/m/15518-799e426c3b4f8358@postgresql.org
https://git.postgresql.org/pg/commitdiff/757c5182f2c42af528386c0419c60c111c678cab

- Fix hstore hash function for empty hstores upgraded from 8.4. Hstore data
generated on pg 8.4 and pg_upgraded to current versions remains in its
original on-disk format unless modified. The same goes for values generated by
the addon hstore-new module on pre-9.0 versions. (The hstoreUpgrade function
converts old values on the fly when read in, but the on-disk value is not
modified by this.) Since old-format empty hstores (and hstore-new hstores)
have representations compatible with the new format, hstoreUpgrade thought it
could get away without modifying such values; but this breaks hstore_hash (and
the new hstore_hash_extended) which assumes bit-perfect matching between
semantically identical hstore values. Only one bit actually differs (the "new
version" flag in the count field) but that of course is enough to break the
hash. Fix by making hstoreUpgrade unconditionally convert all old values to
new format. Backpatch all the way, even though this changes a hash value in
some cases, because in those cases the hash value is already failing - for
example, a hash join between old- and new-format empty hstores will be failing
to match, or a hash index on an hstore column containing an old-format empty
value will be failing to find the value since it will be searching for a hash
derived from a new-format datum. (There are no known field reports of this
happening, probably because hashing of hstores has only been useful in limited
circumstances and there probably isn't much upgraded data being used this
way.) Per concerns arising from discussion of commit eb6f29141be. Original bug
is my fault. Discussion:
https://postgr.es/m/60b1fd3b-7332-40f0-7e7f-f2f04f777747%402ndquadrant.com
https://git.postgresql.org/pg/commitdiff/d5890f49da6a77b1325a3f5822c6b092a2cd41ae

== Pending Patches ==

Julien Demoor sent in another revision of a patch to add a collapse_mode
parameter to pg_notify(), which enables redundant pg_notifies in situations
where delivering the NOTIFY once suffices.

Amit Khandekar sent in a patch to populate &slot->tts_tid wherever tuple->t_self
changes and slot-ify trigger tuples.

Nikolay Shaplov sent in another revision of a patch to split the functionality
of StdRdOptions into individual binary reloptions representations for each
relation kind.

Sergey Cherkashin sent in a patch to add a verification of the connection with
the client during the execution of the SQL query via a new
client_connection_check_interval GUC.

Michael Banck sent in another revision of a patch to add TAP tests for
pg_verify_checksums.

Andrey V. Lepikhov sent in a patch to clarify comments in readRecordBuf.

Pavel Stěhule and Michaël Paquier traded patches to add a \dP for partitioned
things to psql, along with improvements to \di for partitioned indexes.

Amit Langote sent in two revisions of a patch to fix some comments about
EquivalenceClasses.

Dilip Kumar sent in another revision of a patch to provide an interface for
prepare, insert, or fetch the undo records.

John Naylor sent in a patch to update bki.sgml for the OID removal patch.

Haozhou Wang sent in two more revisions of a patch to add hooks to enable,
amongh other things, extensions to implement disk quotas.

David Rowley sent in another revision of a patch to add a documentation section
appendix detailing some limitations of PostgreSQL.

Pavel Stěhule sent in another revision of a patch to implement schema variables.

Iwata Aya sent in another revision of a patch to enable libpq debug logging.

Amit Langote sent in another revision of a patch to speed up planning with
partitions.

Alexander Kuzmenkov sent in another revision of a patch to remove unneeded
self-joins.

Christoph Berg sent in a patch to log the PostgreSQL version number on startup.

Haribabu Kommi sent in a patch to remove the release notes for unsupported
versions of PostgreSQL from HEAD.

Dmitry Dolgov sent in another revision of a patch to implement index skip scan.

Masahiko Sawada sent in a patch to fix an issue in gcov.

Adrien Nayrat sent in another revision of a patch to add a new GUC,
log_statement_sample_rate, to sample log queries.

Michael Banck sent in another revision of a patch to enable online verification
of disk page checksums.

Álvaro Herrera and Fabien COELHO traded patches to enable pgbench to store
SELECT results into variables.

Michaël Paquier and Artur Zakirov traded patches to add extension options to
control TAP and isolation tests.

David Rowley sent in another revision of a patch to allow Append to be used in
place of MergeAppend for some cases.

Paul Guo sent in a patch to allow the fact that a wal sender is stopping to
affect SyncRepReleaseWaiters(). Previously, only the fact that a wal sender was
streaming did.

Christoph Berg sent in a patch to switch pg_regress to output unified diffs by
default.

Evgeniy Efimkin sent in another revision of a patch to add options to the DDL
around SUBSCRIPTIONs.

Patrick Francelle sent in another revision of a patch to document more clearly
what CHECK constraints are and are not good for.

Andres Freund sent in a patch to fix some breakage of pg_dump caused by the
patch which removed "magic" OIDs.

Amit Langote sent in a patch to fix a typo in src/backend/executor/execPartition.c.

David Rowley sent in a patch to fix index locking for deletes.

David Rowley sent in a patch to fix some issues in a previous patch which
improved TID scans.

Kyotaro HORIGUCHI and David Fetter traded patches to enable debugging of tab
completion in psql.

David Rowley sent in a patch to delay locking of partitions during INSERT and
UPDATE.

Thomas Munro sent in another revision of a patch to refactor the checkpointer's
data sync request queue and fix a deadlock by sending without content lock, but
still marked BM_DIRTY.

Thomas Munro sent in a patch to avoid counting zero-filled buffers as 'read' in
EXPLAIN.

Sergey Cherkashin sent in another revision of a patch to implement \dA (access
method) for psql.

Konstantin Knizhnik sent in a patch to make index_getattr more efficient.

Tony REIX sent in a patch to make it possible for PostgreSQL to use Sys-V shared
memory on AIX.

Andrey Borodin sent in a patch to use a correct locking protocol during GIN
posting tree vacuum.

David Rowley sent in two revisions of a patch to allow newly created partitions
to inherit their parent's tablespace.

John Naylor sent in another revision of a patch to prevent creating a free space
map for small tables.

Surafel Temesgen sent in another revision of a patch to implement COPY ... FROM
... FILTER.

Tomáš Vondra sent in another revision of a patch to fix an infelicity between
logical decoding and VACUUM FULL / CLUSTER on table with TOAST-ed data.

David Rowley sent in another revision of a patch to remove single-subpath Append
and MergeAppend nodes.

Paul A Jungwirth sent in two revisions of a patch to implement temporal PRIMARY
KEYs.

Fabien COELHO sent in a patch to fix up some infelicities in the pgbench state
change patch.

Surafel Temesgen sent in another revision of a patch to implement FETCH FIRST
... WITH TIES.

Fabien COELHO sent in another revision of a patch to add a progress indicator to
pgbench.

Jeff Janes sent in two revisions of a patch to avoid checking for a trigger file
on a streaming replica when no trigger file is configured.

Peter Geoghegan sent in another revision of a patch to make all nbtree entries
unique by having heap TIDs participate in comparisons.

Noah Misch sent in another PoC of a patch to create a function trust mechanism.

Surafel Temesgen sent in another revision of a patch to implement FETCH FIRST
... PERCENT.

Dmitry Dolgov sent in another revision of a patch to implement generic type
subscripting.

Dmitry Dolgov sent in another revision of a patch to improve the algorithms for
joining partitions.

Sergei Kornilov sent in a WIP patch to make primary_conninfo and primary_slot_name
PGC_SIGHUP.

Tom Lane sent in a patch to fix tab completion in psql for the latex-longtable
format.

Masahiko Sawada sent in another revision of a patch to create a copy function
for logical and physical replication slots.