== PostgreSQL Weekly News - April 28, 2019 ==

== PostgreSQL Weekly News - April 28, 2019 ==

pgibz will be held in Ibiza, Spain on June 19-23, 2019. The CfP is open.
https://pgibz.io/

PostgreSQL Conference Europe 2019 will be held on October 15-18, 2019 in Milan,
Italy. The CfP is open through July 15, 2019 at https://2019.pgconf.eu/callforpapers
https://2017.pgconf.eu/

== PostgreSQL Product News ==

WAL-G 0.2.9, a backup management system for PostgreSQL written in Go, released.
https://github.com/wal-g/wal-g/releases

pg_partman 4.1.0, a management system for partitioned tables, released.
https://github.com/pgpartman/pg_partman

postgres-checkup 1.0, a tool that automates detailed health checks of PostgreSQL
clusters, released.
https://gitlab.com/postgres-ai-team/postgres-checkup/tags/1.0

pgFormatter 3.4, a formatter/beautifier for SQL code, released.
http://sqlformat.darold.net/

== PostgreSQL Jobs for April ==

http://archives.postgresql.org/pgsql-jobs/2019-04/

== PostgreSQL Local ==

The German-speaking PostgreSQL Conference 2019 will take place on May 10, 2019
in Leipzig.
http://2019.pgconf.de/

PGDay.IT 2019 will take place May 16th and May 17th in Bologna, Italy.
https://2019.pgday.it/en/

PgConf Belgium will take place on May 17, 2019 at the UCLL Campus in Haasrode
(near Leuven). Registration is open.
http://pgconf.be

PGCon 2019 will take place in Ottawa on May 28-31, 2019. Registration is open.
https://www.pgcon.org/2019

Swiss PGDay 2019 will take place in Rapperswil (near Zurich) on June 28, 2019.
Registration is open.
http://www.pgday.ch/2019/

PostgresLondon 2019 will be July 2-3, 2019 with an optional training day on
July 1.
http://postgreslondon.org

PGConf.Brazil 2019 will take place August 1-3, 2019 in São Paulo.
http://pgconf.com.br

The first Austrian pgDay, will take place September 6, 2019 at the Hilton Garden
Inn in Wiener Neustadt.
https://pgday.at/en/

PostgresConf South Africa 2019 will take place in Johannesburg on October 8-9, 2019
The Call for Papers is open through June 30, 2019.
https://postgresconf.org/conferences/SouthAfrica2019

== PostgreSQL in the News ==

Planet PostgreSQL: http://planet.postgresql.org/

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm PST8PDT to david(at)fetter(dot)org(dot)

== Applied Patches ==

Noah Misch pushed:

- Test both 0.0.0.0 and 127.0.0.x addresses to find a usable port. Commit
c098509927f9a49ebceb301a2cb6a477ecd4ac3c changed PostgresNode::get_new_node()
to probe 0.0.0.0 instead of 127.0.0.1, but the new test was less effective for
Windows native Perl. This increased the failure rate of buildfarm members
bowerbird and jacana. Instead, test 0.0.0.0 and concrete addresses. This
restores the old level of defense, but the algorithm is still subject to its
longstanding time of check to time of use race condition. Back-patch to 9.6,
like the previous change. Discussion:
https://postgr.es/m/GrdLgAdUK9FdyZg8VIcTDKVOkys122ZINEb3CjjoySfGj2KyPiMKTh1zqtRp0TAD7FJ27G-OBB3eplxIB5GhcQH5o8zzGZfp0MuJaXJxVxk=@yesql.se
https://git.postgresql.org/pg/commitdiff/4ab02e8156270592c1eae03d5b9ee36bb8c761c2

- Don't write to stdin of a test process that could have already exited.
Instead, close that stdin. Per buildfarm member conchuela. Back-patch to
9.6, where the test was introduced. Discussion:
https://postgr.es/m/26478.1555373328@sss.pgh.pa.us
https://git.postgresql.org/pg/commitdiff/e12a4726127b90b8fc5e75690fe4527f1866412d

Michaël Paquier pushed:

- Fix SHOW ALL command for non-superusers with replication connection. Since
Postgres 10, SHOW commands can be triggered with replication connections in a
WAL sender context, however it missed that a transaction context is needed for
syscache lookups. This commit makes sure that the syscache lookups can happen
correctly by setting a transaction context when running SHOW commands in a WAL
sender. Superuser-only parameters can be displayed using SHOW commands not
only to superusers, but also to members of system role pg_read_all_settings,
which requires a syscache lookup to check if the connected role is a member of
this system role or not, or the instance crashes. Superusers do not need to
check the syscache so it worked correctly in this case. New tests are added
to cover this issue. Reported-by: Alexander Kukushkin Author: Michael Paquier
Reviewed-by: Álvaro Herrera Discussion:
https://postgr.es/m/15734-2daa8761eeed8e20@postgresql.org Backpatch-through:
10
https://git.postgresql.org/pg/commitdiff/c34677fdaa73f089d557554a9cd479b9bd5b5143

- Remove duplicate assignment when initializing logical decoder context. The
private data in the WAL reader is already getting set when allocating it.
Author: Antonin Houska Reviewed-by: Tom Lane Discussion:
https://postgr.es/m/30563.1555329094@localhost
https://git.postgresql.org/pg/commitdiff/5ed4b123b6a678bc6243047846870f7f46e9c67b

- Rework handling of invalid indexes with REINDEX CONCURRENTLY. Per discussion
with others, allowing REINDEX INDEX CONCURRENTLY to work for invalid indexes
when working directly on them can have a lot of value to unlock situations
with invalid indexes without having to use a dance involving DROP INDEX
followed by an extra CREATE INDEX CONCURRENTLY (which would not work for
indexes with constraint dependency anyway). This also does not create extra
bloat on the relation involved as this works on individual indexes, so let's
enable it. Note that REINDEX TABLE CONCURRENTLY still bypasses invalid
indexes as we don't want to bloat the number of indexes defined on a relation
in the event of multiple and successive failures of REINDEX CONCURRENTLY.
More regression tests are added to cover those behaviors, using an invalid
index created with CREATE INDEX CONCURRENTLY. Reported-by: Dagfinn Ilmari
Mannsåker, Álvaro Herrera Author: Michael Paquier Reviewed-by: Peter
Eisentraut, Dagfinn Ilmari Mannsåker Discussion:
https://postgr.es/m/20190411134947.GA22043@alvherre.pgsql
https://git.postgresql.org/pg/commitdiff/a6dcf9df4d91ff0db23579f9114079abe6f3e2bf

- Simplify some ERROR paths clearing wait events and transient files. Transient
files and wait events get normally cleaned up when seeing an exception (be it
in the context of a transaction for a backend or another process like the
checkpointer), hence there is little point in complicating error code paths to
do this work. This shaves a bit of code, and removes some extra handling with
errno which needed to be preserved during the cleanup steps done.
Reported-by: Masahiko Sawada Author: Michael Paquier Reviewed-by: Tom Lane,
Masahiko Sawada Discussion:
https://postgr.es/m/CAD21AoDhHYVq5KkXfkaHhmjA-zJYj-e4teiRAJefvXuKJz1tKQ@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/47ac2033d460cefbbda2e39732e18de521dc6a36

- Fix thinko introduced by 82a5649 in slot.c. When saving a replication slot,
failing to close the temporary path used to save the slot information is
considered as a failure and reported as such. However the code forgot to
leave immediately as other failure paths do. Noticed while looking up at this
area of the code for another patch.
https://git.postgresql.org/pg/commitdiff/9010156445c420361c2cef9557751c0b8b52644b

- Remove dependency to pageinspect in recovery tests. If contrib/pageinspect is
not installed, this causes the test checking the minimum recovery point to
fail. The point is that the dependency with pageinspect is not really
necessary as the test does also all checks with an offline cluster by scanning
directly the on-disk pages, which is enough for the purpose of the test. Per
complaint from Tom Lane. Author: Michael Paquier Discussion:
https://postgr.es/m/17806.1555566345@sss.pgh.pa.us
https://git.postgresql.org/pg/commitdiff/5a9323eab6e2c47e4caee9ab4b5117ba01368253

- Clean up some documentation for log_statement_sample_rate. This was missing
from 88bdbd3. Author: Christoph Berg, Justin Pryzby Discussion:
https://postgr.es/m/20190403215938.GA26375@alvherre.pgsql Discussion:
https://postgr.es/m/20190330224333.GQ5815@telsasoft.com
https://git.postgresql.org/pg/commitdiff/bc540f98599b875e7ee67086291cdb96997db3fc

- Fix collection of typos and grammar mistakes in docs and comments. Author:
Justin Pryzby Discussion:
https://postgr.es/m/20190330224333.GQ5815@telsasoft.com
https://git.postgresql.org/pg/commitdiff/148266fa354a47543f6c0325cd1ea900ead4aac6

- Fix detection of passwords hashed with MD5 or SCRAM-SHA-256. This commit fixes
a couple of issues related to the way password verifiers hashed with MD5 or
SCRAM-SHA-256 are detected, leading to being able to store in catalogs
passwords which do not follow the supported hash formats: - A MD5-hashed entry
was checked based on if its header uses "md5" and if the string length matches
what is expected. Unfortunately the code never checked if the hash only used
hexadecimal characters, as reported by Tom Lane. - A SCRAM-hashed entry was
checked based on only its header, which should be "SCRAM-SHA-256$", but it
never checked for any fields afterwards, as reported by Jonathan Katz.
Backpatch down to v10, which is where SCRAM has been introduced, and where
password verifiers in plain format have been removed. Author: Jonathan Katz
Reviewed-by: Tom Lane, Michael Paquier Discussion:
https://postgr.es/m/016deb6b-1f0a-8e9f-1833-a8675b170aa9@postgresql.org
Backpatch-through: 10
https://git.postgresql.org/pg/commitdiff/ccae190b916f27fbe4079ee4664d34cd1be47b79

- Mention REINDEX CONCURRENTLY in documentation about index maintenance. The
documentation includes a section about index maintenance and reindexing,
mentioning a set of steps based on CREATE INDEX CONCURRENTLY and ALTER TABLE
(for constraint dependencies) to emulate REINDEX CONCURRENTLY. Now that
REINDEX CONCURRENTLY is supported, let's just directly mention it instead.
Reported-by: Peter Geoghegan Author: Michael Paquier Reviewed-by: Peter
Eisentraut, Tom Lane Discussion:
https://postgr.es/m/CAH2-WzmEL168t6w29aKrKXtpq9-apcmp0HC7K-fKt6ZgLXV6Dg@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/a967878414f906a7f88842424c667a5ac31b9625

- Fix more typos and inconsistencies in documentation. This fixes a couple of
grammar mistakes, typos and inconsistencies in the documentation.
Particularly, the configuration parsing allows only "kB" to mean kilobyte but
there were references in the docs to "KB". Some instances of the latter are
still in the code comments. Some parameter values were mentioned with
"Minus-one", and using directly "-1" with proper markups is more helpful to
the reader. Some of these have been pointed out by Justin, and some others
are things I bumped into. Author: Justin Pryzby, Michael Paquier Reviewed-by:
Tom Lane Discussion: https://postgr.es/m/20190330224333.GQ5815@telsasoft.com
https://git.postgresql.org/pg/commitdiff/ac862376037727e744f25030bd8b6090c707247b

Peter Eisentraut pushed:

- Unbreak index optimization for LIKE on bytea. The same code is used to handle
both text and bytea, but bytea is not collation-aware, so we shouldn't call
get_collation_isdeterministic() in that case, since that will error out with
an invalid collation. Reported-by: Jeevan Chalke
<jeevan(dot)chalke(at)enterprisedb(dot)com> Discussion:
https://www.postgresql.org/message-id/flat/CAM2%2B6%3DWaf3qJ1%3DyVTUH8_yG-SC0xcBMY%2BSFLhvKKNnWNXSUDBw%40mail.gmail.com
https://git.postgresql.org/pg/commitdiff/abb9c63b2c00109183cdb9f588d2f23fc383ed38

- Better scaling of images in HTML output. Turn on the previously disabled
automatic scaling of images in HTML output. To avoid images looking too large
on nowadays-normal screens, restrict the width to 75% on such screens. Some
work is still necessary because SVG images without a viewBox still won't
scale, but that will a separate patch. Discussion:
https://www.postgresql.org/message-id/flat/6d2442d1-84a2-36ef-e014-b6d1ece8a139%40postgresql.org
https://git.postgresql.org/pg/commitdiff/e446cde16d676b528f3aa91dcbe5bb9c5a3776d8

- Fix handling of temp and unlogged tables in FOR ALL TABLES publications. If a
FOR ALL TABLES publication exists, temporary and unlogged tables are ignored
for publishing changes. But CheckCmdReplicaIdentity() would still check in
that case that such a table has a replica identity set before accepting
updates. To fix, have GetRelationPublicationActions() return that such a
table publishes no actions. Discussion:
https://www.postgresql.org/message-id/f3f151f7-c4dd-1646-b998-f60bd6217dd3@2ndquadrant.com
https://git.postgresql.org/pg/commitdiff/bb385c4fb0f4eb33d2bc7e484061565ba51cc790

- doc: Update section on NFS. The old section was ancient and didn't seem very
helpful. Here, we add some concrete advice on particular mount options.
Reviewed-by: Joe Conway <mail(at)joeconway(dot)com> Discussion:
https://www.postgresql.org/message-id/flat/e90f24bb-5423-6abb-58ec-501176eb4afc%402ndquadrant.com
https://git.postgresql.org/pg/commitdiff/60bbf0753e337114d4c7d60dbc5a496b1f464cb5

- Update key words table for version 12.
https://git.postgresql.org/pg/commitdiff/686fef22fdba732aeca28b471a362d4fb5675675

- Update config.guess and config.sub.
https://git.postgresql.org/pg/commitdiff/ddcaa596725a023c29fce34ff9914e123f225a5e

Etsuro Fujita pushed:

- Fix thinko in ExecCleanupTupleRouting(). Commit 3f2393edef changed
ExecCleanupTupleRouting() so that it skipped cleaning up subplan resultrels
before calling EndForeignInsert(), but that would cause an issue: when those
resultrels were foreign tables, the FDWs would fail to shut down. Repair by
skipping it after calling EndForeignInsert() as before. Author: Etsuro Fujita
Reviewed-by: David Rowley and Amit Langote Discussion:
https://postgr.es/m/5CAF3B8F.2090905@lab.ntt.co.jp
https://git.postgresql.org/pg/commitdiff/3a45321a491711b556d2cf8f6904ab989b9d0b08

- postgres_fdw: Fix incorrect handling of row movement for remote partitions.
Commit 3d956d9562 added support for update row movement in postgres_fdw. This
patch fixes the following issues introduced by that commit: * When a remote
partition chosen to insert routed rows into was also an UPDATE subplan
target rel that would be updated later, the UPDATE that used a direct
modification plan modified those routed rows incorrectly because those
routed rows were visible to the later UPDATE command. The right fix for this
would be to have some way in postgres_fdw in which the later UPDATE command
ignores those routed rows, but it seems hard to do so with the current
infrastructure. For now throw an error in that case. * When a remote
partition chosen to insert routed rows into was also an UPDATE subplan
target rel, fmstate created for the UPDATE that used a non-direct
modification plan was mistakenly overridden by another fmstate created for
inserting those routed rows into the partition. This caused 1) server crash
when the partition would be updated later, and 2) resource leak when the
partition had been already updated. To avoid that, adjust the treatment of
the fmstate for the inserting. As for #1, since we would also have the
incorrectness issue as mentioned above, error out in that case as well.
Update the docs to mention that postgres_fdw currently does not handle the
case where a remote partition chosen to insert a routed row into is also an
UPDATE subplan target rel that will be updated later. Author: Amit Langote
and Etsuro Fujita Reviewed-by: Amit Langote Backpatch-through: 11 where row
movement in postgres_fdw was added Discussion:
https://postgr.es/m/21e7eaa4-0d4d-20c2-a1f7-c7e96f4ce440@lab.ntt.co.jp
https://git.postgresql.org/pg/commitdiff/5c47049180b8977154f99fdc7988d5846764afa2

- Add FDW documentation notes about insert and update tuple routing and COPY.
Author: Laurenz Albe and Etsuro Fujita Reviewed-by: Laurenz Albe and Amit
Langote Backpatch-through: 11 where support for that by FDWs was added
Discussion:
https://postgr.es/m/bf36a0288e8f31b4f2f40952e225bf892dc1ffc5.camel@cybertec.at
https://git.postgresql.org/pg/commitdiff/90fca7a35aa7ac421f814bdfdf1fee7c30fa82f0

Alexander Korotkov pushed:

- Fix division by zero in _bt_vacuum_needs_cleanup(). Checks inside
_bt_vacuum_needs_cleanup() allow division by zero to happen when
metad->btm_last_cleanup_num_heap_tuples == 0. This commit adjusts the
expression so that no division by zero might happen. Reported-by: Piotr
Stefaniak Discussion:
https://postgr.es/m/DB8PR03MB5931C41F7787A95313F08322F22A0%40DB8PR03MB5931.eurprd03.prod.outlook.com
Reviewed-by: Masahiko Sawada Backpatch-through: 11
https://git.postgresql.org/pg/commitdiff/1e871981823a68ec9e0174b652e8d27e05ce08bb

Tom Lane pushed:

- Avoid possible regression test instability in timestamp.sql. Concurrent
autovacuum could result in a change in the order of the live rows in
timestamp_tbl. While this would not happen with the default autovacuum
parameters, it's fairly easy to hit if autovacuum_vacuum_threshold is made
small enough to allow autovac to decide to process this table. That's a
stumbling block for trying to exercise autovacuum aggressively using the core
regression tests. To fix, replace an unqualified DELETE with a TRUNCATE.
There's a similar DELETE just above (and no order-sensitive queries between),
so this doesn't lose any test coverage and might indeed be argued to improve
it. Discussion: https://postgr.es/m/17428.1555348950@sss.pgh.pa.us
https://git.postgresql.org/pg/commitdiff/47169c25500a70a7d04076860717f2f4f0ea83a7

- Fix failure with textual partition hash keys. Commit 5e1963fb7 overlooked two
places in partbounds.c that now need to pass a collation identifier to the
hash functions for a partition key column. Amit Langote, per report from
Jesper Pedersen Discussion:
https://postgr.es/m/a620f85a-42ab-e0f3-3337-b04b97e2e2f5@redhat.com
https://git.postgresql.org/pg/commitdiff/4b40e44f07c727c7a82b291d3b60098dd99f3f64

- Use [FLEXIBLE_ARRAY_MEMBER] not [1] in MultiSortSupportData. This struct seems
to have not gotten the word about preferred coding style for variable-length
arrays.
https://git.postgresql.org/pg/commitdiff/dde7fb7836c7428f79cb3cd88ca5febb802e767e

- Minor jsonpath fixes. Restore missed "make clean" rule, fix misspelling. John
Naylor Discussion:
https://postgr.es/m/CACPNZCt5B8jDCCGQiFoSuqmg-za_NCy4QDioBTLaNRih9+-bXg@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/b4f96d69ad197731c1f5b959e1234c9ba3517ecb

- Fix assorted minor bogosity in GSSAPI transport error messages. I noted that
some buildfarm members were complaining about %ld being used to format values
that are (probably) declared size_t. Use %zu instead, and insert a cast just
in case some versions of the GSSAPI API declare the length field differently.
While at it, clean up gratuitous differences in wording of equivalent
messages, show the complained-of length in all relevant messages not just
some, include trailing newline where needed, adjust random deviations from
project-standard code layout and message style, etc.
https://git.postgresql.org/pg/commitdiff/8cde7f49483d7e21569f61108fc80a8fe9e83e56

- Fix unportable code in pgbench. The buildfarm points out that UINT64_FORMAT
might not work with sscanf; it's calibrated for our printf implementation,
which might not agree with the platform-supplied sscanf. Fall back to just
accepting an unsigned long, which is already more than the documentation
promises. Oversight in e6c3ba7fb; back-patch to v11, as that was.
https://git.postgresql.org/pg/commitdiff/1a75c1d0c5d967ea2adcd7129092687cded4e7bf

- Fix problems with auto-held portals. HoldPinnedPortals() did things in the
wrong order: it must not mark a portal autoHeld until it's been successfully
held. Otherwise, a failure while persisting the portal results in a server
crash because we think the portal is in a good state when it's not. Also add
a check that portal->status is READY before attempting to hold a pinned
portal. We have such a check before the only other use of HoldPortal(), so it
seems unwise not to check it here. Lastly, rethink the responsibility for
where to call HoldPinnedPortals. The comment for it imagined that it was
optional for any individual PL to call it or not, but that cannot be the case:
if some outer level of procedure has a pinned portal, failing to persist it
when an inner procedure commits is going to be trouble. Let's have SPI do it
instead of the individual PLs. That's not a complete solution, since in
theory a PL might not be using SPI to perform commit/rollback, but such a PL
is going to have to be aware of lots of related requirements anyway. (This
change doesn't cause an API break for any external PLs that might be calling
HoldPinnedPortals per the old regime, because calling it twice during a commit
or rollback sequence won't hurt.) Per bug #15703 from Julian Schauder.
Back-patch to v11 where this code came in. Discussion:
https://postgr.es/m/15703-c12c5bc0ea34ba26@postgresql.org
https://git.postgresql.org/pg/commitdiff/4d5840cea96d7f893389664dd423716b38fded7a

- Don't request pretty-printed output from xmlNodeDump(). xml.c passed format =
1 to xmlNodeDump(), resulting in sometimes getting extra whitespace (newlines
+ spaces) in the output. We don't really want that, first because whitespace
might be semantically significant in some XML uses, and second because it
happens only very inconsistently. Only one case in our regression tests is
affected. This potentially affects the results of xpath() and the XMLTABLE
construct, when emitting nodeset values. Note that the older code in
contrib/xml2 doesn't do this; it seems to have been an aboriginal bad decision
in commit ea3b212fe. While this definitely seems like a bug to me, the small
number of complaints to date argues against back-patching a behavioral change.
Hence, fix in HEAD only, at least for now. Per report from Jean-Marc
Voillequin. Discussion:
https://postgr.es/m/1EC8157EB499BF459A516ADCF135ADCE3A23A9CA@LON-WGMSX712.ad.moodys.net
https://git.postgresql.org/pg/commitdiff/c06e3550dc4163c3ff29a87283b605f0beb50bed

- Avoid order-of-execution problems with ALTER TABLE ADD PRIMARY KEY. Up to now,
DefineIndex() was responsible for adding attnotnull constraints to the columns
of a primary key, in any case where it hadn't been convenient for
transformIndexConstraint() to mark those columns as is_not_null. It (or
rather its minion index_check_primary_key) did this by executing an ALTER
TABLE SET NOT NULL command for the target table. The trouble with this
solution is that if we're creating the index due to ALTER TABLE ADD PRIMARY
KEY, and the outer ALTER TABLE has additional sub-commands, the inner ALTER
TABLE's operations executed at the wrong time with respect to the outer ALTER
TABLE's operations. In particular, the inner ALTER would perform a validation
scan at a point where the table's storage might be inconsistent with its
catalog entries. (This is on the hairy edge of being a security problem, but
AFAICS it isn't one because the inner scan would only be interested in the
tuples' null bitmaps.) This can result in unexpected failures, such as the
one seen in bug #15580 from Allison Kaptur. To fix, let's remove the attempt
to do SET NOT NULL from DefineIndex(), reducing index_check_primary_key's role
to verifying that the columns are already not null. (It shouldn't ever see
such a case, but it seems wise to keep the check for safety.) Instead, make
transformIndexConstraint() generate ALTER TABLE SET NOT NULL subcommands to be
executed ahead of the ADD PRIMARY KEY operation in every case where it can't
force the column to be created already-not-null. This requires only minor
surgery in parse_utilcmd.c, and it makes for a much more satisfying spec for
transformIndexConstraint(): it's no longer having to take it on faith that
someone else will handle addition of NOT NULL constraints. To make that work,
we have to move the execution of AT_SetNotNull into an ALTER pass that
executes ahead of AT_PASS_ADD_INDEX. I moved it to AT_PASS_COL_ATTRS, and put
that after AT_PASS_ADD_COL to avoid failure when the column is being added in
the same command. This incidentally fixes a bug in the only previous usage of
AT_PASS_COL_ATTRS, for AT_SetIdentity: it didn't work either for a newly-added
column. Playing around with this exposed a separate bug in ALTER TABLE ONLY
... ADD PRIMARY KEY for partitioned tables. The intent of the ONLY modifier
in that context is to prevent doing anything that would require holding lock
for a long time --- but the implied SET NOT NULL would recurse to the child
partitions, and do an expensive validation scan for any child where the
column(s) were not already NOT NULL. To fix that, invent a new ALTER
subcommand AT_CheckNotNull that just insists that a child column be already
NOT NULL, and apply that, not AT_SetNotNull, when recursing to children in
this scenario. This results in a slightly laxer definition of ALTER TABLE
ONLY ... SET NOT NULL for partitioned tables, too: that command will now work
as long as all children are already NOT NULL, whereas before it just threw up
its hands if there were any partitions. In passing, clean up the API of
generateClonedIndexStmt(): remove a useless argument, ensure that the output
argument is not left undefined, update the header comment. A small side
effect of this change is that no-such-column errors in ALTER TABLE ADD PRIMARY
KEY now produce a different message that includes the table name, because they
are now detected by the SET NOT NULL step which has historically worded its
error that way. That seems fine to me, so I didn't make any effort to avoid
the wording change. The basic bug #15580 is of very long standing, and these
other bugs aren't new in v12 either. However, this is a pretty significant
change in the way ALTER TABLE ADD PRIMARY KEY works. On balance it seems best
not to back-patch, at least not till we get some more confidence that this
patch has no new bugs. Patch by me, but thanks to Jie Zhang for a preliminary
version. Discussion:
https://postgr.es/m/15580-d1a6de5a3d65da51@postgresql.org Discussion:
https://postgr.es/m/1396E95157071C4EBBA51892C5368521017F2E6E63@G08CNEXMBPEKD02.g08.fujitsu.local
https://git.postgresql.org/pg/commitdiff/f4a3fdfbdcd3763c42111318d004c2e90d072021

- Remove useless comment. Commit e439c6f0c removed IndexStmt.relationId, but not
the comment that had been added to explain it. Said comment was therefore
very confusing.
https://git.postgresql.org/pg/commitdiff/e0fb4c9d013f8c3f6eb31c5fb991e24b4af6dbb1

- Repair assorted issues in locale data extraction. cache_locale_time
(extraction of LC_TIME-related info) had never been taught the lessons we
previously learned about extraction of info related to LC_MONETARY and
LC_NUMERIC. Specifically, commit 95a777c61 taught PGLC_localeconv() that data
coming out of localeconv() was in an encoding determined by the relevant
locale, but we didn't realize that there's a similar issue with strftime().
And commit a4930e7ca hardened PGLC_localeconv() against errors occurring
partway through, but failed to do likewise for cache_locale_time(). So,
rearrange the latter function to perform encoding conversion and not risk
failure while it's got the locales set to temporary values. This time around
I also changed PGLC_localeconv() to treat it as FATAL if it can't restore the
previous settings of the locale values. There is no reason (except possibly
OOM) for that to fail, and proceeding with the wrong locale values seems like
a seriously bad idea --- especially on Windows where we have to also
temporarily change LC_CTYPE. Also, protect against the possibility that we
can't identify the codeset reported for LC_MONETARY or LC_NUMERIC; rather than
just failing, try to validate the data without conversion. The user-visible
symptom this fixes is that if LC_TIME is set to a locale name that implies an
encoding different from the database encoding, non-ASCII localized day and
month names would be retrieved in the wrong encoding, leading to either
unexpected encoding-conversion error reports or wrong output from to_char().
The other possible failure modes are unlikely enough that we've not seen
reports of them, AFAIK. The encoding conversion problems do not manifest on
Windows, since we'd already created special-case code to handle that issue
there. Per report from Juan José Santamaría Flecha. Back-patch to all
supported versions. Juan José Santamaría Flecha and Tom Lane Discussion:
https://postgr.es/m/CAC+AXB22So5aZm2vZe+MChYXec7gWfr-n-SK-iO091R0P_1Tew@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/7ad1cd31bfcbaed122993deb778e6587a226e338

- Fix some minor postmaster-state-machine issues. In sigusr1_handler, don't
ignore PMSIGNAL_ADVANCE_STATE_MACHINE based on pmState. The restriction is
unnecessary (PostmasterStateMachine should work in any state), not
future-proof (since it makes too many assumptions about why the signal might
be sent), and broken even today because a race condition can make it necessary
to respond to the signal in PM_WAIT_READONLY state. The race condition seems
unlikely, but if it did happen, a hot-standby postmaster could fail to shut
down after receiving a smart-shutdown request. In MaybeStartWalReceiver,
don't clear the WalReceiverRequested flag if the fork attempt fails. Leaving
it set allows us to try again in future iterations of the postmaster idle
loop. (The startup process would eventually send a fresh request signal, but
this change may allow us to retry the fork sooner.) Remove an obsolete
comment and unnecessary test in PostmasterStateMachine's handling of
PM_SHUTDOWN_2 state. It's not possible to have a live walreceiver in that
state, and AFAICT has not been possible since commit 5e85315ea. This isn't a
live bug, but the false comment is quite confusing to readers. In passing,
rearrange sigusr1_handler's CheckPromoteSignal tests so that we don't
uselessly perform stat() calls that we're going to ignore the results of. Add
some comments clarifying the behavior of MaybeStartWalReceiver; I very nearly
rearranged it in a way that'd reintroduce the race condition fixed in
e5d494d78. Mea culpa for not commenting that properly at the time.
Back-patch to all supported branches. The PMSIGNAL_ADVANCE_STATE_MACHINE
change is the only one of even minor significance, but we might as well keep
this code in sync across branches. Discussion:
https://postgr.es/m/9001.1556046681@sss.pgh.pa.us
https://git.postgresql.org/pg/commitdiff/0fae846232148754468f6d57b3f98d5b83d90af7

- Apply stopgap fix for bug #15672. Fix DefineIndex so that it doesn't attempt
to pass down a to-be-reused index relfilenode to a child index creation, and
fix TryReuseIndex to not think that reuse is sensible for a partitioned index.
In v11, this fixes a problem where ALTER TABLE on a partitioned table could
assign the same relfilenode to several different child indexes, causing very
nasty catalog corruption --- in fact, attempting to DROP the partitioned table
then leads not only to a database crash, but to inability to restart because
the same crash will recur during WAL replay. Either of these two changes
would be enough to prevent the failure, but since neither action could
possibly be sane, let's put in both changes for future-proofing. In HEAD, no
such bug manifests, but that's just an accidental consequence of having
changed the pg_class representation of partitioned indexes to have relfilenode
= 0. Both of these changes still seem like smart future-proofing. This is
only a stop-gap because the code for ALTER TABLE on a partitioned table with a
no-op type change still leaves a great deal to be desired. As the added
regression tests show, it gets things wrong for comments on child
indexes/constraints, and it is regenerating child indexes it doesn't have to.
However, fixing those problems will take more work which may not get
back-patched into v11. We need a fix for the corruption problem now. Per bug
#15672 from Jianing Yang. Patch by me, regression test cases based on work by
Amit Langote, who also did a lot of the investigative work. Discussion:
https://postgr.es/m/15672-b9fa7db32698269f@postgresql.org
https://git.postgresql.org/pg/commitdiff/c01eb619a83aed460b9ead50831265f2acde3c57

- Update time zone data files to tzdata release 2019a. DST law changes in
Palestine and Metlakatla. Historical corrections for Israel. Etc/UCT is now a
backward-compatibility link to Etc/UTC, instead of being a separate zone that
generates the abbreviation "UCT", which nowadays is typically a typo.
Postgres will still accept "UCT" as an input zone name, but it won't output
it.
https://git.postgresql.org/pg/commitdiff/d312de3fc092c784e743e58be4d1617cfb704b71

- Sync our copy of the timezone library with IANA release tzcode2019a. This
corrects a small bug in zic that caused it to output an incorrect year-2440
transition in the Africa/Casablanca zone. More interestingly, zic has grown a
"-r" option that limits the range of zone transitions that it will put into
the output files. That might be useful to people who don't like the weird GMT
offsets that tzdb likes to use for very old dates. It appears that for dates
before the cutoff time specified with -r, zic will use the zone's
standard-time offset as of the cutoff time. So for example one might do
make install ZIC_OPTIONS='-r @-1893456000' to cause all dates before
1910-01-01 to be treated as though 1910 standard time prevailed indefinitely
far back. (Don't blame me for the unfriendly way of specifying the cutoff
time --- it's seconds since or before the Unix epoch. You can use
extract(epoch ...) to calculate it.) As usual, back-patch to all supported
branches.
https://git.postgresql.org/pg/commitdiff/acb897b806a5f4b23784b1c7e395e67b99a4362b

- Portability fix for zic.c. Missed an inttypes.h dependency in previous patch.
Per buildfarm.
https://git.postgresql.org/pg/commitdiff/48f8fa9ce0f9799c8ae8bcd23c498fe002d5496d

- Avoid postgres_fdw crash for a targetlist entry that's just a Param.
foreign_grouping_ok() is willing to put fairly arbitrary expressions into the
targetlist of a remote SELECT that's doing grouping or aggregation on the
remote side, including expressions that have no foreign component to them at
all. This is possibly a bit dubious from an efficiency standpoint; but it
rises to the level of a crash-causing bug if the expression is just a Param or
non-foreign Var. In that case, the expression will necessarily also appear in
the fdw_exprs list of values we need to send to the remote server, and then
setrefs.c's set_foreignscan_references will mistakenly replace the fdw_exprs
entry with a Var referencing the targetlist result. The root cause of this
problem is bad design in commit e7cb7ee14: it put logic into
set_foreignscan_references that IMV is postgres_fdw-specific, and yet this bug
shows that it isn't postgres_fdw-specific enough. The transformation being
done on fdw_exprs assumes that fdw_exprs is to be evaluated with the
fdw_scan_tlist as input, which is not how postgres_fdw uses it; yet it could
be the right thing for some other FDW. (In the bigger picture, setrefs.c has
no business assuming this for the other expression fields of a ForeignScan
either.) The right fix therefore would be to expand the FDW API so that the
FDW could inform setrefs.c how it intends to evaluate these various
expressions. We can't change that in the back branches though, and we also
can't just summarily change setrefs.c's behavior there, or we're likely to
break external FDWs. As a stopgap, therefore, hack up postgres_fdw so that it
won't attempt to send targetlist entries that look exactly like the fdw_exprs
entries they'd produce. In most cases this actually produces a superior plan,
IMO, with less data needing to be transmitted and returned; so we probably
ought to think harder about whether we should ship tlist expressions at all
when they don't contain any foreign Vars or Aggs. But that's an optimization
not a bug fix so I left it for later. One case where this produces an
inferior plan is where the expression in question is actually a GROUP BY
expression: then the restriction prevents us from using remote grouping. It
might be possible to work around that (since that would reduce to
group-by-a-constant on the remote side); but it seems like a pretty unlikely
corner case, so I'm not sure it's worth expending effort solely to improve
that. In any case the right long-term answer is to fix the API as sketched
above, and then revert this hack. Per bug #15781 from Sean Johnston.
Back-patch to v10 where the problem was introduced. Discussion:
https://postgr.es/m/15781-2601b1002bad087c@postgresql.org
https://git.postgresql.org/pg/commitdiff/8cad5adb9c0be82e9f40d51b02a542439f47de9e

Tomáš Vondra pushed:

- Fix pg_mcv_list deserialization. The memcpy() was copying type OIDs in the
wrong direction, so the deserialized MCV list always had them as 0. This is
mostly harmless except when printing the data in pg_mcv_list_items(), in which
case it reported ERROR: cache lookup failed for type 0 Also added a
simple regression test for pg_mcv_list_items() function, printing a
single-item MCV list. Reported-By: Dean Rasheed Discussion:
https://postgr.es/m/CAEZATCX6T0iDTTZrqyec4Cd6b4yuL7euu4=rQRXaVBAVrUi1Cg@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/3824ca30d162611fdaade403d3aec449fecd0924

- Convert pre-existing stats_ext tests to new style. The regression tests added
in commit 7300a69950 test cardinality estimates using a function that extracts
the interesting pieces from the EXPLAIN output, instead of testing the whole
plan. That seems both easier to understand and less fragile, so this applies
the same approach to pre-existing tests of ndistinct coefficients and
functional dependencies. Discussion:
https://postgr.es/m/dfdac334-9cf2-2597-fb27-f0fb3753f435@2ndquadrant.com
https://git.postgresql.org/pg/commitdiff/dbb984128ebf6b4bd6f12b4579ee9103ef1ca749

- Fix mvdistinct and dependencies size calculations. The formulas used to
calculate size while (de)serializing mvndistinct and functional dependencies
were based on offset() of the structs. But that is incorrect, because the
structures are not copied directly, we we copy the individual fields directly.
At the moment this works fine, because there is no alignment padding on any
platform we support. But it might break if we ever added some fields into any
of the structs, for example. It's also confusing. Fixed by reworking the
macros to directly sum sizes of serialized fields. The macros are now useful
only for serialiation, so there is no point in keeping them in the public
header file. So make them private by moving them to the .c files. Also adds a
couple more asserts to check the serialization, and fixes an incorrect
allocation of MVDependency instead of (MVDependency *). Reported-By: Tom Lane
Discussion: https://postgr.es/m/29785.1555365602@sss.pgh.pa.us
https://git.postgresql.org/pg/commitdiff/d08c44f7a4ec574110bf1587fe8bd682b8557e0f

Fujii Masao pushed:

- Add index terms for reloptions in documentation. Author: Fujii Masao
Reviewed-By: Alvaro Herrera, Michael Paquier Discussion:
https://postgr.es/m/CAHGQGwHyKt9-xkibVguPzYqKgb_2tdw14Ub1XDTu08kyHMiTQA@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/c8e0f6bbdb5ff0f235246af5b567b21dcfc1d22a

- Fix documentation of pg_start_backup and pg_stop_backup functions. This commit
adds the description that "non-exclusive" pg_start_backup and pg_stop_backup
can be executed even during recovery. Previously it was wrongly documented
that those functions are not allowed to be executed during recovery.
Back-patch to 9.6 where non-exclusive backup API was added. Discussion:
https://postgr.es/m/CAHGQGwEuAYrEX7Yhmf2MCrTK81HDkkg-JqsOUh8zw6+zYC5zzw@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/b804c1837b55722369d30be707738c098a75cd7a

- Fix function names in comments. Commit 3eb77eba5a renamed some functions, but
forgot to update some comments referencing to those functions. This commit
fixes those function names in the comments. Kyotaro Horiguchi
https://git.postgresql.org/pg/commitdiff/978b032d1f39fcfe9b7e895ab313913d97db2268

- Fix file path in comment.
https://git.postgresql.org/pg/commitdiff/c247ae09226eeb1c3a77bb54a0736ed7da5622b5

Magnus Hagander pushed:

- Return NULL for checksum failures if checksums are not enabled. Returning 0
could falsely indicate that there is no problem. NULL correctly indicates that
there is no information about potential problems. Also return 0 as
numbackends instead of NULL for shared objects (as no connection can be made
to a shared object only). Author: Julien Rouhaud <rjuju123(at)gmail(dot)com>
Reviewed-by: Robert Treat <rob(at)xzilla(dot)net>
https://git.postgresql.org/pg/commitdiff/252b707bc41cc9bf6c55c18d8cb302a6176b7e48

Bruce Momjian pushed:

- docs: clarify pg_upgrade's recovery behavior. The previous paragraph trying to
explain --check, --link, and no --link modes and the various points of failure
was too complex. Instead, use bullet lists and sublists. Reported-by: Daniel
Gustafsson Discussion:
https://postgr.es/m/qtqiv7hI87s_Xvz5ZXHCaH-1-_AZGpIDJowzlRjF3-AbCr3RhSNydM_JCuJ8DE4WZozrtxhIWmyYTbv0syKyfGB6cYMQitp9yN-NZMm-oAo=@yesql.se
Backpatch-through: 9.4
https://git.postgresql.org/pg/commitdiff/64bbe1039962a75cb292fb557e2e79a58c77ec33

- postgresql.conf.sample: add proper defaults for include actions. Previously,
include actions include_dir, include_if_exists, and include listed
commented-out values which were not the defaults, which is inconsistent with
other entries. Instead, replace them with '', which is the default value.
Reported-by: Emanuel Araújo Discussion:
https://postgr.es/m/CAMuTAkYMx6Q27wpELDR3_v9aG443y7ZjeXu15_+1nGUjhMWOJA@mail.gmail.com
Backpatch-through: 9.4
https://git.postgresql.org/pg/commitdiff/fb9c475597c245562a28d1e916b575ac4ec5c19f

- docs: reorder collation regression test order in paragraph.
Backpatch-through: 10
https://git.postgresql.org/pg/commitdiff/bfbc150e39515c08a5691c7da4291d1b9f2b2b9c

Álvaro Herrera pushed:

- psql: display tablespace for partitioned indexes. Nothing was shown
previously.
https://git.postgresql.org/pg/commitdiff/b036982db786f7ef6c118419e7e832167ad4a6a7

- Tie loose ends in psql's new \dP command. * Remove one unnecessary pg_class
join in SQL command. Not needed, because we use a regclass cast instead. *
Doc: refer to "partitioned relations" rather than specifically tables, since
indexes are also displayed. * Rename "On table" column to "Table", for
consistency with \di. Author: Justin Pryzby Discussion:
https://postgr.es/m/20190407212525.GB10080@telsasoft.com
https://git.postgresql.org/pg/commitdiff/421a2c48328c88eb31f6b29979218f0a350bb739

- Unify error messages. ... for translatability purposes.
https://git.postgresql.org/pg/commitdiff/0a999e1290fc9d8708da61017e732380bc9239a3

- Make pg_dump emit ATTACH PARTITION instead of PARTITION OF. Using PARTITION OF
can result in column ordering being changed from the database being dumped, if
the partition uses a column layout different from the parent's. It's not
pg_dump's job to editorialize on table definitions, so this is not acceptable;
back-patch all the way back to pg10, where partitioned tables where
introduced. This change also ensures that partitions end up in the correct
tablespace, if different from the parent's; this is an oversight in
ca4103025dfe (in pg12 only). Partitioned indexes (in pg11) don't have this
problem, because they're already created as independent indexes and attached
to their parents afterwards. This change also has the advantage that the
partition is restorable from the dump (as a standalone table) even if its
parent table isn't restored. Author: David Rowley Reviewed-by: Álvaro Herrera
Discussion:
https://postgr.es/m/CAKJS1f_1c260nOt_vBJ067AZ3JXptXVRohDVMLEBmudX1YEx-A@mail.gmail.com
Discussion: https://postgr.es/m/20190423185007.GA27954@alvherre.pgsql
https://git.postgresql.org/pg/commitdiff/3b23552ad8bbb1384381b67f860019d14d5b680e

- Fix tablespace inheritance for partitioned rels. Commit ca4103025dfe left a
few loose ends. The most important one (broken pg_dump output) is already
fixed by virtue of commit 3b23552ad8bb, but some things remained: * When
ALTER TABLE rewrites tables, the indexes must remain in the tablespace they
were originally in. This didn't work because index recreation during ALTER
TABLE runs manufactured SQL (yuck), which runs afoul of default_tablespace
in competition with the parent relation tablespace. To fix, reset
default_tablespace to the empty string temporarily, and add the TABLESPACE
clause as appropriate. * Setting a partitioned rel's tablespace to the
database default is confusing; if it worked, it would direct the partitions
to that tablespace regardless of default_tablespace. But in reality it does
not work, and making it work is a larger project. Therefore, throw an error
when this condition is detected, to alert the unwary. Add some docs and
tests, too. Author: Álvaro Herrera Discussion:
https://postgr.es/m/CAKJS1f_1c260nOt_vBJ067AZ3JXptXVRohDVMLEBmudX1YEx-A@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/87259588d0ab0b8e742e30596afa7ae25caadb18

- Fix partitioned index attachment. When an existing index in a partition is
attached to a new index on its parent, we forgot to set the "relispartition"
flag correctly, which meant that it was not possible to find the index in
various operations, such as adding a foreign key constraint that references
that partitioned table. One of four places that was assigning the parent
index was forgetting to do that, so fix by shifting responsibility of updating
the flag to the routine that changes the parent. Author: Amit Langote, Álvaro
Herrera Reported-by: Hubert "depesz" Lubaczewski Discussion:
https://postgr.es/m/CA+HiwqHMsRtRYRWYTWavKJ8x14AFsv7bmAV46mYwnfD3vy8goQ@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/05b38c7e63d1c8bca8c07ab4c8b194eed3c50ec7

- pg_dump: store unused attribs as NULL instead of '\0'. Commit f831d4accda0
changed pg_dump to emit (and pg_restore to understand) NULLs for unused
members in ArchiveEntry structs, as a side effect of some code beautification.
That broke pg_restore of dumps generated with older pg_dump, however, so it
was reverted in 19455c9f5606. Since the archiver version number has been
bumped in 3b925e905de3, we can put it back. Author: Dmitry Dolgov Discussion:
https://postgr.es/m/CA+q6zcXx0XHqLsFJLaUU2j5BDiBAHig=YRoBC_YVq7VJGvzBEA@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/7fcdb5e002145ec95b2bebb4c125751be66c4d1a

Andres Freund pushed:

- docs: cleanup/remove/update references to OID column. I (Andres) missed these
in 578b229718e8f. Author: Justin Pryzby, editorialized a bit by Andres Freund
Reviewed-By: Daniel Verite, Andres Freund Discussion:
https://postgr.es/m/20190408002847.GA904@telsasoft.com
https://git.postgresql.org/pg/commitdiff/f6b39171f3d65155b9390c2c69bc5b3469f923a8

- pg_dump: Remove stray option parsing support for -o. I (Andres) missed this in
578b229718e8f, the removal of WITH OIDS support. Author: Daniel Verite
Discussion:
https://postgr.es/m/f06e9735-3717-4904-8c95-47d0b9c3bb10@manitou-mail.org
https://git.postgresql.org/pg/commitdiff/4d0183592764629d38182e30642b1aef8baeb987

- docs: correct typo-ed path to heapam_handler.c. Reported-By: Michael Paquier,
Michel Pelletier Discussion:
https://postgr.es/m/20190410025531.GA2728@paquier.xyz
https://postgr.es/m/CACxu=v+u_QTeFqdajCHv3i4QmzV_63arVb57R19dSKtThdSLkQ@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/1cebfdee0087624af2fc99a000642c2db4dfbfc9

- Fix potential use-after-free for BEFORE UPDATE row triggers on non-core AMs.
When such a trigger returns the old row version, it naturally get stored in
the slot for the trigger result. When a table AMs doesn't store HeapTuples
internally, ExecBRUpdateTriggers() frees the old row version passed to
triggers - but before this fix it might still be referenced by the slot
holding the new tuple. Noticed when running the out-of-core zheap AM against
the in-core version of tableam. Author: Andres Freund
https://git.postgresql.org/pg/commitdiff/75e03eabeaac8fd229b14f74de55d0e1470903c4

- Fix two memory leaks around force-storing tuples in slots. As reported by Tom,
when ExecStoreMinimalTuple() had to perform a conversion to store the minimal
tuple in the slot, it forgot to respect the shouldFree flag, and leaked the
tuple into the current memory context if true. Fix that by freeing the tuple
in that case. Looking at the relevant code made me (Andres) realize that not
having the shouldFree parameter to ExecForceStoreHeapTuple() was a bad idea.
Some callers had to locally implement the necessary logic, and in one case it
was missing, creating a potential per-group leak in non-hashed aggregation.
The choice to not free the tuple in ExecComputeStoredGenerated() is not
pretty, but not introduced by this commit - I'll start a separate discussion
about it. Reported-By: Tom Lane Discussion:
https://postgr.es/m/366.1555382816@sss.pgh.pa.us
https://git.postgresql.org/pg/commitdiff/88e6ad3054ddd5aa0dee12e5def2c335fe92a414

- Fix slot type issue for fuzzy distance index scan over out-of-core table AM.
For amcanreorderby scans the nodeIndexscan.c's reorder queue holds heap
tuples, but the underlying table likely does not. Before this fix we'd return
different types of slots, depending on whether the tuple came from the reorder
queue, or from the index + table. While that could be fixed by signalling
that the node doesn't return a fixed type of slot, it seems better to instead
remove the separate slot for the reorder queue, and use
ExecForceStoreHeapTuple() to store tuples from the queue. It's not
particularly common to need reordering, after all. This reverts most of the
iss_ReorderQueueSlot related changes to nodeIndexscan.c made in
1a0586de3657cd3, except that now ExecForceStoreHeapTuple() is used instead of
ExecStoreHeapTuple(). Noticed when testing zheap against the in-core version
of tableam. Author: Andres Freund
https://git.postgresql.org/pg/commitdiff/b8b94ea129ffc988c2d30eb2b5aa65a93329b8fa

- Convert gist to compute page level xid horizon on primary. Due to parallel
development, gist added the missing conflict information in c952eae52a3, while
558a9165e08 moved that computation to the primary for the index types that
already had it. Thus adapt gist to also compute on the primary, using
index_compute_xid_horizon_for_tuples() instead of its own copy of the logic.
This also adds pg_waldump support for XLOG_GIST_DELETE records, which
previously was not properly present. Bumps WAL version. Author: Andres
Freund Discussion:
https://postgr.es/m/20190406050243.bszosdg4buvabfrt@alap3.anarazel.de
https://git.postgresql.org/pg/commitdiff/b5f58cf213bdab7e726f7b09c1137b4ea397fc0b

- Allow pg_class xid & multixid horizons to not be set. This allows table AMs
that don't need these horizons. This was already documented in the tableam
relation_set_new_filenode callback, but an assert prevented if from actually
working (the test AM code contained the change itself). Defang the asserts in
the general code, and move the stronger ones into heap AM. Relatedly, after
CLUSTER/VACUUM, we'd always assign a relfrozenxid / relminmxid. Change the
table_relation_copy_for_cluster() interface to allow the AM to overwrite the
horizons that get set on the pg_class entry. This'd also in the future allow
AMs like heap to compute a relfrozenxid during rewrite that's the table's
actual minimum rather than a pre-determined value. Arguably it'd have been
better to move the whole computation / setting of those values into the
callback, but it seems likely that for other reasons it'd be better to be able
to use one value to vacuum/cluster multiple tables (e.g. a toast's horizon
shouldn't be different than the table's). Reported-By: Heikki Linnakangas
Author: Andres Freund Discussion:
https://postgr.es/m/9a7fb9cc-2419-5db7-8840-ddc10c93f122@iki.fi
https://git.postgresql.org/pg/commitdiff/fdc7efcc30b1da950ade72fd35852c21bf8de839

Stephen Frost pushed:

- GSSAPI: Improve documentation and tests. The GSSAPI encryption patch neglected
to update the protocol documentation to describe how to set up a GSSAPI
encrypted connection from a client to the server, so fix that by adding the
appropriate documentation to protocol.sgml. The tests added for encryption
support were overly long and couldn't be run in parallel due to race
conditions; this was largely because each test was setting up its own KDC to
perform the tests. Instead, merge the authentication tests and the encryption
tests into the original test, where we only create one KDC to run the tests
with. Also, have the tests check what the server's opinion is of the
connection and if it was GSS authenticated or encrypted using the
pg_stat_gssapi view. In passing, fix the libpq label for GSSENC-Mode to be
consistent with the "PGGSSENCMODE" environment variable. Missing protocol
documentation pointed out by Michael Paquier. Issues with the tests pointed
out by Tom Lane and Peter Eisentraut. Refactored tests and added
documentation by me. Reviewed by Robbie Harwood (protocol documentation) and
Michael Paquier (rework of the tests).
https://git.postgresql.org/pg/commitdiff/eb882a1b717589777e068dc6671830226f3aa7f0

Peter Geoghegan pushed:

- Prevent O(N^2) unique index insertion edge case. Commit dd299df8 made nbtree
treat heap TID as a tiebreaker column, establishing the principle that there
is only one correct location (page and page offset number) for every index
tuple, no matter what. Insertions of tuples into non-unique indexes proceed as
if heap TID (scan key's scantid) is just another user-attribute value, but
insertions into unique indexes are more delicate. The TID value in scantid
must initially be omitted to ensure that the unique index insertion visits
every leaf page that duplicates could be on. The scantid is set once again
after unique checking finishes successfully, which can force
_bt_findinsertloc() to step right one or more times, to locate the leaf page
that the new tuple must be inserted on. Stepping right within
_bt_findinsertloc() was assumed to occur no more frequently than stepping
right within _bt_check_unique(), but there was one important case where that
assumption was incorrect: inserting a "duplicate" with NULL values. Since
_bt_check_unique() didn't do any real work in this case, it wasn't appropriate
for _bt_findinsertloc() to behave as if it was finishing off a conventional
unique insertion, where any existing physical duplicate must be dead or
recently dead. _bt_findinsertloc() might have to grovel through a substantial
portion of all of the leaf pages in the index to insert a single tuple, even
when there were no dead tuples. To fix, treat insertions of tuples with NULLs
into a unique index as if they were insertions into a non-unique index: never
unset scantid before calling _bt_search() to descend the tree, and bypass
_bt_check_unique() entirely. _bt_check_unique() is no longer responsible for
incoming tuples with NULL values. Discussion:
https://postgr.es/m/CAH2-Wzm08nr+JPx4jMOa9CGqxWYDQ-_D4wtPBiKghXAUiUy-nQ@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/9b10926263d831fac5758f1493c929a49b55669b

- Sanitize line pointers within contrib/amcheck. Adopt a more defensive approach
to accessing index tuples in contrib/amcheck: verify that each line pointer
looks sane before accessing associated tuple using pointer arithmetic based on
line pointer's offset. This avoids undefined behavior and assertion failures
in cases where line pointers are corrupt. Issue spotted following a complaint
about an assertion failure by Grigory Smolkin, which involved a test harness
that deliberately corrupts indexes. This is arguably a bugfix, but no
backpatch given the lack of field reports beyond Grigory's. Discussion:
https://postgr.es/m/CAH2-WzmkurhCqnyLHxk0VkOZqd49+ZZsp1xAJOg7j2x7dmp_XQ@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/a9ce839a31375a6c04189222706ae6b2abe96919

Joe Conway pushed:

- Correct the URL pointing to PL/R. As pointed out by documentation comment, the
URL for PL/R needs to be updated to the correct current repository. Back-patch
to all supported branches.
https://git.postgresql.org/pg/commitdiff/cf3ff97a15c184bf6a665b891f41349c8aba79fb

- Add guidance on making documentation SVG images responsive. Recently added
guidance on adding SVG images to the documentation sources lacks advice on
making the images responsive when rendered in a variety of media types and
viewports. Add some. Patch by Jonathan Katz with some editorialization by me.
Author: Jonathan Katz Discussion:
https://postgr.es/m/6358ae6f-7191-a02b-e7b5-68050636ae71@postgresql.org
https://git.postgresql.org/pg/commitdiff/7dc78d8ef3e62f7e06d7767c63dcede048377b9a

- Add viewBox attribute to storage page layout SVG image. Recently added SVG
image for storage page layout lacks a viewBox attribute which seems necessary
to ensure propoer rendering. Add it. Author: Jonathan Katz Discussion:
https://postgr.es/m/ba31e0e1-4c9b-b309-70e8-8e7ac14fc87e%40postgresql.org
https://git.postgresql.org/pg/commitdiff/29046c44f36099f4c979b1491fcf27db2f9184f9

== Pending Patches ==

Justin Pryzby sent in a patch for psql to make \d pg_toast.foo show its indices.

Alexander Korotkov sent in a PoC patch to make relation truncation use
ExclusiveLock instead of AccessExclusiveLock lock.

David Rowley sent in a WIP patch to improve runtime pruning explain output.

Amit Kapila sent in two revisions of a patch to move the FSM map to relcache.

Kyotaro HORIGUCHI sent in four more revisions of a patch to fix failure of
standby startup caused by tablespace removal.

David Fetter and Fabien COELHO traded patches to add \warn (send output to
standard error) to psql.

Konstantin Evteev sent in a patch to document log-shipping standby server major
version upgrades.

Daniel Vérité sent in another revision of a patch to fix an infelicity between
FETCH_COUNT and combined queries in psql by ignoring FETCH_COUNT and not using a
cursor when combined queries are detected.

Masahiko Sawada sent in another revision of a patch to change lazy_scan_heap so
that it counts the tuples that became dead after HOT pruning in nkeep when index
cleanup is disabled, and change lazy vacuum so that it reports both the number
of kilobytes we freed and the number of kilobytes can be freed after index
cleanup.

Andres Freund and Michaël Paquier traded patches to fix a bug which manifested
as REINDEX INDEX producing a crash for an index of pg_class.

Tom Lane and Michaël Paquier traded patches to fix a problem where
PrepareTempTablespaces was being called in BufFileCreateTemp.

David Fetter and Fabien COELHO traded patches against psql to make it show
detailed table persistence in \dt+.

Antonin Houska sent in another revision of a patch to implement encryption for
data at rest.

Paul Guo sent in a patch to ensure that a file is a regular file (and no, for
example, an pipe) if it exists before calling AllocateFile() for file writing.

Alexander Korotkov sent in another revision of a patch to improve error handling
in JSONPATH.

Amul Sul sent in another revision of a patch to use a better algorithm for
partition-wise JOINs.

Kyotaro HORIGUCHI sent in four more revisions of a patch to fix a failure of
standby startup caused by tablespace removal.

Tom Lane and Amit Langote traded patches to fix an issue where dropping a
partitioned table caused a crash.

Peter Eisentraut and Laurenz Albe traded patches to fix an issue where an
identity column could own more than one sequence.

Kyotaro HORIGUCHI sent in a patch to fix a comment about renamed functions.

Alexey Klyukin sent in a patch to add per-tablespace autovacuum parameters.

Dilip Kumar sent in another revision of a patch to clean up orphaned files using
undo logs.

Fujii Masao sent in a patch to fix an infelicity between pg_waldump and PREPARE
TRANSACTION.

John Naylor sent in a patch to improve the usage of smgrexists by checking
smgr_fsm_nblocks.

Andres Freund sent in a patch to add heapam_relation_size().

Ryo Matsumura sent in another revision of a patch to clarify which roles can
access pg_logical_emit_message().

Iwata Aya sent in another revision of a patch to create a one-line format for
libpq trace logs.

John Naylor sent in a patch to untie free space code from any knowledge about
heaps, and make it the responsibility of each access method that calls these
free space routines to specify its own threshold (possibly zero).

Kyotaro HORIGUCHI sent in another revision of a patch to remove page-read
callback from XLogReaderState.

Masahiko Sawada sent in another revision of a patch to fix vacuum reporting.

Takeshi Ideriha sent in another revision of a patch to create an EXTENSION that
can deal with the DSA area.

Amit Langote sent in a patch to fix an infelicity between
findTargetlistEntrySQL92() and COLLATE.

Fabien COELHO sent in another revision of a patch to psql to add a
SHOW_ALL_RESULTS setting for combined queries.

Tom Lane sent in a patch to fix BUG #15781: subselect on foreign table
(postgres_fdw) can crash (segfault).

Peter Eisentraut sent in a patch to generate the documentation keywords table
automatically.

Tom Lane sent in another revision of a patch to fix some race conditions with
checkpointer and shutdown.

David Fetter sent in a patch from Fabien COELHO to add a way to supply stdin to
TAP tests.