== PostgreSQL Weekly News - May 12, 2019 ==

== PostgreSQL Weekly News - May 12, 2019 ==

PostgreSQL security releases 11.3, 10.8, 9.6.13, 9.5.17, and 9.4.22 released.
Upgrade as soon as possible.
https://www.postgresql.org/about/news/1939/

== PostgreSQL Jobs for May ==

http://archives.postgresql.org/pgsql-jobs/2019-05/

== PostgreSQL Local ==

PGDay.IT 2019 will take place May 16th and May 17th in Bologna, Italy.
https://2019.pgday.it/en/

PgConf Belgium will take place on May 17, 2019 at the UCLL Campus in Haasrode
(near Leuven). Registration is open.
http://pgconf.be

PGCon 2019 will take place in Ottawa on May 28-31, 2019. Registration is open.
https://www.pgcon.org/2019

pgibz will be held in Ibiza, Spain on June 19-23, 2019. The CfP is open.
https://pgibz.io/

Swiss PGDay 2019 will take place in Rapperswil (near Zurich) on June 28, 2019.
Registration is open.
http://www.pgday.ch/2019/

PostgresLondon 2019 will be July 2-3, 2019 with an optional training day on
July 1.
http://postgreslondon.org

PGConf.Brazil 2019 will take place August 1-3, 2019 in São Paulo.
http://pgconf.com.br

The first Austrian pgDay, will take place September 6, 2019 at the Hilton Garden
Inn in Wiener Neustadt.
https://pgday.at/en/

PostgresConf South Africa 2019 will take place in Johannesburg on October 8-9, 2019
The Call for Papers is open through June 30, 2019.
https://postgresconf.org/conferences/SouthAfrica2019

== PostgreSQL in the News ==

Planet PostgreSQL: http://planet.postgresql.org/

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm PST8PDT to david(at)fetter(dot)org(dot)

== Applied Patches ==

Tom Lane pushed:

- Fix style violations in syscache lookups. Project style is to check the
success of SearchSysCacheN and friends by applying HeapTupleIsValid to the
result. A tiny minority of calls creatively did it differently. Bring them
into line with the rest. This is just cosmetic, since HeapTupleIsValid is
indeed just a null check at the moment ... but that may not be true forever,
and in any case it puts a mental burden on readers who may wonder why these
call sites are not like the rest. Back-patch to v11 just to keep the branches
in sync. (The bulk of these errors seem to have originated in v11 or v12,
though a few are old.) Per searching to see if anyplace else had made the
same error repaired in 62148c352.
https://git.postgresql.org/pg/commitdiff/9691aa72e2a7fb146ac759e1f8a8b04962128cc0

- Bring pg_nextoid()'s error messages into line with message style guide.
Noticed while reviewing nearby code. Given all the disclaimers about this not
being meant as user-facing code, I wonder whether we should make these
non-translatable? But in any case there's little excuse for them not to be
good English.
https://git.postgresql.org/pg/commitdiff/bd5e8b627bae9e394352a388d2ad30465eafea2c

- Avoid "invalid memory alloc request size" while reading pg_stat_activity. On a
64-bit machine, if you set track_activity_query_size and max_connections such
that their product exceeds 1GB, shared memory setup will still succeed (given
enough RAM), but attempts to read pg_stat_activity fail with "invalid memory
alloc request size". Work around that by using MemoryContextAllocHuge to
allocate the local copy of the activity strings. Using the "huge" API costs
us nothing extra in normal cases, and it seems better than throwing an error
and/or explaining to people why they can't do this. This situation seems
insanely profligate today, but who knows what people will consider normal in
ten or twenty years? So let's fix it in HEAD but not worry about a
back-patch. Per report from James Tomson. Discussion:
https://postgr.es/m/1CFDCCD6-B268-48D8-85C8-400D2790B2C3@pushd.com
https://git.postgresql.org/pg/commitdiff/8d0ddccec6366a2851da7d350b33659203aa644b

- Clean up the behavior and API of catalog.c's is-catalog-relation tests. The
right way for IsCatalogRelation/Class to behave is to return true for OIDs
less than FirstBootstrapObjectId (not FirstNormalObjectId), without any of the
ad-hoc fooling around with schema membership. The previous code was wrong
because (1) it claimed that information_schema tables were not catalog
relations but their toast tables were, which is silly; and (2) if you dropped
and recreated information_schema, which is a supported operation, the behavior
changed. That's even sillier. With this definition, "catalog relations" are
exactly the ones traceable to the postgres.bki data, which seems like what we
want. With this simplification, we don't actually need access to the pg_class
tuple to identify a catalog relation; we only need its OID. Hence, replace
IsCatalogClass with "IsCatalogRelationOid(oid)". But keep IsCatalogRelation
as a convenience function. This allows fixing some arguably-wrong semantics
in contrib/sepgsql and ReindexRelationConcurrently, which were using an
IsSystemNamespace test where what they really should be using is
IsCatalogRelationOid. The previous coding failed to protect toast tables of
system catalogs, and also was not on board with the general principle that
user-created tables do not become catalogs just by virtue of being renamed
into pg_catalog. We can also get rid of a messy hack in ReindexMultipleTables.
While we're at it, also rename IsSystemNamespace to IsCatalogNamespace,
because the previous name invited confusion with the more expansive semantics
used by IsSystemRelation/Class. Also improve the comments in catalog.c.
There are a few remaining places in replication-related code that are
special-casing OIDs below FirstNormalObjectId. I'm inclined to think those
are wrong too, and if there should be any special case it should just extend
to FirstBootstrapObjectId. But first we need to debate whether a FOR ALL
TABLES publication should include information_schema. Discussion:
https://postgr.es/m/21697.1557092753@sss.pgh.pa.us Discussion:
https://postgr.es/m/15150.1557257111@sss.pgh.pa.us
https://git.postgresql.org/pg/commitdiff/2d7d946cd323ce1c1d3f3ef0e5f2f41591afc1b9

- Repair issues with faulty generation of merge-append plans.
create_merge_append_plan failed to honor the CP_EXACT_TLIST flag: it would
generate the expected targetlist but then it felt free to add resjunk sort
targets to it. This demonstrably leads to assertion failures in v11 and HEAD,
and it's probably just accidental that we don't see the same in older
branches. I've not looked into whether there would be any real-world
consequences in non-assert builds. In HEAD, create_append_plan has sprouted
the same problem, so fix that too (although we do not have any test cases that
seem able to reach that bug). This is an oversight in commit 3fc6e2d7f which
invented the CP_EXACT_TLIST flag, so back-patch to 9.6 where that came in.
convert_subquery_pathkeys would create pathkeys for subquery output values if
they match any EquivalenceClass known in the outer query and are available in
the subquery's syntactic targetlist. However, the second part of that
condition is wrong, because such values might not appear in the subquery
relation's reltarget list, which would mean that they couldn't be accessed
above the level of the subquery scan. We must check that they appear in the
reltarget list, instead. This can lead to dropping knowledge about the
subquery's sort ordering, but I believe it's okay, because any sort key that
the outer query actually has any interest in would appear in the reltarget
list. This second issue is of very long standing, but right now there's no
evidence that it causes observable problems before 9.6, so I refrained from
back-patching further than that. We can revisit that choice if somebody finds
a way to make it cause problems in older branches. (Developing useful test
cases for these issues is really problematic; fixing convert_subquery_pathkeys
removes the only known way to exhibit the create_merge_append_plan bug, and
neither of the test cases added by this patch causes a problem in all
branches, even when considering the issues separately.) The second issue
explains bug #15795 from Suresh Kumar R ("could not find pathkey item to sort"
with nested DISTINCT queries). I stumbled across the first issue while
investigating that. Discussion:
https://postgr.es/m/15795-fadb56c8e44ee73c@postgresql.org
https://git.postgresql.org/pg/commitdiff/24c19e9f66863d83009a370604e40b1eaa71bcdd

- Cope with EINVAL and EIDRM shmat() failures in PGSharedMemoryAttach. There's a
very old race condition in our code to see whether a pre-existing shared
memory segment is still in use by a conflicting postmaster: it's possible for
the other postmaster to remove the segment in between our shmctl() and shmat()
calls. It's a narrow window, and there's no risk unless both postmasters are
using the same port number, but that's possible during parallelized "make
check" tests. (Note that while the TAP tests take some pains to choose a
randomized port number, pg_regress doesn't.) If it does happen, we treated
that as an unexpected case and errored out. To fix, allow EINVAL to be
treated as segment-not-present, and the same for EIDRM on Linux. AFAICS, the
considerations here are basically identical to the checks for acceptable
shmctl() failures, so I documented and coded it that way. While at it, adjust
PGSharedMemoryAttach's API to remove its undocumented dependency on
UsedShmemSegAddr in favor of passing the attach address explicitly. This
makes it easier to be sure we're using a null shmaddr when probing for segment
conflicts (thus avoiding questions about what EINVAL means). I don't think
there was a bug there, but it required fragile assumptions about the state of
UsedShmemSegAddr during PGSharedMemoryIsInUse. Commit c09850992 may have made
this failure more probable by applying the conflicting-segment tests more
often. Hence, back-patch to all supported branches, as that was. Discussion:
https://postgr.es/m/22224.1557340366@sss.pgh.pa.us
https://git.postgresql.org/pg/commitdiff/610747d86e46ae6e94b7288393d08823cc39b498

- Rearrange pgstat_bestart() to avoid failures within its critical section. We
long ago decided to design the shared PgBackendStatus data structure to
minimize the cost of writing status updates, which means that writers just
have to increment the st_changecount field twice. That isn't hooked into any
sort of resource management mechanism, which means that if something were to
throw error between the two increments, the st_changecount field would be left
odd indefinitely. That would cause readers to lock up. Now, since it's also a
bad idea to leave the field odd for longer than absolutely necessary (because
readers will spin while we have it set), the expectation was that we'd treat
these segments like spinlock critical sections, with only short, more or less
straight-line, code in them. That was fine as originally designed, but commit
9029f4b37 broke it by inserting a significant amount of non-straight-line code
into pgstat_bestart(), code that is very capable of throwing errors, not to
mention taking a significant amount of time during which readers will spin. We
have a report from Neeraj Kumar of readers actually locking up, which I
suspect was due to an encoding conversion error in X509_NAME_to_cstring,
though conceivably it was just a garden-variety OOM failure. Subsequent
commits have loaded even more dubious code into pgstat_bestart's critical
section (and commit fc70a4b0d deserves some kind of booby prize for managing
to miss the critical section entirely, although the negative consequences seem
minimal given that the PgBackendStatus entry should be seen by readers as
inactive at that point). The right way to fix this mess seems to be to
compute all these values into a local copy of the process' PgBackendStatus
struct, and then just copy the data back within the critical section proper.
This plan can't be implemented completely cleanly because of the struct's
heavy reliance on out-of-line strings, which we must initialize separately
within the critical section. But still, the critical section is far smaller
and safer than it was before. In hopes of forestalling future errors of the
same ilk, rename the macros for st_changecount management to make it more
apparent that the writer-side macros create a critical section. And to
prevent the worst consequences if we nonetheless manage to mess it up anyway,
adjust those macros so that they really are a critical section, ie they now
bump CritSectionCount. That doesn't add much overhead, and it guarantees that
if we do somehow throw an error while the counter is odd, it will lead to
PANIC and a database restart to reset shared memory. Back-patch to 9.5 where
the problem was introduced. In HEAD, also fix an oversight in commit
b0b39f72b: it failed to teach pgstat_read_current_status to copy st_gssstatus
data from shared memory to local memory. Hence, subsequent use of that data
within the transaction would potentially see changing data that it shouldn't
see. Discussion:
https://postgr.es/m/CAPR3Wj5Z17=+eeyrn_ZDG3NQGYgMEOY6JV6Y-WRRhGgwc16U3Q@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/85ccb6899c6c8639bb3e5962ea3bcce5d886e613

Dean Rasheed pushed:

- Fix security checks for selectivity estimation functions with RLS. In commit
e2d4ef8de8, security checks were added to prevent user-supplied operators from
running over data from pg_statistic unless the user has table or column
privileges on the table, or the operator is leakproof. For a table with RLS,
however, checking for table or column privileges is insufficient, since that
does not guarantee that the user has permission to view all of the column's
data. Fix this by also checking for securityQuals on the RTE, and insisting
that the operator be leakproof if there are any. Thus the leakproofness check
will only be skipped if there are no securityQuals and the user has table or
column privileges on the table -- i.e., only if we know that the user has
access to all the data in the column. Back-patch to 9.5 where RLS was added.
Dean Rasheed, reviewed by Jonathan Katz and Stephen Frost. Security:
CVE-2019-10130
https://git.postgresql.org/pg/commitdiff/1aebfbea83c4a3e1a0aba4b0910135dc5a45666c

- Use checkAsUser for selectivity estimator checks, if it's set. In
examine_variable() and examine_simple_variable(), when checking the user's
table and column privileges to determine whether to grant access to the
pg_statistic data, use checkAsUser for the privilege checks, if it's set. This
will be the case if we're accessing the table via a view, to indicate that we
should perform privilege checks as the view owner rather than the current
user. This change makes this planner check consistent with the check in the
executor, so the planner will be able to make use of statistics if the table
is accessible via the view. This fixes a performance regression introduced by
commit e2d4ef8de8, which affects queries against non-security barrier views in
the case where the user doesn't have privileges on the underlying table, but
the view owner does. Note that it continues to provide the same safeguards
controlling access to pg_statistic for direct table access (in which case
checkAsUser won't be set) and for security barrier views, because of the
nearby checks on rte->security_barrier and rte->securityQuals. Back-patch to
all supported branches because e2d4ef8de8 was. Dean Rasheed, reviewed by
Jonathan Katz and Stephen Frost.
https://git.postgresql.org/pg/commitdiff/a0905056fd6b0927dd33f185adc9e7503515fc0d

Michaël Paquier pushed:

- Add tests for error message generation in partition tuple routing. This adds
extra tests for the error message generated for partition tuple routing in the
executor, using more than three levels of partitioning including partitioned
tables with no partitions. These tests have been added to fix CVE-2019-10129
on REL_11_STABLE. HEAD has no active bugs in this area, but it lacked
coverage. Author: Michael Paquier Reviewed-by: Noah Misch Security:
CVE-2019-10129
https://git.postgresql.org/pg/commitdiff/91248608a61d5504f8ac46534136de9b3717fed2

- Remove some code related to 7.3 and older servers from tools of src/bin/. This
code was broken as of 582edc3, and is most likely not used anymore. Note that
pg_dump supports servers down to 8.0, and psql has code to support servers
down to 7.4. Author: Julien Rouhaud Reviewed-by: Tom Lane Discussion:
https://postgr.es/m/CAOBaU_Y5y=zo3+2gf+2NJC1pvMYPcbRXoQaPXx=U7+C8Qh4CzQ@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/af82f95abb23a56d18fd009ef9eca68ef803a041

- Fix error status of vacuumdb when multiple jobs are used. When running a batch
of VACUUM or ANALYZE commands on a given database, there were cases where it
is possible to have vacuumdb not report an error where it actually should,
leading to incorrect status results. Author: Julien Rouhaud Reviewed-by: Amit
Kapila, Michael Paquier Discussion:
https://postgr.es/m/CAOBaU_ZuTwz7CtqLYJ1Ouuh272bTQPLN8b1bAPk0bCBm4PDMTQ@mail.gmail.com
Backpatch-through: 9.5
https://git.postgresql.org/pg/commitdiff/3ae3c18b362817f9412c380539f1a16c7abb79c9

- Improve and fix some error handling for REINDEX INDEX/TABLE CONCURRENTLY. This
improves the user experience when it comes to restrict several flavors of
REINDEX CONCURRENTLY. First, for INDEX, remove a restriction on shared
relations as we already check after catalog relations. Then, for TABLE, add a
proper error message when attempting to run the command on system catalogs.
The code path of CREATE INDEX CONCURRENTLY already complains about that, but
if a REINDEX is issued then then the error generated is confusing. While on
it, add more tests to check restrictions on catalog indexes and on toast
table/index for catalogs. Some error messages are improved, with wording
suggestion coming from Tom Lane. Reported-by: Tom Lane Author: Michael
Paquier Reviewed-by: Tom Lane Discussion:
https://postgr.es/m/23694.1556806002@sss.pgh.pa.us
https://git.postgresql.org/pg/commitdiff/508300e2e141a9fd87758ce01374c5b0597986fd

- Fix and improve description of locktag types in lock.h. The description of the
lock type for speculative insertions was incorrect, being copy-pasted from
another one. As discussed, also move the description for all the fields of
lock tag types from the structure listing lock tag types to the set of macros
setting each LOCKTAG. Author: John Naylor Discussion:
https://postgr.es/m/CACPNZCtA0-ybaC4fFfaDq_8p_TUOLvGxZH9Dm-=TMHZJarBa7Q@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/752f06443fba7e906cd98987f247297663f19a16

- Fix error reporting in reindexdb. When failing to reindex a table or an index,
reindexdb would generate an extra error message related to a database failure,
which is misleading. Backpatch all the way down, as this has been introduced
by 85e9a5a0. Discussion:
https://postgr.es/m/CAOBaU_Yo61RwNO3cW6WVYWwH7EYMPuexhKqufb2nFGOdunbcHw@mail.gmail.com
Author: Julien Rouhaud Reviewed-by: Daniel Gustafsson, Álvaro Herrera, Tom
Lane, Michael Paquier Backpatch-through: 9.4
https://git.postgresql.org/pg/commitdiff/e51bad8fb4c3f0ad5cb173034afdc0b349c7e488

Álvaro Herrera pushed:

- Revert "Make pg_dump emit ATTACH PARTITION instead of PARTITION OF". ... and
fallout (from branches 10, 11 and master). The change was ill-considered, and
it broke a few normal use cases; since we don't have time to fix it, we'll try
again after this week's minor releases. Reported-by: Rushabh Lathia
Discussion:
https://postgr.es/m/CAGPqQf0iQV=PPOv2Btog9J9AwOQp6HmuVd6SbGTR_v3Zp2XT1w@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/a1ec7402e9a0392f95bc79542af0efcd5c6e7929

- Fix error messages. Some messages related to foreign servers were reporting
the server name without quotes, or not at all; our style is to have all names
be quoted, and the server name already appears quoted in a few other messages,
so just add quotes and make them all consistent. Remove an extra "s" in other
messages (typos introduced by myself in f56f8f8da6af).
https://git.postgresql.org/pg/commitdiff/61639816b870347677e6e6945604e0d9da1837ca

Bruce Momjian pushed:

- docs: fist draft version of the PG 12 release notes. Still needs text markup,
links, word wrap, and indenting.
https://git.postgresql.org/pg/commitdiff/bdf595adbca195fa54a909c74a5233ebc30641a1

- doc: update PG 12 release notes, v2. Adjustments requested by reviewers.
Reported-by: Amit Kapila, Thomas Munro, Andrew Gierth, Amit Langote, Oleg
Bartunov, Michael Paquier, Alvaro Herrera, Tatsuo Ishii Discussion:
https://postgr.es/m/20190506233029.ozwged67i7s4qd6c@momjian.us
https://git.postgresql.org/pg/commitdiff/64084d6857b1d8ac05ae46f658b6c244aa6ab591

- docs: update release notes with fixes. Reported-by: Justin Pryzby
Discussion: https://postgr.es/m/20190508203204.GA25482@telsasoft.com
https://git.postgresql.org/pg/commitdiff/81ddfa2e4d9350eb68f28cde8ae6a7e0b39ef2ac

- doc: more PG 12 release note adjustments. This adds two more items that
should have been included in the beginning. Reported-by: Justin Pryzby
Discussion: https://postgr.es/m/20190508203204.GA25482@telsasoft.com
https://git.postgresql.org/pg/commitdiff/79697d039f567cd52d844077244fb85df10dce19

- doc: fix capitalization in PG 12 release notes. Reported-by: Thomas Munro
Discussion:
https://postgr.es/m/CA+hUKGJpep8uSXoDtVF6iROCRKce-39HEhDPUaYFyMn0U5e9ug@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/32fe7ee2dd2b2aa8620e69451f60b2b35989677c

- doc: more PG 12 wording adjustments. Reported-by: Justin Pryzby Discussion:
https://postgr.es/m/20190510001959.GK3925@telsasoft.com
https://git.postgresql.org/pg/commitdiff/97b1654da7dd38fa50c9b6139f4213a1c47f0c39

- doc: PG 12 wording improvments. Reported-by: Justin Pryzby Discussion:
https://postgr.es/m/20190510001335.GJ3925@telsasoft.com
https://git.postgresql.org/pg/commitdiff/d0bbf871ca243eafcac7a84138741521c1aea3d2

- doc: add markup for PG 12 release note text. I will add links to other parts
of the docs later.
https://git.postgresql.org/pg/commitdiff/c65bcfe9aeecae1c6204ad60612fae43669a88b0

- doc: PG 12 release note adjustment. Reported-by: Justin Pryzby Discussion:
https://postgr.es/m/20190510013449.GL3925@telsasoft.com
https://git.postgresql.org/pg/commitdiff/b299efaea433a7d2c04ce124e4f6588807bbe87a

- docs: PG 12 docs, clarify btree index changes. Reported-by: Peter Geoghegan
Discussion:
https://postgr.es/m/CAH2-WzkSYOM1GJVGtAbRW-OqymoCD=QWYG6ro+GaoOW-jPRuDQ@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/809e248299894b02e8f4eb3ee17829b2ae14ce9d

- docs: properly attribute PG 12 rel item to James Coleman. Reported-by: David
Rowley Discussion:
https://postgr.es/m/CAKJS1f-NDmeA_tb0oRFhrgf19xq3A9MeoyMcckY04Ct=_i0c2A@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/8aa1b0885e3d248dbf3e0b0c544125d13dbc36d0

- doc: reorder attribution of PG 12 btree item. Reported-by: Alexander Korotkov
Discussion:
https://postgr.es/m/CAPpHfdvkM-PkyrK6LQitJUDmC_1kOCEtTuseoVhCT=ew0XJmGg@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/05f9eba3498f4dbc8687f66eff53a2bfb88f2595

- doc: improve PG 12 item on partitioned tables. Reported-by: Amit Langote
Discussion:
https://postgr.es/m/5936b052-5d92-a2c9-75d2-0245fb2330b5@lab.ntt.co.jp
https://git.postgresql.org/pg/commitdiff/13d258ec0e55ebc4378e848934f0f4c0ffac0a6f

- doc: add Heikki to PG 12 release note btree item. Reported-by: Peter
Geoghegan Discussion:
https://postgr.es/m/CAH2-WzkrX-aA7d3OYtQT+8Mspq+tU5vwuVz=FTzMH3CdrSyprA@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/064df0edfee91d3843d54d1f67c05a4634690a12

- docs: adjust PG 12 floating point item. Reported-by: Andrew Gierth
Discussion: https://postgr.es/m/87r295hjur.fsf@news-spur.riddles.org.uk
https://git.postgresql.org/pg/commitdiff/0edc8fc47bc4482ceac85b09575d6372dbbc0bbf

- docs: add links from the PG 12 release notes to the main docs.
https://git.postgresql.org/pg/commitdiff/1708974485e82e1cf4694c683faa70fc5bcf142c

- docs: PG 12 release notes, mention that REINDEX could now fail. This is
because of the new tid in the index entry. Reported-by: Peter Geoghegan
https://git.postgresql.org/pg/commitdiff/d56fd6357a5e0e76fecf20c3dc762c5301290331

- doc: remove pg_config mention from PG 12 release notes. Reported-by: Tom Lane
Discussion: https://postgr.es/m/28209.1556556696@sss.pgh.pa.us
https://git.postgresql.org/pg/commitdiff/31f11f964734dbe2af2915182bf79f69e337d4d7

- docs: remove second mention of btree max length reduction. I already added
that to the incompatibility section as a separate item. Reported-by: Peter
Geoghegan
https://git.postgresql.org/pg/commitdiff/4217d15d91128355ea13e6fb9c031b826e2a1335

Amit Kapila pushed:

- Revert "Avoid the creation of the free space map for small heap relations".
This feature was using a process local map to track the first few blocks in
the relation. The map was reset each time we get the block with enough
freespace. It was discussed that it would be better to track this map on a
per-relation basis in relcache and then invalidate the same whenever vacuum
frees up some space in the page or when FSM is created. The new design would
be better both in terms of API design and performance. List of commits
reverted, in reverse chronological order: 06c8a5090e Improve code comments
in b0eaa4c51b. 13e8643bfc During pg_upgrade, conditionally skip transfer of
FSMs. 6f918159a9 Add more tests for FSM. 9c32e4c350 Clear the local map when
not used. 29d108cdec Update the documentation for FSM behavior.. 08ecdfe7e5
Make FSM test portable. b0eaa4c51b Avoid creation of the free space map for
small heap relations. Discussion:
https://postgr.es/m/20190416180452.3pm6uegx54iitbt5@alap3.anarazel.de
https://git.postgresql.org/pg/commitdiff/7db0cde6b58eef2ba0c70437324cbc7622230320

Peter Eisentraut pushed:

- doc: Generate keywords table automatically. The SQL keywords table in the
documentation had until now been generated by some ad hoc scripting outside
the source tree once for each major release. This changes it to an automated
process. We have the PostgreSQL keywords available in a parseable format in
parser/kwlist.h. For the relevant SQL standard versions, keep the keyword
lists in new text files. A new script generate-keywords-table.pl pulls it all
together and produces a DocBook table. The final output in the documentation
should be identical after this change. Discussion:
https://www.postgresql.org/message-id/flat/07daeadd-8c82-0d95-5e19-e350502cb749%402ndquadrant.com
https://git.postgresql.org/pg/commitdiff/b753bc0c84e51c200ec7de6cefb6f689d13fef62

- Fix table lock levels for REINDEX INDEX CONCURRENTLY. REINDEX CONCURRENTLY
locks tables with ShareUpdateExclusiveLock rather than the ShareLock used by a
plain REINDEX. However, RangeVarCallbackForReindexIndex() was not updated for
that and still used the ShareLock only. This would lead to lock upgrades
later, leading to possible deadlocks. Reported-by: Andres Freund
<andres(at)anarazel(dot)de> Reviewed-by: Andres Freund <andres(at)anarazel(dot)de>
Reviewed-by: Michael Paquier <michael(at)paquier(dot)xyz> Discussion:
https://www.postgresql.org/message-id/flat/20190430151735.wi52sxjvxsjvaxxt%40alap3.anarazel.de
https://git.postgresql.org/pg/commitdiff/add85ead4ab969c1e31d64f4476c2335856f4aa9

- Fix grammar in error message.
https://git.postgresql.org/pg/commitdiff/02daece4ab4cd85b80d04469056e5b631918c9d6

- pg_controldata: Add common gettext flags. So it picks up strings in pg_log_*
calls. This was forgotten when it was added to all other relevant
subdirectories.
https://git.postgresql.org/pg/commitdiff/cd805f46d857291b26ba6eb491ce11b6e0fc9ad3

Magnus Hagander pushed:

- Fix typos and clarify a comment. Author: Daniel Gustafsson <daniel(at)yesql(dot)se>
https://git.postgresql.org/pg/commitdiff/98719af6c2e30d538cd05cfe044f58ba4067b52b

Fujii Masao pushed:

- Add TRUNCATE parameter to VACUUM. This commit adds new parameter to VACUUM
command, TRUNCATE, which specifies that VACUUM should attempt to truncate off
any empty pages at the end of the table and allow the disk space for the
truncated pages to be returned to the operating system. This parameter, if
specified, overrides the vacuum_truncate reloption. If neither the reloption
nor the VACUUM option is used, the default is true, as before. Author: Fujii
Masao Reviewed-by: Julien Rouhaud, Masahiko Sawada Discussion:
https://postgr.es/m/CAD21AoD+qtrSDL=GSma4Wd3kLYLeRC0hPna-YAdkDeV4z156vg@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/b84dbc8eb80b43e554891c459a19969d9fbdefe5

- Fix documentation for the privileges required for replication functions.
Previously it's documented that use of replication functions is restricted to
superusers. This is true for the functions which use replication origin, but
not for pg_logicl_emit_message() and functions which use replication slot. For
example, not only superusers but also users with REPLICATION privilege is
allowed to use the functions for replication slot. This commit fixes the
documentation for the privileges required for those replication functions.
Back-patch to 9.4 (all supported versions). Author: Matsumura Ryo Discussion:
https://postgr.es/m/03040DFF97E6E54E88D3BFEE5F5480F74ABA6E16@G01JPEXMBYT04
https://git.postgresql.org/pg/commitdiff/df631ebc737e9d9cdf8d0691969d404f1bd584a4

Alexander Korotkov pushed:

- Remove word "singleton" out of jsonpath docs. Word "singleton" is hard for
user understanding, especially taking into account there is only one place
it's used in the docs and there is even no definition. Use more evident
wording instead. Discussion:
https://postgr.es/m/23737.1556550645%40sss.pgh.pa.us
https://git.postgresql.org/pg/commitdiff/53ae0b16d6f60a15427e081091b2b81e36e674ee

- Improve error reporting in jsonpath. This commit contains multiple
improvements to error reporting in jsonpath including but not limited to
getting rid of following things: * definition of error messages in macros,
* errdetail() when valueable information could fit to errmsg(), * word
"singleton" which is not properly explained anywhere, * line breaks in error
messages. Reported-by: Tom Lane Discussion:
https://postgr.es/m/14890.1555523005%40sss.pgh.pa.us Author: Alexander
Korotkov Reviewed-by: Tom Lane
https://git.postgresql.org/pg/commitdiff/29ceacc3f93720d3ebb7e7e999f8b7fe9622389c

- Add jsonpath_encoding_1.out changes missed in 29ceacc3f9. Reported-by: Tom
Lane Discussion: https://postgr.es/m/14305.1557268259%40sss.pgh.pa.us
https://git.postgresql.org/pg/commitdiff/e5f978631722bb3cac42f0eb6e65e947e0f088ec

Peter Geoghegan pushed:

- Correct obsolete nbtsort.c minimum key comment. It is no longer possible under
any circumstances for nbtree code to reconstruct a strict lower bound key
(parent page's pivot tuple key) for a right sibling page by retrieving the
first item in the right sibling page.
https://git.postgresql.org/pg/commitdiff/d65b5ccad626e1942c862e8a70f56ad7ee7a8003

- Remove obsolete nbtree split REDO routine comment. Commit dd299df8189, which
added suffix truncation to nbtree, simplified the WAL record format used by
page splits. It became necessary to explicitly WAL-log the new high key for
the left half of a split in all cases, which relieved the REDO routine from
having to reconstruct a new high key for the left page by copying the first
item from the right page. Remove a comment that referred to the previous
practice.
https://git.postgresql.org/pg/commitdiff/d95e36dc384e3068ae2db909c228b1800737b18d

Heikki Linnakangas pushed:

- Remove leftover reference to old "flat file" mechanism in a comment. The flat
file mechanism was removed in PostgreSQL 9.0.
https://git.postgresql.org/pg/commitdiff/256be1050cbbf90b1e44d11c8ed10f98255aa27d

Etsuro Fujita pushed:

- Add missing periods to comments.
https://git.postgresql.org/pg/commitdiff/b7434dc007252d7a5e5f7f85700bd7400b1db201

- postgres_fdw: Fix cost estimation for aggregate pushdown. In commit
7012b132d0, which added support for aggregate pushdown in postgres_fdw, the
expense of evaluating the final scan/join target computed by
make_group_input_target() was not accounted for at all in costing aggregate
pushdown paths with local statistics. The right fix for this would be to have
a separate upper stage to adjust the final scan/join relation (see comments
for apply_scanjoin_target_to_paths()); but for now, fix by adding the tlist
eval cost when costing aggregate pushdown paths with local statistics. Apply
this to HEAD only to avoid destabilizing existing plan choices. Author:
Etsuro Fujita Reviewed-By: Antonin Houska Discussion:
https://postgr.es/m/5C66A056.60007%40lab.ntt.co.jp
https://git.postgresql.org/pg/commitdiff/edbcbe277d795ecc339b0e4fa29bae42ce1a7be9

- Doc: Update FDW documentation about GetForeignUpperPaths(). In commit
d50d172e51, which added support for LIMIT/OFFSET pushdown in postgres_fdw, a
new struct was introduced as the extra parameter of GetForeignUpperPaths() set
for UPPERREL_FINAL, but I forgot to update the documentation to mention that.
Author: Etsuro Fujita Discussion:
https://postgr.es/m/CAPmGK17uSXQDe31oRb-z1nYyT6vVzkstZkA3_Wbq38U92b9BmQ%40mail.gmail.com
https://git.postgresql.org/pg/commitdiff/a0be05bab062eff16eafce3df73b3df453a694f4

Thomas Munro pushed:

- Fix copy-and-paste mistakes in documentation. Reported-by: Vik Fearing
https://git.postgresql.org/pg/commitdiff/098344be663f5fc0ad166e7a9e1cd37721ee34d9

- Probe only 127.0.0.1 when looking for ports on Unix. Commit c0985099, later
adjusted by commit 4ab02e81, probed 0.0.0.0 in addition to 127.0.0.1, for the
benefit of Windows build farm animals. It isn't really useful on Unix
systems, and turned out to be a bit inconvenient to users of some corporate
firewall software. Switch back to probing just 127.0.0.1 on non-Windows
systems. Back-patch to 9.6, like the earlier changes. Discussion:
https://postgr.es/m/CA%2BhUKG%2B21EPwfgs4m%2BtqyRtbVqkOUvP8QQ8sWk9%2Bh55Aub1H3A%40mail.gmail.com
https://git.postgresql.org/pg/commitdiff/8efe710d9c84502b3e6a9487937bccf881f56d9c

- Fix SxactGlobalXmin tracking. Commit bb16aba50 broke the code that maintains
SxactGlobalXmin. It could get stuck when a well-timed READ ONLY transaction
runs. If SxactGlobalXmin stops advancing, transactions on the
FinishedSerializableTransactions queue are never cleaned up, so resources are
effectively leaked. Revert that hunk of the commit. Also revert another
similar hunk that was probably harmless, but unnecessary and unjustified,
relating to the DOOMED flag in case of RO_SAFE early release. Author: Thomas
Munro Reported-by: Tom Lane Discussion:
https://postgr.es/m/16170.1557251214%40sss.pgh.pa.us
https://git.postgresql.org/pg/commitdiff/47a338cfcd67139a1f91892b080934fcfc3aea03

Andres Freund pushed:

- Remove reindex_catalog test from test schedules. As none of the approaches for
avoiding the deadlock issues seem promising enough, and all the expected
reindex related changes have been made, apply 60c2951e1bab7e to master as
well. Discussion: https://postgr.es/m/4622.1556982247@sss.pgh.pa.us
https://git.postgresql.org/pg/commitdiff/5997a8f4d7478ae6bac4dab12ca1f883e41a7aa1

Andrew Gierth pushed:

- Fix editing error in floating-point docs. My fault; the error was introduced
in the Ryu patch.
https://git.postgresql.org/pg/commitdiff/b721e201a0bcf0f9e1795c295e134e47d698e80c

Noah Misch pushed:

- Honor TEMP_CONFIG in TAP suites. The buildfarm client uses TEMP_CONFIG to
implement its extra_config setting. Except for stats_temp_directory,
extra_config now applies to TAP suites; extra_config values seen in the past
month are compatible with this. Back-patch to 9.6, where PostgresNode was
introduced, so the buildfarm can rely on it sooner. Reviewed by Andrew
Dunstan and Tom Lane. Discussion:
https://postgr.es/m/20181229021950.GA3302966@rfd.leadboat.com
https://git.postgresql.org/pg/commitdiff/54c2ecb56707ed00844b8678a79570dd34cb95a3

== Pending Patches ==

Peter Geoghegan sent in a patch to standardize line pointers/item pointer terminology.

Thomas Munro sent in two revisions of a patch to detach from DSM segments last
in resowner.c

Ashwin Agrawal sent in a patch to adjust the toy table AM implementation to
match the latest APIs.

Thomas Munro sent in a patch to probe only 127.0.0.1 when looking for ports on
Unix.

Dilip Kumar, Thomas Munro, and Kuntal Ghosh traded patches to clean up orphaned
files using undo logs.

Ryo Matsumura sent in another revision of a patch to add PREPARE to eCPG.

David Fetter sent in two more revisions of a patch to add a way to supply stdin
to TAP tests.

David Fetter sent in two more revisions of a patch to add an ALL to EXPLAIN.

Kyotaro HORIGUCHI sent in three more revisions of a patch to fix partial
aggregation for statistical functions.

John Naylor sent in two revisions of a patch to fix a copy-paste-o comment in
lock.h.

Heikki Linnakangas sent in a patch to detect internal GiST page splits correctly
during index build.

Laurenz Albe sent in another revision of a patch to untangle some stuff about
identity sequences.

Tomáš Vondra sent in another revision of a patch to add a per-slice overflow
file.

Justin Pryzby sent in another revision of a patch to clean up/remove/update
references to OID columns.

Thomas Munro sent in a patch to prevent SxactGlobalXmin tracking from leaking
resources.

Thomas Munro sent in a patch to add a SMGR discriminator to buffer tags.

Pavel Stěhule sent in another revision of a patch to implement schema variables.

Paul A Jungwirth sent in another revision of a patch to implement range_agg.

Masahiko Sawada sent in a patch to add a toast.vacuum_index_cleanup GUC.

Masahiko Sawada sent in two options for a patch either to add an --index-cleanup
option to vacuumdb, or to add --truncate option to vacuumdb.

Dean Rasheed sent in a patch to plug a data leak to unprivileged users in
multivariate MCV stats.

Andres Freund sent in a patch to a test for the speculative insert abort case.

Julien Rouhaud sent in three revisions of a patch to fix a bug in reindexdb's
error reporting where reindexdb could report an extraneous message saying an
error happened while reindexing a database if it failed reindexing a table or an
index.

Dmitry Dolgov sent in another revision of a patch to implement index skip scans.

Fabien COELHO sent in a patch to make libpq documentation navigable between
functions.

Julien Rouhaud sent in a patch to clean up and refactor reindexdb.c.

Rikard Falkeborn sent in a patch to fix a dead code return value in jsonb_util.

Jonathan S. Katz sent in a patch to fix a typo in the release notes.