== PostgreSQL Weekly News - April 14, 2019 ==

== PostgreSQL Weekly News - April 14, 2019 ==

PostgresConf South Africa 2019 will take place in Johannesburg on October 8-9, 2019
The Call for Papers is open through June 30, 2019.
https://postgresconf.org/conferences/SouthAfrica2019

== PostgreSQL Product News ==

pgMustard, a user interface for 'explain analyze' which provides performance
tips, released.
https://www.pgmustard.com/

repods, a cloud data platform based on PostgreSQL, released.
https://repods.io

pgAdmin4 4.5, a web- and native GUI control center for PostgreSQL, released.
https://www.pgadmin.org/docs/pgadmin4/dev/release_notes_4_5.html

== PostgreSQL Jobs for April ==

http://archives.postgresql.org/pgsql-jobs/2019-04/

== PostgreSQL Local ==

The German-speaking PostgreSQL Conference 2019 will take place on May 10, 2019
in Leipzig.
http://2019.pgconf.de/

PGDay.IT 2019 will take place May 16th and May 17th in Bologna, Italy.
https://2019.pgday.it/en/

PGCon 2019 will take place in Ottawa on May 28-31, 2019. Registration is open.
https://www.pgcon.org/2019

Swiss PGDay 2019 will take place in Rapperswil (near Zurich) on June 28, 2019.
The CfP is open through April 18, 2019, and registration is open.
http://www.pgday.ch/2019/

PostgresLondon 2019 will be July 2-3, 2019 with an optional training day on
July 1.
http://postgreslondon.org

PGConf.Brazil 2019 will take place August 1-3, 2019 in São Paulo.
http://pgconf.com.br

The first Austrian pgDay, will take place September 6, 2019 at the Hilton Garden
Inn in Wiener Neustadt.
https://pgday.at/en/

== PostgreSQL in the News ==

Planet PostgreSQL: http://planet.postgresql.org/

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm PST8PDT to david(at)fetter(dot)org(dot)

== Applied Patches ==

Tom Lane pushed:

- Avoid fetching past the end of the indoption array. pg_get_indexdef_worker
carelessly fetched indoption entries even for non-key index columns that don't
have one. 99.999% of the time this would be harmless, since the code wouldn't
examine the value ... but some fine day this will be a fetch off the end of
memory, resulting in SIGSEGV. Detected through valgrind testing. Odd that
the buildfarm's valgrind critters haven't noticed.
https://git.postgresql.org/pg/commitdiff/80a96e066eecb6bd1788964b5911a405d932a784

- Fix EvalPlanQualStart to handle partitioned result rels correctly. The
es_root_result_relations array needs to be shallow-copied in the same way as
the main es_result_relations array, else EPQ rechecks on partitioned result
relations fail, as seen in bug #15677 from Norbert Benkocs. Amit Langote,
isolation test case added by me Discussion:
https://postgr.es/m/15677-0bf089579b4cd02d@postgresql.org Discussion:
https://postgr.es/m/19321.1554567786@sss.pgh.pa.us
https://git.postgresql.org/pg/commitdiff/a8cb8f124679e0c373fdd07108b136e1cf1ee14a

- Fix improper interaction of FULL JOINs with lateral references.
join_is_legal() needs to reject forming certain outer joins in cases where
that would lead the planner down a blind alley. However, it mistakenly
supposed that the way to handle full joins was to treat them as applying the
same constraints as for left joins, only to both sides. That doesn't work, as
shown in bug #15741 from Anthony Skorski: given a lateral reference out of a
join that's fully enclosed by a full join, the code would fail to believe that
any join ordering is legal, resulting in errors like "failed to build any
N-way joins". However, we don't really need to consider full joins at all for
this purpose, because we effectively force them to be evaluated in syntactic
order, and that order is always legal for lateral references. Hence, get rid
of this broken logic for full joins and just ignore them instead. This seems
to have been an oversight in commit 7e19db0c0. Back-patch to all supported
branches, as that was. Discussion:
https://postgr.es/m/15741-276f1f464b3f40eb@postgresql.org
https://git.postgresql.org/pg/commitdiff/45f8eaa8e3031c9cb12deb1b5e294bc052b378f2

- Test some more cases with partitioned tables in EvalPlanQual. We weren't
testing anything involving EPQ on UPDATEs that move tuples into different
partitions. Depending on the implementation, it might be that these cases
aren't actually very interesting ... but given our thin coverage of EPQ in
general, I think it's a good idea to have a test case. Amit Langote, minor
tweak by me Discussion:
https://postgr.es/m/7889df35-ad1a-691a-00e3-4d4b18f364e3@lab.ntt.co.jp
https://git.postgresql.org/pg/commitdiff/a2418f9e238794fcaaf00bbd5b8f953ca2856aa0

- Prevent inlining of multiply-referenced CTEs with outer recursive refs. This
has to be prevented because inlining would result in multiple self-references,
which we don't support (and in fact that's disallowed by the SQL spec, see
statements about linearly vs. nonlinearly recursive queries). Bug fix for
commit 608b167f9. Per report from Yaroslav Schekin (via Andrew Gierth)
Discussion: https://postgr.es/m/87wolmg60q.fsf@news-spur.riddles.org.uk
https://git.postgresql.org/pg/commitdiff/9476131278c7bfc435ad9a21fc8e981272ac0dd2

- Fix backwards test in operator_precedence_warning logic. Warnings about unary
minus might have been wrong. It's a bit surprising that nobody noticed yet
... probably the precedence-warning feature hasn't really been used much in
the field. Rikard Falkeborn Discussion:
https://postgr.es/m/CADRDgG6fzA8A2oeygUw4=o7ywo4kvz26NxCSgpq22nMD73Bx4Q@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/4cae471d1b6bec7493dcb2ca156382bef738f293

- Remove redundant and ineffective test for btree insertion fast path.
indexing.sql's test for this feature was added along with the feature in
commit 2b2727343. However, shortly later that test was rendered ineffective
by commit 074251db6, which limited when the optimization would be applied, so
that the test didn't test it. Since then, commit dd299df81 added new tests (in
btree_index.sql) that actually do test the feature. Code coverage comparisons
confirm that this test sequence adds no meaningful coverage, and it's rather
expensive, accounting for nearly half of the runtime of indexing.sql according
to my measurements. So let's remove it. Per advice from Peter Geoghegan.
Discussion: https://postgr.es/m/735.1554935715@sss.pgh.pa.us
https://git.postgresql.org/pg/commitdiff/f72d9a5e7dda5f89336a60e8b720ef9964a67177

- Remove duplicative polygon SP-GiST sequencing test. Code coverage comparisons
confirm that the tests using quad_poly_tbl_ord_seq1/quad_poly_tbl_ord_idx1 hit
no code paths not also covered by the similar tests using
quad_poly_tbl_ord_seq2/quad_poly_tbl_ord_idx2. Since these test cases are
pretty expensive, they need to contribute more than zero benefit. In passing,
make quad_poly_tbl_ord_seq2 a temp table, since there seems little reason to
keep it around after the test. Discussion:
https://postgr.es/m/735.1554935715@sss.pgh.pa.us
https://git.postgresql.org/pg/commitdiff/4aaa3b5cf1c33d713b8eefa3612a9112cbcf1074

- Move plpgsql error-trapping tests to a new module-specific test file. The test
for statement timeout has a 2-second timeout, which was only moderately
annoying when it was written, but nowadays it contributes a pretty significant
chunk of the elapsed time needed to run the core regression tests on a fast
machine. We can improve this situation by pushing the test into a
plpgsql-specific test file instead of having it in a core regression test.
That's a clean win when considering just the core tests. Even when
considering check-world or a buildfarm test run, we should come out ahead
because the core tests get run more times in those sequences. Furthermore,
since the plpgsql tests aren't currently parallelized, it seems likely that
the timing problems reflected in commit f1e671a0b (which increased that
timeout from 1 sec to 2) will be much less severe in this context. Hence,
let's try cutting the timeout back to 1 second in hopes of a further win for
check-world. We can undo that if buildfarm experience proves it to be a bad
idea. To give the new test file some modicum of intellectual coherency, I
moved the surrounding tests related to error-trapping along with the statement
timeout test proper. Those other tests don't run long enough to have any
particular bearing on test-runtime considerations. The tests are the same as
before, except with minor adjustments to not depend on an externally-created
table. Discussion: https://postgr.es/m/735.1554935715@sss.pgh.pa.us
https://git.postgresql.org/pg/commitdiff/6726d8d476b424633ebdc7068da3f5a6e6da10af

- Split up a couple of long-running regression test scripts. The point of this
change is to increase the potential for parallelism while running the core
regression tests. Most people these days are using parallel testing modes on
multi-core machines, so we might as well try a bit harder to keep multiple
cores busy. Hence, a test that runs much longer than others in its parallel
group is a candidate to be sub-divided. In this patch, create_index.sql and
join.sql are split up. I haven't changed the content of the tests in any way,
just moved them. I moved create_index.sql's SP-GiST-related tests into a new
script create_index_spgist, and moved its btree multilevel page deletion test
over to the existing script btree_index. (btree_index is a more natural home
for that test, and it's shorter than others in its parallel group, so this
doesn't hurt total runtime of that group.) There might be room for more
aggressive splitting of create_index, but this is enough to improve matters
considerably. Likewise, I moved join.sql's "exercises for the hash join code"
into a new file join_hash. Those exercises contributed three-quarters of the
script's runtime. Which might well be excessive ... but for the moment, I'm
satisfied with shoving them into a different parallel group, where they can
share runtime with the roughly-equally-lengthy gist test. (Note for anybody
following along at home: there are interesting interactions between the
runtimes of create_index and anything running in parallel with it, because the
tests of CREATE INDEX CONCURRENTLY in that file will repeatedly block waiting
for concurrent transactions to commit. As committed in this patch,
create_index and create_index_spgist have roughly equal runtimes, but that's
mostly an artifact of forced synchronization of the CONCURRENTLY tests; when
run serially, create_index is much faster. A followup patch will reduce the
runtime of create_index_spgist and thereby also create_index.) Discussion:
https://postgr.es/m/735.1554935715@sss.pgh.pa.us
https://git.postgresql.org/pg/commitdiff/385d396b807bdd7034ad3d0cea3c921d7cb04faa

- Speed up sort-order-comparison tests in create_index_spgist. This test script
verifies that KNN searches of an SP-GiST index produce the same sort order as
a seqscan-and-sort. The FULL JOINs used for that are exceedingly slow,
however. Investigation shows that the problem is that the initial join is on
the rank() values, and we have a lot of duplicates due to the data set
containing 1000 duplicate points. We're therefore going to produce 1000000
join rows that have to be thrown away again by the join filter. We can
improve matters by using row_number() instead of rank(), so that the initial
join keys are unique. The catch is that that makes the results sensitive to
the sorting of rows with equal distances from the reference point. That
doesn't matter for the actually-equal points, but as luck would have it, the
data set also contains two distinct points that have identical distances to
the origin. So those two rows could legitimately appear in either order,
causing unwanted output from the check queries. However, it doesn't seem like
it's the job of this test to check whether the <-> operator correctly computes
distances; its charter is just to verify that SP-GiST emits the values in
distance order. So we can dodge the indeterminacy problem by having the check
only compare row numbers and distances not the actual point values. This
change reduces the run time of create_index_spgist by a good three-quarters,
on my machine, with ensuing beneficial effects on the runtime of create_index
(thanks to interactions with CREATE INDEX CONCURRENTLY tests in the latter).
I see a net improvement of more than 2X in the runtime of their parallel test
group. Discussion: https://postgr.es/m/735.1554935715@sss.pgh.pa.us
https://git.postgresql.org/pg/commitdiff/5874c7055702e1cf5e58543f11dfcff6de2cc260

- Re-order some regression test scripts for more parallelism. Move the strings,
numerology, insert, insert_conflict, select and errors tests to be parts of
nearby parallel groups, instead of executing by themselves. (Moving "select"
required adjusting the constraints test, which uses a table named "tmp" as
select also does. There don't seem to be any other conflicts.) Move psql and
stats_ext to the next parallel group, where the rules test also has a long
runtime. To make it safe to run stats_ext in parallel with rules, I adjusted
the latter to only dump views/rules from the pg_catalog and public schemas,
which was what it was doing anyway. stats_ext makes some views in a transient
schema, which now will not affect rules. Reorder serial_schedule to match
parallel_schedule. Discussion:
https://postgr.es/m/735.1554935715@sss.pgh.pa.us
https://git.postgresql.org/pg/commitdiff/798070ec058fe75757587c9e9916cc35c623f427

- Prevent memory leaks associated with relcache rd_partcheck structures. The
original coding of generate_partition_qual() just copied the list of predicate
expressions into the global CacheMemoryContext, making it effectively
impossible to clean up when the owning relcache entry is destroyed --- the
relevant code in RelationDestroyRelation() only managed to free the topmost
List header :-(. This resulted in a session-lifespan memory leak whenever a
table partition's relcache entry is rebuilt. Fortunately, that's not normally
a large data structure, and rebuilds shouldn't occur all that often in
production situations; but this is still a bug worth fixing back to v10 where
the code was introduced. To fix, put the cached expression tree into its own
small memory context, as we do with other complicated substructures of
relcache entries. Also, deal more honestly with the case that a partition has
an empty partcheck list; while that probably isn't a case that's very
interesting for production use, it's legal. In passing, clarify comments
about how partitioning-related relcache data structures are managed, and add
some Asserts that we're not leaking old copies when we overwrite these data
fields. Amit Langote and Tom Lane Discussion:
https://postgr.es/m/7961.1552498252@sss.pgh.pa.us
https://git.postgresql.org/pg/commitdiff/5f1433ac5e7f943b29ef01266b6b8fc915e6b917

Michaël Paquier pushed:

- Add more tests for partition tuple routing with dropped attributes. As bug
#15733 has proved, we are lacking coverage for partition tuple routing with
dropped attributes when involving three levels of partitioning or more. There
was only an active bug in this area for v11, and HEAD is proving to handle
those scenarios fine, still it lacked some coverage for the previous problem.
Author: Amit Langote, Michael Paquier Discussion:
https://postgr.es/m/15733-7692379e310b80ec@postgresql.org
https://git.postgresql.org/pg/commitdiff/964bae4d8456e5406753027fa5a70181ddb4c835

- Tweak wording of documentation for pg_checksums. Author: Justin Pryzby
Discussion: https://postgr.es/m/20190329143210.GI5815@telsasoft.com
https://git.postgresql.org/pg/commitdiff/e3865c37544d77bd4205dd5361592797b97d1e93

- Fix more strcmp() calls using boolean-like comparisons for result checks. Such
calls can confuse the reader as strcmp() uses an integer as result. The places
patched here have been spotted by Thomas Munro, David Rowley and myself.
Author: Michael Paquier Reviewed-by: David Rowley Discussion:
https://postgr.es/m/20190411021946.GG2728@paquier.xyz
https://git.postgresql.org/pg/commitdiff/d527fda6216780281b90f48820ae978c61c7905c

- Switch TAP tests of pg_rewind to use a role with minimal permissions. Up to
now the tests of pg_rewind have been using a superuser for all the tests
(which is the default of many tests actually, and something that ought to be
reviewed) when involving an online source server, still it is possible to use
a non-superuser role to do that as long as this role is granted permissions to
execute all the source-side functions used for the rewind. This is possible
since v11, and was already documented as of bfc8068. This will allow to catch
up easily any change in pg_rewind if the tool begins to use more backend-side
functions, so as the properties introduced by v11 are kept. Per suggestion
from Peter Eisentraut. Author: Michael Paquier Reviewed-by: Magnus Hagander
Discussion: https://postgr.es/m/20190411041336.GM2728@paquier.xyz
https://git.postgresql.org/pg/commitdiff/d4e2a843e6d6f325c070ee80a0c117ec11675e74

- Fix typos in reloptions.c. Author: Kirk Jamison Discussion:
https://postgr.es/m/D09B13F772D2274BB348A310EE3027C6493463@g01jpexmbkw24
https://git.postgresql.org/pg/commitdiff/d87ab88686fb60ad5a34373de05bb20e632cf003

- Revert "Switch TAP tests of pg_rewind to use a role with minimal permissions".
This reverts commit d4e2a84, which added a new user with limited permissions
to run the TAP tests of pg_rewind. Buildfarm machine members on Windows
jacana and bowerbird have been complaining about that, the new role not being
able to run the rewind because SSPI is not configured to allow it. Fixing the
test requires passing down directly the new user to pg_regress with
--create-role so as SSPI can work properly. Reported-by: Andrew Dunstan
Discussion:
https://postgr.es/m/3cd43d33-f415-cc41-ade3-7230ab15b2c9@2ndQuadrant.com
https://git.postgresql.org/pg/commitdiff/db8db624e826efbe16aab1ae921bae071f98f099

- Switch TAP tests of pg_rewind to use non-superuser role, take two. Up to now
the tests of pg_rewind have been using a superuser for all its tests (which is
the default of many tests actually, and something that ought to be reviewed)
when involving an online source server, still it is possible to use a
non-superuser role to do that as long as this role is granted permissions to
execute all the source-side functions used for the rewind. This is possible
since v11, and was already documented as of bfc8068. PostgresNode::init is
extended so as callers of this routine can add extra options to configure the
authentication of a new node, which gets used by this commit, and allows the
tests to work properly on Windows where SSPI is used. This will allow to
catch up easily any change in pg_rewind if the tool begins to use more
backend-side functions, so as the properties introduced by v11 are kept. Per
suggestion from Peter Eisentraut. Author: Michael Paquier Reviewed-by: Magnus
Hagander Discussion: https://postgr.es/m/20190411041336.GM2728@paquier.xyz
https://git.postgresql.org/pg/commitdiff/d9f543e9e9be15f92abdeaf870e57ef289020191

Andres Freund pushed:

- Fix a number of issues around modifying a previously updated row. This commit
fixes three, unfortunately related, issues: 1) Since 5db6df0c01, the
introduction of DML via tableam, it was possible to trigger "ERROR:
unexpected table_lock_tuple status: 1" when updating a row that was
previously updated in the same transaction - but only when the previously
updated row was before updated in a concurrent transaction (and READ
COMMITTED was used). The reason for that was that that case simply wasn't
expected. Fixing that lead to: 2) Even before the above commit, there were
error checks (introduced in 6868ed7491b7) preventing a row being updated by
different commands within the same statement (say in a function called by
an UPDATE) - but that check wasn't performed when the row was first
updated in a concurrent transaction - instead the second update was
silently skipped in that case. After this change we throw the same error as
we'd without the concurrent transaction. 3) The error messages (introduced in
6868ed7491b7) preventing such updates emitted the same error message for
both DELETE and UPDATE ("tuple to be updated was already modified by an
operation triggered by the current command"). While that could be changed
separately, it made it hard to write tests that verify the correct correct
behavior of the code. This commit changes heap's implementation of
table_lock_tuple() to return TM_SelfModified instead of TM_Invisible
(previously loosely modeled after EvalPlanQualFetch), and teaches
nodeModifyTable.c to handle that in response to table_lock_tuple() and not
just in response to table_(delete|update). Additionally it fixes the wrong
error message (see 3 above). The comment for table_lock_tuple() is also
adjusted to state that TM_Deleted won't return information in TM_FailureData -
it'll not always be available. This also adds tests to ensure that
DELETE/UPDATE correctly error out when affecting a row that concurrently was
modified by another transaction. Author: Andres Freund Reported-By: Tom Lane,
when investigating a bug bug fix to another bug by Amit Langote
Discussion: https://postgr.es/m/19321.1554567786@sss.pgh.pa.us
https://git.postgresql.org/pg/commitdiff/41f5e04aec6cf63ba8392adf70e9289e9c3706d6

- Reset memory context once per tuple in validateForeignKeyConstraint. When
using tableam ExecFetchSlotHeapTuple() might return a separately allocated
tuple. We could use the shouldFree argument to explicitly free it, but it
seems more robust to to protect Also add a CHECK_FOR_INTERRUPTS() after each
tuple. It's likely that each AM has (heap does) a CFI somewhere in the
relevant path, but it seems more robust to have one in
validateForeignKeyConstraint() itself. Note that this only affects the cases
that couldn't be optimized to be verified with a query. Author: Andres Freund
Reviewed-By: Tom Lane (in an earlier version) Discussion:
https://postgr.es/m/19030.1554574075@sss.pgh.pa.us
https://postgr.es/m/CAKJS1f_SHKcPYMsi39An5aUjhAcEMZb6Cx1Sj1QWEWSiKJkBVQ@mail.gmail.com
https://postgr.es/m/20180711185628.mrvl46bjgk2uxoki@alap3.anarazel.de
https://git.postgresql.org/pg/commitdiff/4c9e1bd0a37e7b79dfc797dd91627336e871c1b0

- tableam: comment and formatting fixes. Author: Heikki Linnakangas Discussion:
https://postgr.es/m/9a7fb9cc-2419-5db7-8840-ddc10c93f122@iki.fi
https://git.postgresql.org/pg/commitdiff/6421011ea24db3a59dc7a0058a03e91981d53635

Fujii Masao pushed:

- Add vacuum_truncate reloption. vacuum_truncate controls whether vacuum tries
to truncate off any empty pages at the end of the table. Previously vacuum
always tried to do the truncation. However, the truncation could cause some
problems; for example, ACCESS EXCLUSIVE lock needs to be taken on the table
during the truncation and can cause the query cancellation on the standby even
if hot_standby_feedback is true. Setting this reloption to false can be
helpful to avoid such problems. Author: Tsunakawa Takayuki Reviewed-By:
Julien Rouhaud, Masahiko Sawada, Michael Paquier, Kirk Jamison and Fujii Masao
Discussion:
https://postgr.es/m/CAHGQGwE5UqFqSq1=kV3QtTUtXphTdyHA-8rAj4A=Y+e4kyp3BQ@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/119dcfad988d5b5d9f52b256087869997670aa36

Peter Eisentraut pushed:

- doc: Add note about generated columns in foreign tables. Explain that it is
not enforced that querying a generated column returns data that is consistent
with the data that was stored. This is similar to the note about constraints
nearby. Reported-by: Amit Langote <amitlangote09(at)gmail(dot)com>
https://git.postgresql.org/pg/commitdiff/348f57ce5be96190491e2153abb47060884f8ebf

- doc: Update serial explanation. The CREATE SEQUENCE command should include a
data type specification, since PostgreSQL 10. Reported-by: mjf(at)pearson(dot)co(dot)uk
https://git.postgresql.org/pg/commitdiff/c66000385fe1dc94a6d5525dcd192f17b551fb9b

- doc: Fix whitespace. Author: Julien Rouhaud <rjuju123(at)gmail(dot)com>
https://git.postgresql.org/pg/commitdiff/122fa9f942478f8fdcfba961e01d172574369293

- Replace tabs with spaces in one .sql file. Let's at least keep this consistent
within the same file.
https://git.postgresql.org/pg/commitdiff/9efe068e48f09d915660576f5672bfa9f1c0eb53

- pg_restore: Make not verbose by default. This was accidentally changed in
cc8d41511721d25d557fc02a46c053c0a602fed0. Reported-by: Christoph Berg
<myon(at)debian(dot)org>
https://git.postgresql.org/pg/commitdiff/765525c8c2c6e55abe8c0cd43bf5c728926d76d4

- Fix REINDEX CONCURRENTLY of partitions. In case of a partition index, when
swapping the old and new index, we also need to attach the new index as a
partition and detach the old one. Also, to handle partition indexes, we not
only need to change dependencies referencing the index, but also dependencies
of the index referencing something else. The previous code did this only
specifically for a constraint, but we also need to do this for partitioned
indexes. So instead write a generic function that does it for all
dependencies. Author: Michael Paquier <michael(at)paquier(dot)xyz> Author: Peter
Eisentraut <peter(dot)eisentraut(at)2ndquadrant(dot)com> Discussion:
https://www.postgresql.org/message-id/flat/DF4PR8401MB11964EDB77C860078C343BEBEE5A0%40DF4PR8401MB1196.NAMPRD84.PROD.OUTLOOK.COM#154df1fedb735190a773481765f7b874
https://git.postgresql.org/pg/commitdiff/ef6f30fe77af69a8c775cca82bf993b10c9889ee

Noah Misch pushed:

- Avoid "could not reattach" by providing space for concurrent allocation. We've
long had reports of intermittent "could not reattach to shared memory" errors
on Windows. Buildfarm member dory fails that way when
PGSharedMemoryReAttach() execution overlaps with creation of a thread for the
process's "default thread pool". Fix that by providing a second region to
receive asynchronous allocations that would otherwise intrude into
UsedShmemSegAddr. In pgwin32_ReserveSharedMemoryRegion(), stop trying to free
reservations landing at incorrect addresses; the caller's next step has been
to terminate the affected process. Back-patch to 9.4 (all supported
versions). Reviewed by Tom Lane. He also did much of the prerequisite
research; see commit bcbf2346d69f6006f126044864dd9383d50d87b4. Discussion:
https://postgr.es/m/20190402135442.GA1173872@rfd.leadboat.com
https://git.postgresql.org/pg/commitdiff/617dc6d299c957e2784320382b3277ede01d9c63

- Define WIN32_STACK_RLIMIT throughout win32 and cygwin builds. The MSVC build
system already did this, and commit 617dc6d299c957e2784320382b3277ede01d9c63
used it in a second file. Back-patch to 9.4, like that commit. Discussion:
https://postgr.es/m/CAA8=A7_1SWc3+3Z=-utQrQFOtrj_DeohRVt7diA2tZozxsyUOQ@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/ba3fb5d4fb9227719759740b6d7771e719c3318b

- Consistently test for in-use shared memory. postmaster startup scrutinizes any
shared memory segment recorded in postmaster.pid, exiting if that segment
matches the current data directory and has an attached process. When the
postmaster.pid file was missing, a starting postmaster used weaker checks.
Change to use the same checks in both scenarios. This increases the chance of
a startup failure, in lieu of data corruption, if the DBA does "kill -9 `head
-n1 postmaster.pid` && rm postmaster.pid && pg_ctl -w start". A postmaster
will no longer stop if shmat() of an old segment fails with EACCES. A
postmaster will no longer recycle segments pertaining to other data
directories. That's good for production, but it's bad for integration tests
that crash a postmaster and immediately delete its data directory. Such a test
now leaks a segment indefinitely. No "make check-world" test does that.
win32_shmem.c already avoided all these problems. In 9.6 and later, enhance
PostgresNode to facilitate testing. Back-patch to 9.4 (all supported
versions). Reviewed (in earlier versions) by Daniel Gustafsson and Kyotaro
HORIGUCHI. Discussion:
https://postgr.es/m/20190408064141.GA2016666@rfd.leadboat.com
https://git.postgresql.org/pg/commitdiff/c098509927f9a49ebceb301a2cb6a477ecd4ac3c

- When Perl "kill(9, ...)" fails, try "pg_ctl kill". Per buildfarm member
jacana, the former fails under msys Perl 5.8.8. Back-patch to 9.6, like the
code in question. Discussion:
https://postgr.es/m/GrdLgAdUK9FdyZg8VIcTDKVOkys122ZINEb3CjjoySfGj2KyPiMKTh1zqtRp0TAD7FJ27G-OBB3eplxIB5GhcQH5o8zzGZfp0MuJaXJxVxk=@yesql.se
https://git.postgresql.org/pg/commitdiff/947a35014fdc2ec74cbf06c7dbac6eea6fae90c6

- MSYS: Translate REGRESS_SHLIB to a Windows file name. Per buildfarm member
jacana. Back-patch to v11; earlier branches skip the affected test under
msys. Discussion:
https://postgr.es/m/GrdLgAdUK9FdyZg8VIcTDKVOkys122ZINEb3CjjoySfGj2KyPiMKTh1zqtRp0TAD7FJ27G-OBB3eplxIB5GhcQH5o8zzGZfp0MuJaXJxVxk=@yesql.se
https://git.postgresql.org/pg/commitdiff/9daefff1226087602d25837b6b30154b3a916ea8

Heikki Linnakangas pushed:

- Fix example in comment. Author: Adrien Nayrat
https://git.postgresql.org/pg/commitdiff/16954e22e2a881a4a5ead82f364635bfa63cc9df

Álvaro Herrera pushed:

- Fix memory leak in pgbench. Commit 25ee70511ec2 introduced a memory leak in
pgbench: some PGresult structs were not being freed during error bailout,
because we're now doing more PQgetResult() calls than previously. Since
there's more cleanup code outside the discard_response() routine than in it,
refactor the cleanup code, removing the routine. This has little effect
currently, since we abandon processing after hitting errors, but if we ever
get further pgbench features (such as testing for serializable transactions),
it'll matter. Per Coverity. Reviewed-by: Michaël Paquier
https://git.postgresql.org/pg/commitdiff/fe0e0b4fc7f0cdc2333bda08b199422a1e77a551

- Fix typo.
https://git.postgresql.org/pg/commitdiff/4dba0f6dc407f965924d6c1007ac1bb5cc209dde

- Fix declaration after statement. This style is frowned upon. I inadvertently
introduced one in commit fe0e0b4fc7f0. (My compiler does not complain about
it, even though -Wdeclaration-after-statement is specified. Weird.) Author:
Masahiko Sawada
https://git.postgresql.org/pg/commitdiff/65d857d92c418d732e3531a3761a32f2e352cb35

Thomas Munro pushed:

- Fix typos.
https://git.postgresql.org/pg/commitdiff/255044889d419354b46a2bf8907b83507d695af5

- Improve comment in sync.h. Per off-list complaint from Andres Freund.
https://git.postgresql.org/pg/commitdiff/d614aae02e8f878438716b7fd2642b8240b3f2b3

- Fix GetNewTransactionId()'s interaction with xidVacLimit. Commit ad308058
switched to returning a FullTransactionId, but failed to load the potentially
updated value in the case where xidVacLimit is reached and we release and
reacquire the lock. Repair, closing bug #15727. While reviewing that commit,
also fix the size computation used by EstimateTransactionStateSize() and
switch to the mul_size() macro traditionally used in such expressions.
Author: Thomas Munro Reported-by: Roman Zharkov Discussion:
https://postgr.es/m/15727-0be246e7d852d229%40postgresql.org
https://git.postgresql.org/pg/commitdiff/f7feb020c3d8d5aff24204af28359b99ee65bf8f

Amit Kapila pushed:

- Avoid counting transaction stats for parallel worker cooperating.transaction.
The transaction that is initiated by the parallel worker to cooperate with the
actual transaction started by the main backend to complete the query execution
should not be counted as a separate transaction. The other internal
transactions started and committed by the parallel worker are still counted as
separate transactions as we that is what we do in other places like
autovacuum. This will partially fix the bloat in transaction stats due to
additional transactions performed by parallel workers. For a complete fix, we
need to decide how we want to show all the transactions that are started
internally for various operations and that is a matter of separate patch.
Reported-by: Haribabu Kommi Author: Haribabu Kommi Reviewed-by: Amit Kapila,
Jamison Kirk and Rahila Syed Backpatch-through: 9.6 Discussion:
https://postgr.es/m/CAJrrPGc9=jKXuScvNyQ+VNhO0FZk7LLAShAJRyZjnedd2D61EQ@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/bdf35744bdf70208fc4d0f4b76f7d4bce3cf326b

Bruce Momjian pushed:

- doc: adjust libpq wording to be neither/nor. Reported-by: postgresql(at)cohi(dot)at
Discussion:
https://postgr.es/m/155419437926.737.10876947446993402227@wrigleys.postgresql.org
Backpatch-through: 9.4
https://git.postgresql.org/pg/commitdiff/d4888a3f766a7c02a9330c658c76c60e1ab8cc3e

Michael Meskes pushed:

- Fix off-by-one check that can lead to a memory overflow in ecpg. Patch by Liu
Huailing <liuhuailing(at)cn(dot)fujitsu(dot)com>
https://git.postgresql.org/pg/commitdiff/ed16ba3248d486ec8b2fdfc75e442620d675b617

Magnus Hagander pushed:

- Show shared object statistics in pg_stat_database. This adds a row to the
pg_stat_database view with datoid 0 and datname NULL for those objects that
are not in a database. This was added particularly for checksums, but we were
already tracking more satistics for these objects, just not returning it.
Also add a checksum_last_failure column that holds the timestamptz of the last
checksum failure that occurred in a database (or in a non-dataabase file), if
any. Author: Julien Rouhaud <rjuju123(at)gmail(dot)com>
https://git.postgresql.org/pg/commitdiff/77bd49adba4711b4497e7e39a5ec3a9812cbd52a

== Pending Patches ==

Justin Pryzby sent in another revision of a patch to clean up the docs for
log_statement_sample_rate.

Jie Zhang sent in a patch to update zh_CN.po.

Fabien COELHO sent in a patch for pgbench to add an option to show the actual
builtin script code.

Fabien COELHO sent in a patch for pgbench to implement a strict TPC-B-conformant
benchmark.

Fabien COELHO sent in a patch for pgbench to add \aset, which stores all result
sets.

Yuzuko Hosoya and Kyotaro HORIGUCHI traded patches to fix an issue with
inconsistent partition pruning.

Heikki Linnakangas sent in three more revisions of a patch to fix confusion
about the different kinds of slots in IndexOnlyScans, add a toy table AM
implementation to play with, add a test for bug with index reorder queue slot
type confusion, and fix confusion about the type of slot used for Index Scan's
projections.

Haribabu Kommi sent in another revision of a patch to MSVC Build support with
Visual Studio 2019.

Iwata Aya sent in another revision of a patch to make one-line libpq trace
output for logging.

Konstantin Knizhnik sent in another revision of a patch to implement
autoprepare.

Amit Langote and Jesper Pedersen traded patches to fix an infelicity between
hash partitons and UPDATE.

Anastasia Lubennikova and Andrey V. Lepikhov traded patches to fix a
gist_optimized_wal_intarray_test bug.

Robert Treat sent in a patch to fix a mistake in the documentation around target
column type restrictions in logical replication.

Amit Khandekar sent in another revision of a patch to implement minimal logical
decoding on standbys.

Ashwin Agrawal sent in a patch to implement compressed in-core columnar storage.

Álvaro Herrera and David Rowley traded patches to fix an infelicity between
pg_dump and partition tablespaces.

Peifeng Qiu sent in another revision of a patch to speed up builds on Windows by
generating symbol definitions batch-wise.

Etsuro Fujita sent in another revision of a patch to fix a bug in tuple-routing
for partitions some of which are foreign tables.

Fabien COELHO sent in another revision of a patch for pgbench to add minimal
stats on initialization.

Justin Pryzby sent in another revision of a patch to clean up, remove, and
update references to OID column.

Konstantin Knizhnik sent in a patch to implement a ptrack utility, which detects
updated blocks at the file level. This would be infrastructure for, among other
things, a future incremental backup.

Thomas Munro sent in a PoC patch to implement ExecScrollSlot() for hash join
prefetch.

Haribabu Kommi sent in another revision of a patch to libpq to support
connecting to a standby server as a higher priority.

Takeshi Ideriha sent in another revision of a patch to prevent syscache from
bloating with negative cache entries.

David Rowley and Justin Pryzby traded patches to reinstate the warnings
regarding large partition heirarchies, and document some features of declarative
partitioning for which there are no plans to port to legacy inheritance.

Jeevan Chalke and Peter Eisentraut traded patches to fix an index optimization
for LIKE on bytea.

Antonin Houska sent in a patch to DRY up reading of XLOG pages.

David Rowley sent in a patch to warn about the consequences of calling
pg_stat_reset().

Juan José Santamaría Flecha to ensure that the TM format for to_char() respects
encoding.

Fabien COELHO sent in a patch to implement a new SHOW_ALL_RESULTS option for
psql, which shows all results of a combined query (\;) instead of only the last
one.

Laurenz Albe sent in a patch to ensure that identity columns own only a single
sequence.