Make the server reject extraneous data after an SSL or GSS encryption handshake (Tom Lane)

A man-in-the-middle with the ability to inject data into the TCP connection could stuff some cleartext data into the start of a supposedly encryption-protected database session. This could be abused to send faked SQL commands to the server, although that would only work if the server did not demand any authentication data. (However, a server relying on SSL certificate authentication might well not do so.)

The PostgreSQL Project thanks Jacob Champion for reporting this problem. (CVE-2021-23214)

Make libpq reject extraneous data after an SSL or GSS encryption handshake (Tom Lane)

A man-in-the-middle with the ability to inject data into the TCP connection could stuff some cleartext data into the start of a supposedly encryption-protected database session. This could probably be abused to inject faked responses to the client's first few queries, although other details of libpq's behavior make that harder than it sounds. A different line of attack is to exfiltrate the client's password, or other sensitive data that might be sent early in the session. That has been shown to be possible with a server vulnerable to CVE-2021-23214.

The PostgreSQL Project thanks Jacob Champion for reporting this problem. (CVE-2021-23222)

Fix physical replication for cases where the primary crashes after shipping a WAL segment that ends with a partial WAL record (Álvaro Herrera)

If the primary did not survive long enough to finish writing the rest of the incomplete WAL record, then the previous crash-recovery logic had it back up and overwrite WAL starting from the beginning of the incomplete WAL record. This is problematic since standby servers may already have copies of that WAL segment. They will then see an inconsistent next segment, and will not be able to recover without manual intervention. To fix, do not back up over a WAL segment boundary when restarting after a crash. Instead write a new type of WAL record at the start of the next WAL segment, informing readers that the incomplete WAL record will never be finished and must be disregarded.

When applying this update, it's best to update standby servers before the primary, so that they will be ready to handle this new WAL record type if the primary happens to crash.

Fix CREATE INDEX CONCURRENTLY to wait for the latest prepared transactions (Andrey Borodin)

Rows inserted by just-prepared transactions might be omitted from the new index, causing queries relying on the index to miss such rows. The previous fix for this type of problem failed to account for PREPARE TRANSACTION commands that were still in progress when CREATE INDEX CONCURRENTLY checked for them. As before, in installations that have enabled prepared transactions (max_prepared_transactions > 0), it's recommended to reindex any concurrently-built indexes in case this problem occurred when they were built.

Avoid race condition that can cause backends to fail to add entries for new rows to an index being built concurrently (Noah Misch, Andrey Borodin)

While it's apparently rare in the field, this case could potentially affect any index built or reindexed with the CONCURRENTLY option. It is recommended to reindex any such indexes to make sure they are correct.

Fix float4 and float8 hash functions to produce uniform results for NaNs (Tom Lane)

Since PostgreSQL's floating-point types deem all NaNs to be equal, it's important for the hash functions to produce the same hash code for all bit-patterns that are NaNs according to the IEEE 754 standard. This failed to happen before, meaning that hash indexes and hash-based query plans might produce incorrect results for non-canonical NaN values. ('-NaN'::float8 is one way to produce such a value on most machines.) It is advisable to reindex hash indexes on floating-point columns, if there is any possibility that they might contain such values.

Fix REINDEX CONCURRENTLY to preserve operator class parameters that were attached to the target index (Michael Paquier)

Prevent data loss during crash recovery of CREATE TABLESPACE, when wal_level = minimal (Noah Misch)

If the server crashed between CREATE TABLESPACE and the next checkpoint, replay would fully remove the contents of the new tablespace's directory, relying on subsequent WAL replay to restore everything within that directory. This interacts badly with optimizations that skip writing WAL (one example is COPY into a just-created table). Such optimizations are applied only when wal_level is minimal, which is not the default in v10 and later.

Ensure that the relation cache is invalidated for a table being attached to or detached from a partitioned table (Amit Langote, Álvaro Herrera)

This oversight could allow misbehavior of subsequent inserts/updates addressed directly to the partition, but only in currently-existing sessions.

Ensure that the relation cache is invalidated for all partitions of a partitioned table that is being added to or removed from a publication (Hou Zhijie, Vignesh C)

This oversight could lead to improper replication behavior until all currently-existing sessions have exited.

Ensure that the relation cache is invalidated when creating or dropping a FOR ALL TABLES publication (Hou Zhijie, Vignesh C)

This oversight could lead to improper replication behavior until all currently-existing sessions have exited.

Don't discard a cast to the same type with unspecified type modifier (Tom Lane)

For example, if column f1 is of type numeric(18,3), the parser used to simply discard a cast like f1::numeric, on the grounds that it would have no run-time effect. That's true, but the exposed type of the expression should still be considered to be plain numeric, not numeric(18,3). This is important for correctly resolving the type of larger constructs, such as recursive UNIONs.

Fix updates of element fields in arrays of domain over composite (Tom Lane)

A command such as UPDATE tab SET fld[1].subfld = val failed if the array's elements were domains rather than plain composites.

Disallow the combination of FETCH FIRST WITH TIES and FOR UPDATE SKIP LOCKED (David Christensen)

FETCH FIRST WITH TIES necessarily fetches one more row than requested, since it cannot stop until it finds a row that is not a tie. In our current implementation, if FOR UPDATE is used then that row will also get locked even though it is not returned. That results in undesirable behavior if the SKIP LOCKED option is specified. It's difficult to change this without introducing a different set of undesirable behaviors, so for now, forbid the combination.

Disallow creating an ICU collation if the current database's encoding won't support it (Tom Lane)

Previously this was allowed, but then the collation could not be referenced because of the way collation lookup works; you could not use the collation, nor even drop it.

Disallow ALTER INDEX index ALTER COLUMN col SET (options) (Nathan Bossart, Michael Paquier)

While the parser accepted this, it's undocumented and doesn't actually work.

Fix corner-case loss of precision in numeric power() (Dean Rasheed)

The result could be inaccurate when the first argument is very close to 1.

Avoid regular expression errors with capturing parentheses inside {0} (Tom Lane)

Regular expressions like (.){0}...\1 drew “invalid backreference number”. Other regexp engines such as Perl don't complain, though, and for that matter ours doesn't either in some closely related cases. Worse, it could throw an assertion failure instead. Fix it so that no error is thrown and instead the back-reference is silently deemed to never match.

Prevent regular expression back-references from sometimes matching when they shouldn't (Tom Lane)

The regexp engine was careless about clearing match data for capturing parentheses after rejecting a partial match. This could allow a later back-reference to match in places where it should fail for lack of a defined referent.

Fix regular expression performance bug with back-references inside iteration nodes (Tom Lane)

Incorrect back-tracking logic could result in exponential time spent looking for a match. Fortunately the problem is masked in most cases by other optimizations.

Fix incorrect results from AT TIME ZONE applied to a time with time zone value (Tom Lane)

The results were incorrect if the target time zone was specified by a dynamic timezone abbreviation (that is, one that is defined as equivalent to a full time zone name, rather than a fixed UTC offset).

Fix planner error with pulling up subquery expressions into function rangetable entries (Tom Lane)

If a function in FROM laterally references the output of some sub-SELECT earlier in the FROM clause, and we are able to flatten that sub-SELECT into the outer query, the expression(s) copied into the function expression were not fully processed. This could lead to crashes at execution.

Fix mistranslation of PlaceHolderVars to inheritance child relations (Tom Lane)

This error could result in assertion failures, or in mis-planning of queries having partitioned or inherited tables on the nullable side of an outer join.

Avoid using MCV-only statistics to estimate the range of a column (Tom Lane)

There are corner cases in which ANALYZE will build a most-common-values (MCV) list but not a histogram, even though the MCV list does not account for all the observed values. In such cases, keep the planner from using the MCV list alone to estimate the range of column values.

Fix restoration of a Portal's snapshot inside a subtransaction (Bertrand Drouvot)

If a procedure commits or rolls back a transaction, and then its next significant action is inside a new subtransaction, snapshot management went wrong, leading to a dangling pointer and probable crash. A typical example in PL/pgSQL is a COMMIT immediately followed by a BEGIN ... EXCEPTION block that performs a query.

Clean up correctly if a transaction fails after exporting its snapshot (Dilip Kumar)

This oversight would only cause a problem if the same session attempted to export a snapshot again. The most likely scenario for that is creation of a replication slot (followed by rollback) and then creation of another replication slot.

Prevent wraparound of overflowed-subtransaction tracking on standby servers (Kyotaro Horiguchi, Alexander Korotkov)

This oversight could cause significant performance degradation (manifesting as excessive SubtransSLRU traffic) on standby servers.

Ensure that prepared transactions are properly accounted for during promotion of a standby server (Michael Paquier, Andres Freund)

There was a narrow window where a prepared transaction could be omitted from a snapshot taken by a concurrently-running session. If that session then used the snapshot to perform data updates, erroneous results or data corruption could occur.

Disallow LISTEN in background workers (Tom Lane)

There's no infrastructure to support this, so if someone did it, it would only result in preventing cleanup of the NOTIFY queue.

Send NOTIFY signals to other backends during transaction commit, not in the server's idle loop (Artur Zakirov, Tom Lane)

This change allows notifications to be delivered immediately after an intra-procedure COMMIT. It also allows logical replication workers to send notifications.

Refuse to rewind a cursor marked NO SCROLL if it has been held over from a previous transaction due to the WITH HOLD option (Tom Lane)

We have long forbidden fetching backwards from a NO SCROLL cursor, but for historical reasons the prohibition didn't extend to cases in which we rewind the query altogether and then re-fetch forwards. That exception leads to inconsistencies, particularly for held-over cursors which may not have stored all the data necessary to rewind. Disallow rewinding for non-scrollable held-over cursors to block the worst inconsistencies. (v15 will remove the exception altogether.)

Fix possible failure while saving a WITH HOLD cursor at transaction end, if it had already been read to completion (Tom Lane)

Fix detection of a relation that has grown to the maximum allowed length (Tom Lane)

An attempt to extend a table or index past the limit of 2^32-1 blocks was rejected, but not soon enough to prevent inconsistent internal state from being created.

Correctly track the presence of data-modifying CTEs when expanding a DO INSTEAD rule (Greg Nancarrow, Tom Lane)

The previous failure to do this could lead to problems such as unsafely choosing a parallel plan.

Fix incorrect reporting of permissions failures on extended statistics objects (Tomas Vondra)

The code typically produced “cache lookup error” rather than the intended message.

Fix incorrect snapshot handling in parallel workers (Greg Nancarrow)

This oversight could lead to misbehavior in parallel queries if the transaction isolation level is less than REPEATABLE READ.

Fix logical decoding to correctly ignore toast-table changes for transient tables (Bertrand Drouvot)

Logical decoding normally ignores changes in transient tables such as those created during an ALTER TABLE heap rewrite. But that filtering wasn't applied to the associated toast table if any, leading to possible errors when rewriting a table that's being published.

Fix logical decoding's memory usage accounting to handle TOAST data correctly (Bertrand Drouvot)

Ensure that walreceiver processes create all required archive notification files before exiting (Fujii Masao)

If a walreceiver exited exactly at a WAL segment boundary, it failed to make a notification file for the last-received segment, thus delaying archiving of that segment on the standby.

Fix computation of the WAL range to include in a backup manifest when a timeline change is involved (Kyotaro Horiguchi)

Avoid trying to lock the OLD and NEW pseudo-relations in a rule that uses SELECT FOR UPDATE (Masahiko Sawada, Tom Lane)

Fix parser's processing of aggregate FILTER clauses (Tom Lane)

If the FILTER expression is a plain boolean column, the semantic level of the aggregate could be mis-determined, leading to not-per-spec behavior. If the FILTER expression is itself a boolean-returning aggregate, an error should be thrown but was not, likely resulting in a crash at execution.

Ensure that the correct lock level is used when renaming a table (Nathan Bossart, Álvaro Herrera)

For historical reasons, ALTER INDEX ... RENAME can be applied to any sort of relation. The lock level required to rename an index is lower than that required to rename a table or other kind of relation, but the code got this wrong and would use the weaker lock level whenever the command is spelled ALTER INDEX.

Prevent ALTER TYPE/DOMAIN/OPERATOR ... SET from changing extension membership (Tom Lane)

ALTER ... SET executed by an extension script would cause the target object to become a member of the extension if it was not already. In itself this isn't too troubling, since there's little reason for an extension script to touch an object not belonging to the extension. But ALTER TYPE SET will recurse to dependent domains, thus causing them to also become extension members. This causes unwanted side-effects from extension upgrade scripts that use that command to adjust the properties of a base type belonging to the extension. Fix by redefining these ALTER cases to never change extension membership.

Avoid trying to clean up LLVM state after an error within LLVM (Andres Freund, Justin Pryzby)

This prevents a likely crash during backend exit after a fatal LLVM error.

Avoid null-pointer-dereference crash when dropping a role that owns objects being dropped concurrently (Álvaro Herrera)

Prevent “snapshot reference leak” warning when lo_export() or a related function fails (Heikki Linnakangas)

Ensure that scans of SP-GiST indexes are counted in the statistics views (Tom Lane)

Incrementing the number-of-index-scans counter was overlooked in the SP-GiST code, although per-tuple counters were advanced correctly.

Fix inefficient code generation for CoerceToDomain expression nodes (Ranier Vilela)

Recalculate relevant wait intervals if recovery_min_apply_delay is changed during recovery (Soumyadeep Chakraborty, Ashwin Agrawal)

Fix infinite loop if a simplehash.h hash table reaches 2^32 elements (Yura Sokolov)

It seems unlikely that this bug has been hit in practice, as it would require work_mem settings of hundreds of gigabytes for existing uses of simplehash.h.

Avoid O(N^2) behavior in some list-manipulation operations (Nathan Bossart, Tom Lane)

These changes fix slow processing in several scenarios, including: when a standby replays a transaction that held many exclusive locks on the primary; when many files are due to be unlinked after a checkpoint; when hash aggregation involves many batches; and when pg_trgm extracts indexable conditions from a complex regular expression. Only the first of these scenarios has actually been reported from the field, but they all seem like plausible consequences of inefficient list deletions.

Reduce memory consumption during calculation of extended statistics (Justin Pryzby, Tomas Vondra)

Add more defensive checks around B-tree posting list splits (Peter Geoghegan)

This change should help detect index corruption involving duplicate table TIDs.

Disallow setting huge_pages to on when shared_memory_type is sysv (Thomas Munro)

Previously, this setting was accepted, but it did nothing for lack of any implementation.

Fix missing libpq functions on AIX (Tony Reix)

Code reorganization led to the following documented functions not being exported from libpq on AIX: pg_encoding_to_char(), pg_utf_mblen(), pg_char_to_encoding(), pg_valid_server_encoding(), and pg_valid_server_encoding_id(). Restore them to visibility.

Fix ecpg to recover correctly after malloc() failure while establishing a connection (Michael Paquier)

Fix misevaluation of stable functions called in the arguments of a PL/pgSQL CALL statement (Tom Lane)

They were being called with an out-of-date snapshot, so that they would not see any database changes made since the start of the session's top-level command.

Allow EXIT out of the outermost block in a PL/pgSQL routine (Tom Lane)

If the routine does not require an explicit RETURN, this usage should be valid, but it was rejected.

Remove pg_ctl's hard-coded limits on the total length of generated commands (Phil Krylov)

For example, this removes a restriction on how many command-line options can be passed through to the postmaster. Individual path names that pg_ctl deals with, such as the postmaster executable's name or the data directory name, are still limited to MAXPGPATH bytes in most cases.

Fix pg_dump to dump non-global default privileges correctly (Neil Chen, Masahiko Sawada)

If a global (unrestricted) ALTER DEFAULT PRIVILEGES command revoked some present-by-default privilege, for example EXECUTE for functions, and then a restricted ALTER DEFAULT PRIVILEGES command granted that privilege again for a selected role or schema, pg_dump failed to dump the restricted privilege grant correctly.

Make pg_dump acquire shared lock on partitioned tables that are to be dumped (Tom Lane)

This oversight was usually pretty harmless, since once pg_dump has locked any of the leaf partitions, that would suffice to prevent significant DDL on the partitioned table itself. However problems could ensue when dumping a childless partitioned table, since no relevant lock would be held.

Improve pg_dump's performance by avoiding making per-table queries for RLS policies, and by avoiding repetitive calls to format_type() (Tom Lane)

These changes provide only marginal improvement when dumping from a local server, but a dump from a remote server can benefit substantially due to fewer network round-trips.

Fix crash in pg_dump when attempting to dump trigger definitions from a pre-8.3 server (Tom Lane)

Fix incorrect filename in pg_restore's error message about an invalid large object TOC file (Daniel Gustafsson)

Ensure that pgbench exits with non-zero status after a socket-level failure (Yugo Nagata, Fabien Coelho)

The desired behavior is to finish out the run but then exit with status 2. Also, fix the reporting of such errors.

Fix failure of contrib/btree_gin indexes on "char" (not char(n)) columns, when an indexscan using the < or <= operator is performed (Tom Lane)

Such an indexscan failed to return all the entries it should.

Change contrib/pg_stat_statements to read its “query texts” file in units of at most 1GB (Tom Lane)

Such large query text files are very unusual, but if they do occur, the previous coding would fail on Windows 64 (which rejects individual read requests of more than 2GB).

Fix null-pointer crash when contrib/postgres_fdw tries to report a data conversion error (Tom Lane)

Add spinlock support for the RISC-V architecture (Marek Szuba)

This is essential for reasonable performance on that platform.

Support OpenSSL 3.0.0 (Peter Eisentraut, Daniel Gustafsson, Michael Paquier)

Set correct type identifier on OpenSSL BIO (I/O abstraction) objects created by PostgreSQL (Itamar Gafni)

This oversight probably only matters for code that is doing tasks like auditing the OpenSSL installation. But it's nominally a violation of the OpenSSL API, so fix it.

Fix our pkg-config files to again support static linking of libpq (Peter Eisentraut)

Make pg_regexec() robust against an out-of-range search_start parameter (Tom Lane)

Return REG_NOMATCH, instead of possibly crashing, when search_start is past the end of the string. This case is probably unreachable within core PostgreSQL, but extensions might be more careless about the parameter value.

Ensure that GetSharedSecurityLabel() can be used in a newly-started session that has not yet built its critical relation cache entries (Jeff Davis)

Use the CLDR project's data to map Windows time zone names to IANA time zones (Tom Lane)

When running on Windows, initdb attempts to set the new cluster's timezone parameter to the IANA time zone matching the system's prevailing time zone. We were using a mapping table that we'd generated years ago and updated only fitfully; unsurprisingly, it contained a number of errors as well as omissions of recently-added zones. It turns out that CLDR has been tracking the most appropriate mappings, so start using their data. This change will not affect any existing installation, only newly-initialized clusters.

Update time zone data files to tzdata release 2021e for DST law changes in Fiji, Jordan, Palestine, and Samoa, plus historical corrections for Barbados, Cook Islands, Guyana, Niue, Portugal, and Tonga.

Also, the Pacific/Enderbury zone has been renamed to Pacific/Kanton. Also, the following zones have been merged into nearby, more-populous zones whose clocks have agreed with them since 1970: Africa/Accra, America/Atikokan, America/Blanc-Sablon, America/Creston, America/Curacao, America/Nassau, America/Port_of_Spain, Antarctica/DumontDUrville, and Antarctica/Syowa. In all these cases, the previous zone name remains as an alias.