Re: The dangers of streaming across versions of glibc: A cautionary tale

On Wed, Aug 6, 2014 at 09:24:17PM +0000, Matthew Kelly wrote:
> The following is a real critical problem that we ran into here at TripAdvisor,
> but have yet figured out a clear way to mitigate.
>
> TL;DR:
> Streaming replicas—and by extension, base backups—can become dangerously broken
> when the source and target machines run slightly different versions of glibc.
> Particularly, differences in strcoll and strcoll_l leave "corrupt" indexes on
> the slave. These indexes are sorted out of order with respect to the strcoll
> running on the slave. Because postgres is unaware of the discrepancy is uses
> these "corrupt" indexes to perform merge joins; merges rely heavily on the
> assumption that the indexes are sorted and this causes all the results of the
> join past the first poison pill entry to not be returned. Additionally, if the
> slave becomes master, the "corrupt" indexes will in cases be unable to enforce
> uniqueness, but quietly allow duplicate values.

No surprise; I have been expecting to hear about such breakage, and am
surprised we hear about it so rarely. We really have no way of testing
for breakage either. :-(

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ Everyone has their own god. +