Re: When to switch to Postgres 8.0?

Christopher Browne <cbbrowne(at)acm(dot)org> writes:
> The world rejoiced as jao(at)geophile(dot)com wrote:
>> Suppose 8.0 is released in December or January. Is it a sane thing
>> to put 8.0 in a shipping product within the first two months of its
>> release?
>> ...
>> Suppose I don't use any new features. The question is then whether
>> I'm likely to be affected by a regression.
>> ...
>> We'll be doing extensive testing between code freeze and our ship
>> date.

> Certainly a good move...

My advice would be "trust, but verify" ... that is, use 8.0 and test it
as hard as you can. You will be contributing something back to the
community by so testing, and you will probably end up with a better
product than if you took the "conservative" route of using 7.4.

While it's certainly true that 8.0 includes some major changes, so did
the last several releases; and nonetheless my feeling is that each
release has been higher quality on average than its predecessor. We
are on average introducing fewer (post-release) bugs than we fix.

To try to provide some hard evidence, I dug through the recent CVS
change logs. Attached are all the post-release 7.4 bug fixes that
did not either (a) represent a bug that also existed in 7.3, or
(b) represent a fix in a feature that did not exist in 7.3. Thus
this list would be a fair gauge of your risk had you updated to 7.4
on Day Zero. (Disclaimer: I didn't try to evaluate some ecpg and
tsearch bugs that were fixed in this interval; maybe they were in
7.3 too, or maybe not.)

There are twenty bugs in this list. It is worth pointing out that in
the same interval there were also twenty nontrivial patches applied to
both 7.3 and 7.4 branches (ie, bugs older than 7.4), as well as several
bugs that were fixed in 7.4 but not in 7.3 because the change seemed too
invasive/risky for the older branch. Moreover, only one of the
new-in-7.4 bugs poses any risk of data corruption, whereas three of the
7.4-and-7.3 bugs did.

I'm too lazy to do a comparable analysis of the prior releases, but my
gut feeling is that every release since 6.5 or so has been as good or
better than its predecessor, even on day zero. Yeah, there are
localized regressions, but there are also lots of bugs fixed. The way
to defend yourself against being bit by a localized regression is to get
involved with testing your applications against the beta releases, as
early and aggressively as possible.

The only aspect of 8.0 that I don't care to play Pollyanna about is the
native Windows port. That's a whole lot of new code with no track
record and not that much field testing yet. But as long as you are
thinking of running on a non-Microsoft OS, I'd say "go for it".

regards, tom lane

2004-08-17 19:15 tgl

* src/backend/access/nbtree/: nbtinsert.c (REL7_4_STABLE),
nbtinsert.c: Fix bug introduced into _bt_getstackbuf() on
2003-Feb-21: the initial value of 'start' could be past the end of
the page, if the page was split by some concurrent inserting
process since we visited it. In this situation the code could look
at bogus entries and possibly find a match (since after all those
entries still contain what they had before the split). This would
lead to 'specified item offset is too large' followed by 'PANIC:
failed to add item to the page', as reported by Joe Conway for
scenarios involving heavy concurrent insertion activity.

7.4.3

2004-05-10 22:21 tgl

* src/backend/optimizer/plan/planner.c (REL7_4_STABLE): Repair
recalculation failure for nested sub-SELECTs, per bug report from
Didier Moens. Bug is new in 7.4, and was caused by not updating
everyplace I should've when replacing locParam markers by allParam.

2004-03-12 19:54 tgl

* src/backend/executor/: nodeAgg.c (REL7_4_STABLE), nodeAgg.c:
Repair memory leakage introduced into the non-hashed aggregate case
by 7.4 rewrite for hashed aggregate support. If the transition
data type is pass-by-reference, the transValue must be pfreed when
starting a new group boundary, else we have a one-value-per-group
leakage. Thanks to Rae Steining for providing a reproducible test
case.

7.4.2

2004-03-02 13:56 tgl

* src/: backend/executor/execAmi.c, backend/executor/execMain.c,
include/executor/executor.h (REL7_4_STABLE),
backend/executor/execAmi.c, backend/executor/execMain.c,
include/executor/executor.h: Junkfilter logic to force a projection
step during SELECT INTO was too simplistic; it recognized SELECT *
FROM but not SELECT * FROM LIMIT. Per bug report from Jeff Bohmer.

2004-02-29 12:36 tgl

* src/backend/optimizer/plan/: createplan.c (REL7_4_STABLE),
createplan.c: make_sort_from_pathkeys()'s method for choosing which
of several equivalent sort expressions to use was broken: you can't
just look at the relation membership, you have to actually grovel
over the individual Vars in each expression. I think this did work
when it was written, but it was broken by subsequent optimizations
that made join relations not propagate every single input variable
upward. Must find the Var that got propagated, not choose one at
random. Per bug report from Daniel O'Neill.

2004-02-24 17:59 tgl

* src/include/catalog/: pg_type.h (REL7_4_STABLE), pg_type.h:
anyarray really needs to be declared with typalign = 'd', so that
entries in pg_statistic are correctly aligned if they contain
values that require double alignment. Too bad we cannot force
initdb for this in 7.4 branch.

2004-02-15 10:40 meskes

* src/interfaces/ecpg/preproc/preproc.y (REL7_4_STABLE): - Allowed
some C keywords to be used as SQL column names. This used to work
in 7.3.*

2004-02-13 17:26 tgl

* src/backend/optimizer/plan/: planner.c (REL7_4_STABLE),
planner.c: Repair optimization bug I introduced in a moment of
brain fade back in Nov 2002: when constant-expression
simplification removes all the aggregate function calls from a
query, that doesn't mean we can act as though there never were any
aggregates. Per bug report from Gabor Szucs.

2004-01-28 02:46 tgl

* src/backend/parser/: parse_agg.c (REL7_4_STABLE), parse_agg.c:
Fix oversight in check_ungrouped_columns optimization that avoids
unnecessary checks for complex grouping expressions: we cannot
check whether the expressions are simple Vars until after we apply
flatten_join_alias_vars, because in the case of FULL JOIN that
routine can introduce non-Var expressions. Per example from Joel
Knight.

2004-01-23 19:37 tgl

* src/backend/optimizer/path/: joinrels.c (REL7_4_STABLE),
joinrels.c: Repair planner failure for cases involving Cartesian
products inside IN (sub-SELECT) constructs. We must force a
clauseless join of the sub-select member relations, but it wasn't
happening because the code thought it would be able to use the join
clause arising from the IN.

2004-01-21 21:23 tgl

* src/: backend/executor/execMain.c, backend/executor/execScan.c,
backend/executor/execUtils.c, backend/executor/nodeAppend.c,
include/executor/executor.h, include/nodes/execnodes.h
(REL7_4_STABLE), backend/executor/execMain.c,
backend/executor/execScan.c, backend/executor/execUtils.c,
backend/executor/nodeAppend.c, include/executor/executor.h,
include/nodes/execnodes.h: Fix oversight in optimization that
avoids an unnecessary projection step when scanning a table that we
need all the columns from. In case of SELECT INTO, we have to
check that the hasoids flag matches the desired output type, too.
Per report from Mike Mascari.

2004-01-17 21:15 tgl

* src/backend/commands/: copy.c (REL7_4_STABLE), copy.c: Don't use
%s-with-precision format spec to truncate data being displayed in a
COPY error message. It seems that glibc gets indigestion if it is
asked to truncate strings that contain invalid UTF-8 encoding
sequences. vsnprintf will return -1 in such cases, leading to
looping and eventual memory overflow in elog.c. Instead use our
own, more robust pg_mbcliplen routine. I believe this problem
accounts for several recent reports of unexpected 'out of memory'
errors during COPY IN.

2004-01-17 19:31 tgl

* src/backend/optimizer/plan/createplan.c (REL7_4_STABLE): Repair
faulty plan generation in cases where we choose to implement an IN
clause by mergejoin, and a type coercion is needed just above the
subplan. A more extensive patch will follow in HEAD.

2003-12-30 15:05 tgl

* src/backend/executor/: nodeIndexscan.c (REL7_4_STABLE),
nodeIndexscan.c: Avoid running out of memory during hash_create, by
not passing a number-of-buckets that exceeds the size we actually
plan to allow the hash table to grow to. Per trouble report from
Sean Shanny.

2003-12-28 12:43 tgl

* src/interfaces/libpq/: fe-protocol3.c (REL7_4_STABLE),
fe-protocol3.c: Fix sanity-check code that mistakenly assumed error
and notice messages could never exceed 30K. Per report from
Andreas Pflug.

7.4.1

2003-12-17 14:49 tgl

* src/backend/parser/: parse_coerce.c (REL7_4_STABLE),
parse_coerce.c: Reorder tests in parse_coerce so that
ANY/ANYELEMENT/ANYARRAY coercion does not affect UNKNOWN-type
literals or Params. This fixes the recent complaint about
count('x') being broken, and improves consistency in a few other
respects too.

2003-12-17 12:07 tgl

* src/backend/optimizer/path/: allpaths.c, joinrels.c
(REL7_4_STABLE), allpaths.c, joinrels.c: Repair planner failure
when there are multiple IN clauses, each with a join in its
subselect. In this situation we *must* build a bushy plan because
there are no valid left-sided or right-sided join trees.
Accordingly, hoary sanity check needs an update. Per report from
Alessandro Depase.

2003-12-08 13:19 tgl

* src/backend/optimizer/util/: relnode.c (REL7_4_STABLE),
relnode.c: Whole-row references were broken for subqueries and
functions, because attr_needed/attr_widths optimization failed to
allow for Vars with attno zero in this case. Per report from
Tatsuo Ishii.

2003-12-03 12:45 tgl

* src/: backend/optimizer/path/costsize.c,
backend/optimizer/path/pathkeys.c, backend/utils/cache/lsyscache.c,
include/utils/lsyscache.h (REL7_4_STABLE),
backend/optimizer/path/costsize.c,
backend/optimizer/path/pathkeys.c, backend/utils/cache/lsyscache.c,
include/utils/lsyscache.h: Planner failed to be smart about
binary-compatible expressions in pathkeys and hash bucket-size
estimation. Issue has been there awhile but is more critical in
7.4 because it affects varchar columns. Per report from Greg
Stark.

2003-11-25 14:17 tgl

* src/backend/: executor/nodeHashjoin.c,
optimizer/plan/createplan.c (REL7_4_STABLE): Band-aid solution for
problems with SubPlans used in hash join clauses, per report from
Andrew Holm-Hansen. The difficulty arises from the fact that the
planner allowed a Hash node's hashkeys to share substructure with
the parent HashJoin node's hashclauses, plus some rather bizarre
choices about who initializes what during executor startup. A
cleaner but more invasive solution is to not store hashkeys
separately in the plan tree at all, but let the HashJoin node
deconstruct hashclauses during executor startup. I plan to fix it
that way in HEAD.