Slony-I 1.2.0 Released

As per the release notes:

Release 1.2 of Slony-I has numerous improvements over previous
versions.

Many of them should represent near-invisible enhancements that improve
how Slony-I manages replication configuration. These include:

- A major revision of memory management to limit memory usage by slon
daemons. In earlier versions, slon would try to load 100 tuples
into memory at a time. If you have tables with Very Large bytea or
character varying columns, this could lead to loading 100 x 50MB
into memory, twice, doing terrible things to memory consumption.

slon now loads tuples in, directly, only if they are less than a
certain size, and handles those 50MB tuples individually.

- Log switching: periodically, Slony-I will switch between storing
replication data in sl_log_1 and sl_log_2, which allows regularly
cleaning these tables out via TRUNCATE and which prevents some data
loss problems relating to cases where different sets originate on
different nodes, when transaction IDs roll over the 32 bit limits.

- pg_listener is now used dramatically less than it used to be, which
diminishes the number of dead tuples you will find in this table.

This, along with log switching, should improve Slony-I behaviour on
systems where long-running transactions are common. Older versions
of Slony-I could suffer quite badly on systems that get hit by
long-running transactions.

- DDL scripts are broken into individual statements

This is more a bug fix than an enhancement; it now permits DDL
scripts to create new tables and columns, and reference them later
in the script.

In the past, DDL was submitted to the postmaster as a single query,
which meant that all of them had to reference the state of
pg_catalog as it was before the DDL ran. So you could add as many
columns to tables as you liked; you could NOT, then, reference those
columns, because the query processor would discover that the new
column didn't exist as at "before the DDL ran."

There is now a statement parser which splits scripts into individual
SQL statements and passes them to the database back end
individually.

- Slony-I tables are now marked "WITHOUT OIDS" so that they do not
consume OIDS.

It's only particularly important for sl_log_1/sl_log_2/sl_seqlog,
but the change has been applied to all the tables Slony-I uses.
UPGRADE FUNCTIONS will remove OIDs from Slony-I tables in existing
schemas, too.

- When possible (based on log switching functionality), partial
indexes on sl_log_1 and sl_log_2 are created on a per-origin-node
basis. This provides the performance boost of having an easily
recognisable index, but without the risk of having XIDs from
different nodes mixed together in one index, where roll-over could
Cause Problems...

These features are generally configurable, but the defaults ought to
allow improved behaviour for all but the most "Extreme Uses."

There are also numerous enhancements that are more directly visible:

- Windows support

A group of developers has contributed changes to allow running
Slony-I as a Windows service.

- PostgreSQL Version Compatibility Changes

Slony-I developers have been following the 8.2 release expected in
fall 2006; Slony-I 1.2 can be expected to work with it without
problems.

Compatibility with version 7.3 has been dropped; version 1.2 notably
makes use of some array functionality that didn't exist in 7.3. If
you are on 7.3, you're on a REALLY OLD version of PostgreSQL, and
really ought to upgrade. You may use Slony-I 1.1.5 to get to a more
modern version, and then upgrade to 1.2 as needed.

- Process structure is revised considerably; in 1.1.0, an attempt was
made to create an internal 'watchdog' that would restart threads
that fell over for whatever reason.

In 1.2, this is massively revised. The "main" slon thread should,
as of 1.2, never fall down, which means that there should no longer
be a need for watchdog processes to watch carefully.

This makes slon very much less fragile than it was in earlier
versions. This may revise how you want to start it up... In the
past, slon processes tended to fall over easily, mandating having
some form of "watchdog." The new behaviour points more towards
"init" / "rc.d"-like handling, where, upon boot-up, one "rc.d" script
might start up PostgreSQL, another one starts pgpool, and a third
(which must be invoked after PostgreSQL is started) starts up a
slon.

- Subscribe set aggressively locks tables on the subscriber to avoid
failures

This may become a configurable option so that sites where they
Really Know What They're Doing and are sure that they won't deadlock
themselves can leave the tables more open.

But for most users, it's really preferable to lock the tables down
on the subscriber so that you don't get 18 hours into subscribing an
18GB replication set, hit a deadlock, and have to start all over.
(The slon will automatically try again; the irritation is that you
may have been depending on that getting done by Monday morning...)

- As has been the case for fairly much each release that has been
made, the documentation has been significantly extended. The "admin
guide" has been augmented and cleaned up.

Notable additions include a listing of "Best Practices" (due in
great part to discoveries by the oft-unsung heroes of Afilias' Data
Services department) and a fairly comprehensive listing of log
messages you may expect to see in your Slony-I logs.

- A lot of fixes to the build environment

One noticeable upshot of this should be that (on Unix-like systems;
Windows(tm) appears to differ) you oughtn't need to specify a whole
lot of --with-pg[options] on the ./configure command line. You should
be able to get most of what's needed simply by specifying the path to
pg_config via --with-pgconfigdir.

- slon "lag interval" option

You can tell a node to lag behind by a particular interval of time.

slon -l "4 hours" [and probably other options :-)]

will cause the slon to ignore events until they reach the age
corresponding to the requested interval. That way, you can hold a
node behind by 4 hours.

This is a potential "foot gun" as there are some cases (MOVE SET,
FAILOVER) where events have to be coordinated across all nodes with
near-simultaneity.

- slon "stop after event" option

You can tell a slon to terminate itself as soon as it receives a
certain event from a certain node.

Just as with "lag interval," this could be a bit of a foot gun...

- slon "run program after each log shipped SYNC" option

You can specify a program to run each time a SYNC event
is successfully closed off.

- Bug 1538 - if there is only one node, sl_event, sl_log_1 never get
cleared out

Logic added to cleanupevent() to clear out old sl_event entries if
there is just one node. That then allows the cleanup thread to
clear sl_log_1 etc.

- Bug 1566 - Force all replication to occur in the ISO datestyle.
This ensures that we can apply date/timestamps regardless of the
datestyle they were entered in.

- Force all replication to occur with standards_conforming_strings set
to off. This ensures we can replicate a node running on 8.2 without
extra escape chars showing up in the data.
--
let name="cbbrowne" and tld="cbbrowne.com" in String.concat "@" [name;tld];;
http://cbbrowne.com/info/slony.html
"Bother," said Pooh as he struggled with sendmail.cf.
"It never does quite what I want."
"I wish Christopher Robin were here.".