Re: [PATCH 10/16] Introduce the concept that wal has a 'origin' node

On Tuesday, June 19, 2012 12:47:54 AM Christopher Browne wrote:
> On Mon, Jun 18, 2012 at 11:50 AM, Andres Freund <andres(at)2ndquadrant(dot)com>
wrote:
> > Hi Simon,
> >
> > On Monday, June 18, 2012 05:35:40 PM Simon Riggs wrote:
> >> On 13 June 2012 19:28, Andres Freund <andres(at)2ndquadrant(dot)com> wrote:
> >> > This adds a new configuration parameter multimaster_node_id which
> >> > determines the id used for wal originating in one cluster.
> >>
> >> Looks good and it seems this aspect at least is commitable in this CF.
> >
> > I think we need to agree on the parameter name. It currently is
> > 'multimaster_node_id'. In the discussion with Steve we got to
> > "replication_node_id". I don't particularly like either.
> >
> > Other suggestions?
> I wonder if it should be origin_node_id? That is the term Slony uses.
I think in the slony context its clear that thats related to replication. Less
so in a general postgres guc. So maybe replication_origin_node_id?

> >> * Size of field. 16 bits is enough for 32,000 master nodes, which is
> >> quite a lot. Do we need that many? I think we may have need for a few
> >> flag bits, so I'd like to reserve at least 4 bits for flag bits, maybe
> >> 8 bits. Even if we don't need them in this release, I'd like to have
> >> them. If they remain unused after a few releases, we may choose to
> >> redeploy some of them as additional nodeids in future. I don't foresee
> >> complaints that 256 master nodes is too few anytime soon, so we can
> >> defer that decision.
> >
> > I wished we had some flag bits available before as well. I find 256 nodes
> > a pretty low value to start with though, 4096 sounds better though, so I
> > would be happy with 4 flag bits. I think for cascading setups and such
> > you want to add node ids for every node, not only masters...
> Even though the number of nodes that can reasonably participate in
> replication is likely to be not too terribly large, it might be good
> to allow larger values, in case someone is keen on encoding something
> descriptive in the node number.
Well, having a large number space makes the wal records bigger which is not
something I can see us getting through with. People have gone through
considerable length to avoid that.
Perhaps we can have a mapping system catalog at some point that includes
additional infromation to each node id like a name and where its at wrt
replication...

> I recall the Slony-II project having a notion of attaching a permanent
> UUID-based node ID to each node. As long as there is somewhere decent
> to find a symbolically significant node "name," I like the idea of the
> ID *not* being in a tiny range, and being UUID/OID-like...
I think adding 14 bytes (16bytes of an ooid - 2 bytes available) is out of
question...

> > Any opinions from others on this?
> >
> >> * Do we want origin_id as a parameter or as a setting in pgcontrol?
> >> IIRC we go to a lot of trouble elsewhere to avoid problems with
> >> changing on/off parameter values. I think we need some discussion to
> >> validate where that should live.
> >
> > Hm. I don't really forsee any need to have it in pg_control. What do you
> > want to protect against with that?
> > It would need to be changeable anyway, because otherwise it would need to
> > become a parameter for initdb which would suck for anybody migrating to
> > use replication at some point.
> >
> > Do you want to protect against problems in replication setups after
> > changing the value?
> In Slony, changing the node ID is Not Something That Is Done. The ID
> is captured in *way* too many places to be able to have any hope of
> updating it in a coordinated way. I should be surprised if it wasn't
> similarly troublesome here.
If you update you will need to reset consuming nodes, yes. Imo thats still
something else than disallowing changing the parameter entirely. Requiring
initdb for that seems like it would make experimentation too hard.

We need to allow at least changing the setting from no node id to an initial
one.

Greetings,

Andres
--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services