Re: [BUGS] server crash in very big transaction [postgresql

On Wed, Aug 25, 2004 at 11:21:49AM +1000, Gavin Sherry wrote:
> On Tue, 24 Aug 2004, Tom Lane wrote:

> > 1. Allow XLOG records to be larger than 64K.
> >
> > 2. Split transaction commit into multiple XLOG records when there are
> > many subtransactions.
>
> [snip]
>
> > I'm inclined to go with #1. There are various ways we could do it
> > but the most straightforward would be to just widen the xl_len field
> > to 32 bits. This would cost either 4 or 8 bytes per XLOG record
> > (because of MAXALIGN restrictions) but we could more than buy that back
> > by eliminating the xl_prev and/or xl_xact_prev fields, which have no use
> > in the current system. (They were intended to support UNDO but it seems
> > clear that we will never do that.)

If we agree to never implement UNDO, there's a bunch of other code that
could be removed. Is there anyone that thinks we have any chance of not
doing it?

OTOH, if those fields are unused, we could just remove them for now in
any case. It's unlikely that there won't be a catalog update for some
other reason before someone implements UNDO anyway.

> As for extending the length of xl_len, what happens if someone now has
> 2^30 subtransaction IDs (as unlikely as that sounds)?

The commit xlog record also carries dropped table information, 12 bytes
apiece (on 32 bit machines?). It's unlikely that anyone will drop 2^13
tables on a single transaction, but it adds to the child xid list.

> > Or we could assign an rmgr value to represent an "extension" record that
> > is to be merged with a following "normal" record. This is kinda klugy
> > but would avoid wasting bits on xl_len in the vast majority of records.
> > Also we'd not have to force an initdb since the file format would
> > remain upward-compatible.
>
> This is a better idea, I think, as it avoids the problems above and, as
> you say, will be binary compatible.

I also think this is a good idea. Would it be generalized or only
applicable to xl_xact_{commit,abort} records?

--
Alvaro Herrera (<alvherre[a]dcc.uchile.cl>)
"Vivir y dejar de vivir son soluciones imaginarias.
La existencia está en otra parte" (Andre Breton)