Re: Add 64-bit XIDs into PostgreSQL 15

On Mon, Nov 28, 2022 at 11:06 PM Peter Geoghegan <pg(at)bowt(dot)ie> wrote:

> On Mon, Nov 28, 2022 at 1:52 PM Bruce Momjian <bruce(at)momjian(dot)us> wrote:
> > I think the problem is that we still have bloat with 64-bit XIDs,
> > specifically pg_xact and pg_multixact files. Yes, that bloat is less
> > serious, but it is still an issue worth reporting in the server logs,
> > though not serious enough to stop the server from write queries.
>
> That's definitely a big part of it.
>
> Again, I don't believe that the idea is fundamentally without merit.
> Just that it's not worth it, given that having more XID space is very
> much not something that I think fixes most of the problems. And given
> the real risk of serious bugs with something this invasive.

> I believe that it would be more useful to focus on just not getting
> into trouble in the first place, as well as on mitigating specific
> problems that lead to the system reaching xidStopLimit in practice. I
> don't think that there is any good reason to allow datfrozenxid to go
> past about a billion. When it does the interesting questions are
> questions about what went wrong, and how that specific failure can be
> mitigated in a fairly direct way.
>
> We've already used way to much "XID space runway", so why should using
> even more help? It might, I suppose, but it almost seems like a
> distraction to me, as somebody that wants to make things better for
> users in general. As long as the system continues to misbehave (in
> whatever way it happens to be misbehaving), why should any amount of
> XID space ever be enough?
>

So I think the problem is that PostgreSQL is becoming more and more
scalabile, hardware is becoming more capable, and certain use cases are
continuing to scale up. Over time, we tend to find ourselves approaching
the end of the runway at ever higher velocities. That's a problem that
will get significantly worse over time.

Of course, as I think we agree, the priorities should be (in order):
1. Avoid trouble
2. Recover from trouble early
3. Provide more and better options for recovery.

I think 64bit xids are a very good idea, but they really fit in this bottom
tier. Not being up against mathematical limits to the software when
things are going bad is certainly a good thing. But I am really worried
about the attitude that this patch really avoids trouble because in many
cases, I don;t think it does and therefore I believe we need to make sure
we are not reducing visibility of underlying problems.

>
> I think that we'll be able to get rid of freezing in a few years time.
> But as long as we have freezing, we have these problems.
>
> --
> Peter Geoghegan
>