Re: [RFC] Removing "magic" oids

Hi,

On 2018-09-29 20:48:10 -0700, Andres Freund wrote:
> In my opinion the current WITH OIDs system has numerous weaknesses:
>
> 1) The fact that oids are so magic means that if we get pluggable
> storage, the design of the potential pluggable systems is constrained
> and similar magic has to be present everywhere.
>
> 2) The fact that the oids in each table have the same counter to be
> based on means that oid wraparounds have much worse consequences
> performance wise than necessary. E.g. once the global counter has
> wrapped, all toast tables start to be significantly slower.
>
> It would be much better if most database objects had their own
> counters.
>
> 3) For some oid using objects (toast, large objects at the very least)
> it'd be quite worthwhile to switch to 8 byte ids. Currently that's
> hard to do, because it'd break on-disk compatibility.
>
> 4) There's a lot of special case code around for dealing with oids.
>
> 5a) The fact that system table oids don't show up in selects by default
> makes it more work than necessary to look at catalogs.
>
> 5b) Similarly, it's fairly annoying when debugging not to trivially see
> oids for catalog structs.
>
>
> I think we should drop WITH OIDs support. pg_dump should convert WITH
> OIDs tables into tables that have an explicit oid column (with an
> appropriate default function), pg_upgrade should refuse to upgrade them.
> We've defaulted WITH OIDs to off for quite a while now, so that's
> hopefully not going to be too painful.
>
> For catalog tables, I think we should just add explicit oid columns.
> That obviously requires a fair amount of change, but it's not too bad.
> One issue here is that we we want to support "manual" inserts both for
> initdb, and for emergency work.
>
>
> The attached *PROTOTYPE* *WIP* patch removes WITH OIDs support, converts
> catalog table oids to explicit oid columns, and makes enough
> infrastructure changes to make plain pg_regress pass (after the
> necessary changes of course). Only superficial psql and no pg_dump
> changes.
>
> There's plenty of issues with the prototype, but overall I'm pretty
> pleased.
>
>
> There's three major areas I'm not so sure about:
>
> 1) There's a few places dealing with system tables that don't deal with
> a hardcoded system table. Since there's no notion of "table has oid"
> and "which column is the oid column) anymore, we need some way to
> deal with that. So far I've just extended existing objectaddress.c
> code to deal with that, but it's not that pretty.
>
> 2) We need to be able to manually insert into catalog tables. Both
> initdb and emergency surgery. My current hack is doing so directly
> in nodeModifyTable.c but that's beyond ugly. I think we should add
> an explicit DEFAULT clause to those columns with something like
> nextoid('tablename', 'name_of_index') or such.
>
> 3) The quickest way to deal with the bootstrap code was to just assign
> all oids for oid carrying tables that don't yet have any assigned.
> That doesn't generally seem terrible, although it's certainly badly
> implemented right now. That'd mean we'd have three ranges of oids
> probably, unless we somehow have the bootstrap code advance the
> in-database oid counter to a right state before we start with
> post-bootstrap work. I like the idea of all bootstrap time oids
> being determined by genbki.pl (I think that'll allow to remove some
> code too).
>
>
> I do wonder about ripping the oid counter out entirely, and replacing it
> with sequences. But that seems like a separate project.
>
>
> I'll polish this up further in the not too far away future. But I'd
> really like to get some feedback before I sink more time into this.

Does anybody have engineering / architecture level comments about this
proposal?

Greetings,

Andres Freund