Re: WIP: a way forward on bootstrap data

From: John Naylor <jcnaylor(at)gmail(dot)com>
To: Alvaro Herrera <alvherre(at)alvh(dot)no-ip(dot)org>
Cc: David Fetter <david(at)fetter(dot)org>, Peter Eisentraut <peter(dot)eisentraut(at)2ndquadrant(dot)com>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: WIP: a way forward on bootstrap data
Date: 2017-12-28 11:26:23
Message-ID: CAJVSVGVJHwD8sfDfZW9TbCHWKf=C1YDRM-rF=2JenRU_y+VcFg@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

I wrote:

> On 12/22/17, Alvaro Herrera <alvherre(at)alvh(dot)no-ip(dot)org> wrote:
>> Maybe a generated .h file containing defines for OIDs from all catalogs
>> is okay. Of course, not all symbols are to be listed -- we should have
>> a special marker in the data lines for those that are. Maybe something
>> like this
>>
>> { oid => '403', descr => 'b-tree index access method',
>> amname => 'btree', amhandler => 'bthandler', amtype => 'i',
>> cpp_symbol => 'BTREE_AM_OID' },
>>
>> (where 'cpp_symbol' would be skipped by genbki explicitly).
>
> The last part makes sense and would be a fairly mechanical change. I'm
> not sure of the best way to include those generated symbols back in
> the code again, though. I think a single file might not be desirable
> because of namespace pollution.
[snip]

I've attached version 5 (rebased onto be2343221fb), which removes OID
#define symbols from the headers and stores them with the records they
refer to.

I went ahead with your suggestion to try a single generated header. I
believe there is no chance of namespace pollution since the symbols
already have a nomenclature that reflects what catalog they belong to.
This required some additional Makefile changes, since some code
outside the backend needs certain OID symbols to be visible. There are
probably bugs, but I wanted to share the initial design. The MSVC
changes are untested. In addition, FindDefinedSymbol() now doesn't
work for catalog headers, so I added a new function to search within
the data.

On the plus side, there is now a mechanism to generate pg_type OID
symbols, and ECPG's knowledge of types is now maintained
automatically.

Since I had to rebase over recent additions to SP-GiST opclasses
anyway, I restructured the patches to have a clean separation between
data migration and data compaction. This makes the patches easier to
follow.

The pg_proc defaults have been tweaked slightly to match those
suggested by Andrew Dunstan [1].

There are now human-readable entries for oprcom and oprnegate in
pg_operator.dat.

Finally, assorted cosmetic improvements and README/comment editing.

[1] https://www.postgresql.org/message-id/b76d153a-33d7-7827-746c-1109f7bf529d%40dunslane.net
--

-John Naylor

Attachment Content-Type Size
v5-0001-Remove-hard-coded-schema-knowledge-about-pg_attri.patch text/x-patch 15.3 KB
v5-0002-Data-conversion-infrastructure.patch text/x-patch 24.6 KB
v5-0003-Mechanical-data-conversion.patch.tar.gz application/x-gzip 123.5 KB
v5-0004-Hand-edits-of-data-files.patch text/x-patch 136.7 KB
v5-0005-Update-catalog-scripts-to-read-data-files.patch text/x-patch 22.9 KB
v5-0006-Remove-data-from-catalog-headers.patch.tar.gz application/x-gzip 108.1 KB
v5-0007-Remove-OID-define-symbols-from-catalog-headers.patch text/x-patch 34.4 KB
v5-0008-Implement-data-compaction-strategies.patch text/x-patch 36.3 KB
v5-0009-Data-file-compaction.patch.tar.gz application/x-gzip 161.7 KB
v5-0010-Replace-OIDs-with-human-readable-names.patch.tar.gz application/x-gzip 127.9 KB
v5-0011-Type-aliases-for-OID-lookups.patch text/x-patch 13.8 KB
v5-0012-Reduce-indentation-level.patch text/x-patch 19.8 KB
v5-0013-Move-toast-index-macros-to-the-pg_-catalog-header.patch text/x-patch 82.5 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Andrey Borodin 2017-12-28 11:37:48 Re: New gist vacuum.
Previous Message Yugo Nagata 2017-12-28 09:04:22 [PATCH] GET DIAGNOSTICS FUNCTION_NAME