From: | Alexander Korotkov <aekorotkov(at)gmail(dot)com> |
---|---|
To: | pgsql-hackers <pgsql-hackers(at)postgresql(dot)org> |
Subject: | WIP: Access method extendability |
Date: | 2014-10-15 12:08:38 |
Message-ID: | CAPpHfdsXwZmojm6Dx+TJnpYk27kT4o7Ri6X_4OSWcByu1Rm+VA@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
Hackers,
Postgres was initially designed to support access methods extendability.
This extendability lives to present day. However, this is mostly internal
in-core extendability. One can quite easily add new access method into
PostgreSQL core. But if one try to implement access method as external
module, he will be faced with following difficulties:
1. Need to directly insert into pg_am, because of no "CREATE ACCESS
METHOD" command. And no support of dependencies between am and opclasses
etc.
2. Module can't define xlog records. So, new am would be not WAL-logged.
The first problem is purely mechanical. Nothing prevents us to implement
"CREATE ACCESS METHOD" and "DROP ACCESS METHOD" commands and support all
required dependencies.
Problem of WAL is a bit more complex. According to previous discussions, we
don't want to let extensions declare their own xlog records. If we let them
then recovery process will depend on extensions. That is much violates
reliability. Solution is to implement some generic xlog record which is
able to represent difference between blocks in some general manner.
3 patches are attached:
1. CREATE/DROP ACCESS METHOD commands. With support of dependencies.
2. New generic xlog record type.
3. Bloom contrib module as example of usage of previous two features.
This module was posted few years ago by Teodor Sigaev. Now, it's wrapped as
an extension and WAL-logged.
Patches are in WIP state. No documentation and very little of comments.
However, it believe that it's enough to do some general concept review.
Some notes about generic xlog format. Generic xlog represent difference
between pages as operations set of two kinds:
1. Move memory inside the page. Optional flag is to zero gap on a
previous memory location.
2. Copy memory fragment of memory from xlog record to page. As an option
bitwise logical operations are supported as well as plain copy.
Generic xlog can open page in two modes:
1. Create mode: page is zeroed independent on its lsn.
2. Update mode: page is updated only if it's lsn is lower than record lsn
Usually, xlog record is filled in critical sections when memory allocations
is prohibited. Thus, user have to previously initialize it with knowledge
of pages count, total operations count and total length of data.
------
With best regards,
Alexander Korotkov.
Attachment | Content-Type | Size |
---|---|---|
create-am.1.patch.gz | application/x-gzip | 6.3 KB |
generic-xlog.1.patch.gz | application/x-gzip | 4.1 KB |
bloom-contrib.1.patch.gz | application/x-gzip | 37.4 KB |
From | Date | Subject | |
---|---|---|---|
Next Message | Simon Riggs | 2014-10-15 12:27:40 | Re: Buffer Requests Trace |
Previous Message | MauMau | 2014-10-15 12:03:42 | Re: [9.4 bug] The database server hangs with write-heavy workload on Windows |