Re: GSoC proposal - "make an unlogged table logged"

On Thu, Apr 3, 2014 at 7:26 AM, Heikki Linnakangas
<hlinnakangas(at)vmware(dot)com> wrote:
> On 04/01/2014 08:39 PM, Heikki Linnakangas wrote:
>> On 03/07/2014 05:36 AM, Tom Lane wrote:
>>> =?ISO-8859-1?Q?Fabr=EDzio_de_Royes_Mello?= <fabriziomello(at)gmail(dot)com>
>>> writes:
>>>> Do you think is difficult to implement "ALTER TABLE ... SET UNLOGGED"
>>>> too?
>>>> Thinking in a scope of one GSoC, of course.
>>>
>>> I think it's basically the same thing. You might hope to optimize it;
>>> but you have to create (rather than remove) an init fork, and there's
>>> no way to do that in exact sync with the commit.
>>
>> You just have to include that information with the commit WAL record, no?
>
> No-one's replied yet, but perhaps the worry is that after you've written the
> commit record, you have to go ahead with removing/creating the init fork,
> and that is seen as too risky. If a creat() or unlink() call fails, that
> will have to be a PANIC, and crash recovery will likewise have to PANIC if
> the forks still cannot be removed/created.

Yeah, that's the concern.

If I may digress for a moment, unlogged materialized views are not
supported. This is because we have this facility where if a
materialized view hasn't been populated yet, you get an error when you
try to scan it. If we allowed unlogged materialized views, then
they'd get reset to empty rather than to not-populated, because the
not-populated status is stored in the catalog, not the filesystem. I
still wish we'd never added the notion of populated in the first
place, but Kevin felt it was essential, so we ended up here.

Anyway, the idea that I had for fixing the unlogged materialized view
case was to add a new 64-bit integer to the control file that gets
bumped every time we start crash recovery, and which also gets
recorded in pg_class. The value 0 would be reserved, and all pg_class
entries for non-unlogged relations would store 0. For unlogged
relations, we could check whether the value in pg_class equals the
current value; if not, the relation should be viewed as not-populated.

This is not too far from a solution from the problem we need to solve
here. If we want to make an unlogged relation logged, we can go ahead
and remove the init forks right away, knowing that the pg_class update
changing relpersistence and this new value won't take effect until
commit. If the system meanwhile crashes, a backend connected to the
relevant database has enough state to recognize that the relation is
in this in-between state. Before we can again use that relation, we
need to rebuild the init fork and reset it. Of course, it's not too
clear exactly how that state cleanup happens; as one option, we could
just require the user to run a manual TRUNCATE.

This would not be totally without precedent, because CREATE INDEX
CONCURRENTLY leaves crap behind that the user has to reindex or drop,
but it's clearly not ideal. Another option would be to try to make
autovacuum put things right, but that of course might not happen right
away.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company