Re: GSoC proposal - "make an unlogged table logged"

On 2014-04-03 14:26:50 +0300, Heikki Linnakangas wrote:
> On 04/01/2014 08:39 PM, Heikki Linnakangas wrote:
> >On 03/07/2014 05:36 AM, Tom Lane wrote:
> >>=?ISO-8859-1?Q?Fabr=EDzio_de_Royes_Mello?= <fabriziomello(at)gmail(dot)com> writes:
> >>>Do you think is difficult to implement "ALTER TABLE ... SET UNLOGGED" too?
> >>>Thinking in a scope of one GSoC, of course.
> >>
> >>I think it's basically the same thing. You might hope to optimize it;
> >>but you have to create (rather than remove) an init fork, and there's
> >>no way to do that in exact sync with the commit.
> >
> >You just have to include that information with the commit WAL record, no?
>
> No-one's replied yet

That might be because it was a month after the initial discussion, and
at least I'd temporarily lost track of the thread ;)

> , but perhaps the worry is that after you've written the
> commit record, you have to go ahead with removing/creating the init fork,
> and that is seen as too risky. If a creat() or unlink() call fails, that
> will have to be a PANIC, and crash recovery will likewise have to PANIC if
> the forks still cannot be removed/created.

That's part of the worry, yes. It's also creeping code dealing with
unlogged relations into a fairly critical place
(RecordTransactionCommit()) where it really doesn't seem to belong.

> My first thought is that that seems ok. It's unlikely that an unlink() of a
> small file in the data directory would fail. Creation could be done with a
> temporary name first and renamed into place, to avoid running out of disk
> space in the critical section.

I continue to feel that that's far too much impact for a minor
feature. Even if it could be made work reliably, it'll be a fair amount
of seldomly used infrastructure.

> If that's not acceptable, one idea off the top of my head is to somehow
> stamp the init forks when making an unlogged table logged, with the XID of
> the transcation. Crash recovery could then check the clog to see if the
> transaction committed, and ignore any init fork files belonging to committed
> transactions. (Same in reverse when making a logged table unlogged).

I've thought about that - after all, the logical decoding stuff uses
that trick in some places - but it has the grave disadvantage that it
requires a full directory scan to fully remove a relation. That seems to
be a heavy price.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services