Re: Rewriting Free Space Map

"Heikki Linnakangas" <heikki(at)enterprisedb(dot)com> writes:
> Tom Lane wrote:
>> You're cavalierly waving away a whole boatload of problems that will
>> arise as soon as you start trying to make the index AMs play along
>> with this :-(.

> It doesn't seem very hard.

The problem is that the index AMs are no longer in control of what goes
where within their indexes, which has always been their prerogative to
determine. The fact that you think you can kluge btree to still work
doesn't mean that it will work for other AMs.

>> Also, I don't think that "use the special space" will scale to handle
>> other kinds of maps such as the proposed dead space map. (This is
>> exactly why I said the other day that we need a design roadmap for all
>> these ideas.)

> It works for anything that scales linearly with the relation itself. The
> proposed FSM and visibility map both fall into that category.

It can work only with prearrangement among all the maps that are trying
to coexist in the same special space. Every time a new one comes along,
we'd have to reconsider the space allocation, re-optimize tradeoffs,
and force an initdb that could not possibly be implemented in-place.

If we had a short list of maps in mind with no real prospect of
changing, then I think this would be acceptable; but that doesn't seem
to be the case. It's certainly foolish to start detail design on
something like this when we don't even have the "roadmap" I asked
for about what sorts of maps we are agreed we want to have.

>> The idea that's becoming attractive to me while contemplating the
>> multiple-maps problem is that we should adopt something similar to
>> the old Mac OS idea of multiple "forks" in a relation.

> Hmm. You also need to teach at least xlog.c and xlogutils.c about the
> map forks, for full page images and the invalid page tracking.

Well, you'd have to teach them something anyway, for any incarnation
of maps that they might need to update.

> I also wonder what the performance impact of extending BufferTag is.

That's a fair objection, and obviously something we'd need to check.
But I don't recall seeing hash_any so high on any profile that I think
it'd be a big problem.

> My original thought was to have a separate RelFileNode for each of the
> maps. That would require no smgr or xlog changes, and not very many
> changes in the buffer manager, though I guess you'd more catalog
> changes. You had doubts about that on the previous thread
> (http://archives.postgresql.org/pgsql-hackers/2007-11/msg00204.php) but
> the "map forks" idea certainly seems much more invasive than that.

The main problems with that are (a) the need to expose every type of map
in pg_class and (b) the need to pass all those relfilenode numbers down
to pretty low levels of the system. The nice thing about the fork idea
is that you don't need any added info to uniquely identify what relation
you're working on. The fork numbers would be hard-wired into whatever
code needed to know about particular forks. (Of course, these same
advantages apply to using special space in an existing file. I'm
just suggesting that we can keep these advantages without buying into
the restrictions that special space would have.)

regards, tom lane