Re: Dead Space Map version 2

Hello, long time no see.

This topic looks interesting. I'm enrious of Itagaki-san and others.
I can't do now what I want, due to other work that I don't want to do
(isn't my boss seeing this?). I wish I could join the community some
day and contribute to the development like the great experts here.
# I can't wait to try Itagakis-san's latest patch for load distributed
checkpoint in my environment and report the result.
# But I may not have enough time...

Let me give some comment below.

From: "Heikki Linnakangas" <heikki(at)enterprisedb(dot)com>
> While I understand that 100% reliable coverage is a significantly
> stronger guarantee, I don't see any particular problems in
implementing
> that. WAL logging isn't that hard.
>
> I won't insist, I'm not the one doing the programming after all.
> Anything is better than what we have now. However, I do hope that
> whatever is implemented doesn't need a complete rewrite to make it
100%
> reliable in the future.
>
> The basic wish I have is to not use a fixed size shared memory area
like
> FSM for the DSM. I'd like it to use the shared buffers instead,
which
> makes the memory management and tuning easier. And it also makes it
> easier to get the WAL logging right, even if it's not done for 8.3
but
> added later.
>

I hope for the same thing as Heikki-san. Though I'm relatively new to
PostgreSQL source code, I don't think it is very difficult (at least
for experts here) to implement the reliable space management scheme,
so I proposed the following before -- not separate memory area for
FSM, but treating it the same way as data files in the shared buffers.
Though Tom-san is worrying about performance, what makes the
performance degrade greatly? Additional WAL records for updating
space management structures are written sequentially in batch.
Additional dirty shared buffers are written efficiently by kernel (at
least now.) And PostgreSQL is released from the giant lwlock for FSM.
Some performance degradation would surely result. However,
reliability is more important because "vacuum" is almost the greatest
concern for real serious users (not for hobbists who enjoy
performance.) Can anybody say to users "we are working hard, but our
work may not be reliable and sometimes fails. Can you see if our
vacuuming effort failed and try this...?"

And I'm afraid that increasing the number of configuration parameters
is unacceptable for users. It is merely the excuse of developers.
PostgreSQL already has more than 100 parameters. Some of them, such
as bgwriter_*, are difficult for normal users to understand. It's
best to use shared_buffers parameter and show how to set it in the
document.
Addressing the vacuum problem correctly is very important. I hope you
don't introduce new parameters for unfinished work and force users to
check the manual to change the parameters in later versions, i.e.
"managed_* parameters are not supported from this release. Please use
shared_buffers..." Is it a "must" to release 8.3 by this summer? I
think that delaying the release a bit for correct (reliable) vacuum
resolution is worth.

From: "Takayuki Tsunakawa" <tsunakawa(dot)takay(at)jp(dot)fujitsu(dot)com>
> Yes! I'm completely in favor of Itagaki-san. Separating the cache
for
> FSM may produce a new configuration parameter like fsm_cache_size,
> which the normal users would not desire (unless they like enjoying
> difficult DBMS.)
> I think that integrating the treatment of space management structure
> and data area is good. That means, for example, implementing "Free
> Space Table" described in section 14.2.2.1 of Jim Gray's book
> "Transaction Processing: Concepts and Techniques", though it may
have
> been discussed in PostgreSQL community far long ago (really?). Of
> course, some refinements may be necessary to tune to PostgreSQL's
> concept, say, creating one free space table file for each data file
to
> make the implementation easy. It would reduce the source code
solely
> for FSM.
>
> In addition, it would provide the transactional space management.
If
> I understand correctly, in the current implementation, updates to
FSM
> are lost when the server crashes, aren't they? The idea assumes
that
> FSM will be rebuilt by vacuum because vacuum is inevitable. If
> updates to space management area were made transactional, it might
> provide the infrastructure for "vacuumless PostgreSQL."