Re: patch submission: truncate trailing nulls from heap rows to reduce the size of the null bitmap [Review]

On Mon, Jun 24, 2013 at 4:05 PM, Simon Riggs <simon(at)2ndquadrant(dot)com> wrote:
> I think its rather a shame that the proponents of this patch have
> tried so hard to push this through without working variations on the
> theme. Please guys, go away and get creative; rethink the approach so
> that you can have your lunch without anybody else losing theirs. I
> would add that I have seen the use case and want to support it, but
> we're looking to add to the codebase not just steal small bites of
> performance from existing use cases.

If there's an actual performance consequence of applying this patch,
then I think that's a good reason for rejecting it. But if the best
argument we can come up with is that we might someday try to do even
more clever things with the tuple's natts value, I guess I'm not very
impressed. The reason why we have to rewrite the table when someone
adds a column with a non-NULL default is because we need to store the
new value in every row. Sure, we could defer that in this particular
case. But users might be mighty dismayed to see CLUSTER or VACUUM
FULL -- or a dump-and-reload! -- cause the table to become much LARGER
than it was before. Having some kind of column-oriented compression
would be darn nifty, but this particular path doesn't seem
particularly promising to me.

I think the larger and more fundamental problem with natts is that
there's just not very much bit space available there. Even as it is,
someone who adds and drops columns with any regularity will eventually
hit the wall, and there aren't any good alternatives at that point.
Were there more bit space available here, we could even do things like
allow some special cases of ALTER TABLE .. SET DATA TYPE not to
rewrite the table; natts could become a sort of tuple version number,
and we'd remember how to convert to newer versions on the fly. But
with only 2048 values available, it's not really feasible to start
consuming them for any more operations than we already do. I'm
generally a big fan of the principle that no single patch should be
allowed to crowd out room for future extensibility in this particular
area, but in this case I think the bit space available is *already* so
limited that we're not likely to get much further with it without
changing the storage format.

So, Tom, how's that pluggable storage format coming? :-)

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company