Re: preserving forensic information when we freeze

On Thu, Jan 2, 2014 at 12:46 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Andres Freund <andres(at)2ndquadrant(dot)com> writes:
>> On 2014-01-02 09:40:54 -0500, Tom Lane wrote:
>>> Actually, I thought the function approach was a good proposal. You are
>>> right that func(tab.*) isn't going to work, because it's going to get a
>>> Datum-ified tuple not a pointer to raw on-disk storage. But an inspection
>>> function that's handed a ctid could work.
>
>> Well, we discussed that upthread, and the overhead of going through a
>> function is quite noticeable because the tuple needs to be fetched from
>> the heap again.
>
> Yeah, I read those results, but that seems like it could probably be
> optimized. I'm guessing the function was doing a new heap_open every
> time? There's probably a way around that.

Yeah, it was. I don't see any plausible way around that, but I'm all ears.

> In any case, upon further reflection I'm not convinced that doing this
> with a SELECT-based query is the right thing, no matter whether the query
> looks at a function or a system column; because by definition, you'll only
> be able to see tuples that are visible to your current snapshot. For real
> forensics work, you need to be able to see all tuples, which makes me
> think that something akin to pgstattuple is the right API; that is "return
> a set of the header info for all tuples on such-and-such pages of this
> relation". That should dodge any performance problem, because the
> heap_open overhead could be amortized across lots of tuples, and it also
> sidesteps all problems with adding new system columns.

I both agree and disagree with this. I think that pgstattuple is
useful, and I further agree that adding a stat to it about how much of
the heap is frozen would be worthwhile. However, an aggregate number
isn't always what you want, and being able to scrutinize specific
tuples is, I think, a useful thing. I'm not sure that it needs to be
fast, which is why I think a function rather than a system column
might be OK, but I think it ought to be possible.

>> Upthread there's a POC patch of mine, that started to explore what's
>> necessary to simply never store system columns (except maybe oid) in
>> pg_attribute. While it passes the regression tests it's not complete,
>> but the amount of work looks reasonable.
>
> I think this will inevitably break a lot of code, not all of it ours,
> so I'm not in favor of pursuing that direction.

I thought that that approach had been discussed previously and found
desirable on the grounds that it would cut down on the size of
pg_attribute, but it's not something I want to rush through when we
have a release to get out the door.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company