Re: mdnblocks() sabotages error checking in _mdfd_getseg()

On 2015-12-10 11:10:10 -0500, Robert Haas wrote:
> (More broadly, as Kevin was pointing out to me yesterday, md.c looks
> like it could do with a face lift. Keeping a linked list of 1GB
> segments and chasing down the list to find the length of the file may
> have been fine when relations over 1GB were rare, but that's now
> routine. Some relations may be over 1TB, and using a linked list of
> 1000 entries to keep track of all of those segments does not seem like
> an especially good choice.

Yes, that sucks. I've posted a patch for that at
http://archives.postgresql.org/message-id/20141031223210.GN13584%40awork2.anarazel.de
- I don't have access to to either the workload or a good testing
machine anymore though, so I've kinda lost interest for a while.

I'll try to push the patch with the renaming suggested downthread by Tom
soonish.

> In fact, having no way to get the relation length other than scanning
> 1000 files doesn't seem like an especially good choice even if we used
> a better data structure. Putting a header page in the heap would make
> getting the length of a relation O(1) instead of O(segments), and for
> a bonus, we'd be able to reliably detect it if a relation file
> disappeared out from under us. That's a difficult project and
> definitely not my top priority, but this code is old and crufty all
> the same.)

The md layer doesn't really know whether it's dealing with an index, or
with an index, or ... So handling this via a metapage doesn't seem
particularly straightforward.

Andres