RE: [HACKERS] mdnblocks is an amazing time sink in huge relations

> -----Original Message-----
> From: owner-pgsql-hackers(at)postgreSQL(dot)org
> [mailto:owner-pgsql-hackers(at)postgreSQL(dot)org]On Behalf Of Tom Lane
> Sent: Tuesday, October 12, 1999 1:12 PM
> To: pgsql-hackers(at)postgreSQL(dot)org
> Subject: [HACKERS] mdnblocks is an amazing time sink in huge relations
>
>
> I have been doing some testing with multi-segment relations.
> (Not having sixty gigabytes laying around, like some folk I've been
> talking to recently, I rebuilt with a very small RELSEG_SIZE for
> testing...) I have discovered that performance goes to zero as the
> number of segments gets large --- in particular, if the number of
> segments exceeds the number of kernel file descriptors that fd.c
> thinks it can use, you can take a coffee break while inserting a
> few tuples :-(. The main problem is mdnblocks() in md.c, which
> is called for each tuple inserted; it insists on touching every
> segment of the relation to verify its length via lseek(). That's an
> unreasonable number of kernel calls in any case, and if you add a file
> open and close to each touch because of file-descriptor thrashing,
> it's coffee break time.
>

It's me who removed the following code from mdnblocks().
The code caused a disaster in case of mdtruncate().

if (v->mdfd_lstbcnt == RELSEG_SIZE
|| ....

At that time I was anxious about the change of performance
but I've forgotten,sorry.

> It seems to me that mdnblocks should assume that all segments
> previously verified to be of length RELSEG_SIZE are still of that
> length, and only do an _mdnblocks() call on the last element of the
> segment chain. That way the number of kernel interactions needed
> is a constant independent of the number of segments in the table.
> (This doesn't make truncation of the relation by a different backend
> any more or less dangerous; we're still hosed if we don't get a
> notification before we try to do something with the relation.
> I think that is fixed, and if it's not this doesn't make it worse.)
>

Currently only the first segment is opened when mdopen() etc is
called. Would you change to check all segments at first open ?

If there is a relation which has multi-segment of size 0,which would
be the last segment ?

Regards.

Hiroshi Inoue
Inoue(at)tpf(dot)co(dot)jp