Re: [HACKERS] More stats about skipped vacuums

At Tue, 06 Feb 2018 19:24:37 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI <horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp> wrote in <20180206(dot)192437(dot)229464841(dot)horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp>
> At Tue, 6 Feb 2018 14:50:01 +0900, Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> wrote in <CAD21AoCRn6Q0wGG7UwGVsQJZbocNsRaZByJomUy+-GRkVH-i9A(at)mail(dot)gmail(dot)com>
> > The implementation of autovacuum work-item has been changed (by commit
> > 31ae1638) because dynamic shared memory is not portable enough. IIUC
> > this patch is going to do the similar thing. Since stats collector is
> > also a critical part of the server, should we consider whether we can
> > change it? Or the portability problem is not relevant with this patch?
>
> Thank you for the pointer. I digged out the following thread from
> this and the the patch seems to be the consequence of the
> discussion. I'll study it and think what to do on this.
>
> https://www.postgresql.org/message-id/20170814005656.d5tvz464qkmz66tq@alap3.anarazel.de

Done. The dominant reason for the ripping-off is that the
work-items array was allocated in a fixed-size DSA segment at
process startup time and wouldn't be resized.

Based on the reason, it fails to run when
dynamic_shared_memory_type = none and it is accompanied by
several cleanup complexities. The decision there is we should go
for just using static shared memory rather than adding complexity
for nothing. If it really needs to be expandable in the future,
it's the time to use DSA. (But would still maintain a fallback
stuff.)

Thinking of this, I think that this patch has a reason to use DSA
but needs some additional work.

- Fallback mechanism when dynamic_shared_memory_type = none

This means that the old file based stuff lives alongside the
DSA stuff. This is used when '= none' and on failure of DSA
mechanism.

- Out-of transactoin clean up stuff

Something like the following patch.

https://www.postgresql.org/message-id/20170814231638.x6vgnzlr7eww4bui@alvherre.pgsql

And as known problems:

- Make it use less LWLocks.

- DSHash shrink mechanism. Maybe need to find the way to detect
the necessity to shrink.

regards,

--
Kyotaro Horiguchi
NTT Open Source Software Center