Re: [rfc] overhauling pgstat.stat

Sent from my iPad

On 08-Sep-2013, at 4:27, Tomas Vondra <tv(at)fuzzy(dot)cz> wrote:

> On 5.9.2013 09:36, Atri Sharma wrote:
>> On Thu, Sep 5, 2013 at 10:59 AM, Alvaro Herrera
>> <alvherre(at)2ndquadrant(dot)com> wrote:
>>> Satoshi Nagayasu wrote:
>>>
>>>> But, for now, I think we should have a real index for the
>>>> statistics data because we already have several index storages,
>>>> and it will allow us to minimize read/write operations.
>>>>
>>>> BTW, what kind of index would be preferred for this purpose?
>>>> btree or hash?
>>>
>>> I find it hard to get excited about using the AM interface for
>>> this purpose. To me it makes a lot more sense to have separate,
>>> much simpler code. We don't need any transactionality, user
>>> defined types, user defined operators, or anything like that.
>>
>> +1.
>>
>> But, would not rewriting a lot of existing functionalities
>> potentially lead to points of contention and/or much more effort?
>
> Don't forget the stats are written only by the postmaster, all the
> regular backends only read it (and eventually send updates back). But
> yes, contention might be a problem, because there will have to be some
> kind of locking that is not needed now when the postmaster is writing
> fresh copy into a new file.
>
> But I think we need to implement something and then measure this.
> Because it might even with the contention the overall performance might
> actually improve.
>
> I'd vote to try a simple approach first - adding some simple array
> 'index' allowing fast access to particular records etc. At least that
> was my plan. But feel free to implement something more advanced (e.g. a
> BTree storage) and we can compare

Another thing I would want to explore is if we could somehow prioritise the more frequently accessed records to reduce their access times even more.I am thinking on the lines of self organising lists.I am not sure if and how it would be possible to implement this idea of self organising in BTree or any other tree though.

Regards,

Atri