Re: Thoughts on statistics for continuously advancing columns

On Wed, 2009-12-30 at 17:16 +0100, Tom Lane wrote:
> I think the cleanest solution to this would be to make ANALYZE
> cheaper, perhaps by finding some way for it to work incrementally.

What if when inserting/deleting a tuple, some random sample of them
would be passed into an auto-analyze buffer ?

Then a special process (the auto-analyze daemon) would process them and
update the statistics incrementally based on the new values found (which
might or might not be mathematically feasible).

The overhead for each backend process would be kept in limits by the
rate at which you randomly send or not send the change to the analyze
buffer.

The processing overhead would be kept in limits by the processing rate
of the auto-analyze process, which can be made to periodically sleep or
it could be made to span multiple processes (on multiprocessor systems).

If the buffer is full, then you skip putting in it... so it also could
autotune itself to a sustainable rate.

Of course as with all my other posts on hackers, this is all mostly
hand-waving, I have no clue about the feasibility of all this with
regard to the current state of the code (which I didn't read, I
unfortunately found myself hating reading C code beyond reason, and
writing any of it till now resumed to copy-paste-modify).

Cheers,
Csaba.

Csaba Nagy
Software Engineer


eCircle
P: +49 (0)89 / 120 09-783 | F: +49 (0)89 / 120 09-750
E: c(dot)nagy(at)ecircle(dot)com
Nymphenburger Str. 86, 80636 Mnchen

Stay in touch
Web: www.ecircle.com/de | Newsletter: www.ecircle.com/index.php?id=63&L=0

Fr Hilfe mit dem eC-messenger wenden Sie sich bitte an unseren
Support: support-de(at)ecircle(dot)com(dot)

Neuste Untersuchungen
Ein unschlagbares Doppel: E-mail-Marketing & Webanalyse
Download Whitepaper: www.ecircle.com/index.php?id=61&L=0

eCircle AG, HRB 136 334, Handelsregister Mnchen Vorstand:
Volker Wiewer (Vorsitzender), Thomas Wilke, Lars Wssner,
Alexander Meyer Vorsitzender des Aufsichtsrates: Dr. Mark Wssner