Re: Fast insertion indexes: why no developments

On Wed, Oct 30, 2013 at 4:23 AM, Leonardo Francalanci <m_lists(at)yahoo(dot)it>wrote:

>
>
> 1) I haven't achieved what I need: realtime indexing. I can't query the
> "current 15 minutes" table efficiently. Plus, K*log(N) is not that great
> when you have a lot of K.
>

Are partitions read-only once time has moved on, or can stragglers show up
that need to be inserted into older partitions?

You could periodically merge older partitions into larger tables, index
those aggregated tables, then transactionally disinherit the old partitions
and inherit the new aggregated one. This would keep the value of K down,
at the expense of re-writing data multiple times (but all method write data
multiple times, some just hide it from you).

> 2) I'm not suggesting that this is top priority. I'm asking if there's
> something else, other than "we don't have time for this", that I don't
> know. In fact, I don't even know if those indexes types would really help
> in my (specific) case. That's why my original question was "why aren't
> there developments in this area": I didn't mean to imply someone should do
> it. I just wanted to know if those indexes were already discussed (and
> maybe dismissed for some reason) in the past...
>
>
There have been several ideas discussed, but not a whole lot of time to
implement them.

By the way, what is the transaction structure of your inserts? Are they
large batches between commits, or is each row committed?

Cheers,

Jeff