From: | Alfred Perlstein <alfred(at)freebsd(dot)org> |
---|---|
To: | Mark Kirkwood <mark(dot)kirkwood(at)catalyst(dot)net(dot)nz>, Robert Haas <robertmhaas(at)gmail(dot)com>, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> |
Cc: | Stephen Frost <sfrost(at)snowman(dot)net>, Bruce Momjian <bruce(at)momjian(dot)us>, Jim Nasby <Jim(dot)Nasby(at)bluetreble(dot)com>, Alex Ignatov <a(dot)ignatov(at)postgrespro(dot)ru>, Vladimir Sitnikov <sitnikov(dot)vladimir(at)gmail(dot)com>, Josh Berkus <josh(at)agliodbs(dot)com>, "Joshua D(dot) Drake" <jd(at)commandprompt(dot)com>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org> |
Subject: | Re: Why we lost Uber as a user |
Date: | 2016-08-16 23:37:36 |
Message-ID: | 20e8f6de-f4db-e96a-3c54-71ace3c8dbbc@freebsd.org |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
On 8/2/16 10:02 PM, Mark Kirkwood wrote:
> On 03/08/16 02:27, Robert Haas wrote:
>>
>> Personally, I think that incremental surgery on our current heap
>> format to try to fix this is not going to get very far. If you look
>> at the history of this, 8.3 was a huge release for timely cleanup of
>> dead tuple. There was also significant progress in 8.4 as a result of
>> 5da9da71c44f27ba48fdad08ef263bf70e43e689. As far as I can recall, we
>> then made no progress at all in 9.0 - 9.4. We made a very small
>> improvement in 9.5 with 94028691609f8e148bd4ce72c46163f018832a5b, but
>> that's pretty niche. In 9.6, we have "snapshot too old", which I'd
>> argue is potentially a large improvement, but it was big and invasive
>> and will no doubt pose code maintenance hazards in the years to come;
>> also, many people won't be able to use it or won't realize that they
>> should use it. I think it is likely that further incremental
>> improvements here will be quite hard to find, and the amount of effort
>> will be large relative to the amount of benefit. I think we need a
>> new storage format where the bloat is cleanly separated from the data
>> rather than intermingled with it; every other major RDMS works that
>> way. Perhaps this is a case of "the grass is greener on the other
>> side of the fence", but I don't think so.
>>
> Yeah, I think this is a good summary of the state of play.
>
> The only other new db development to use a non-overwriting design like
> ours that I know of was Jim Starky's Falcon engine for (ironically)
> Mysql 6.0. Not sure if anyone is still progressing that at all now.
>
> I do wonder if Uber could have successfully tamed dead tuple bloat
> with aggressive per-table autovacuum settings (and if in fact they
> tried), but as I think Robert said earlier, it is pretty easy to come
> up with a highly update (or insert + delete) workload that makes for a
> pretty ugly bloat component even with real aggressive autovacuuming.
I also wonder if they had used "star schema" which to my understanding
would mean multiple tables to replace the single-table that has multiple
indecies to work around the write amplification problem in postgresql.
>
> Cheers
>
> Mark
>
>
>
From | Date | Subject | |
---|---|---|---|
Next Message | Alfred Perlstein | 2016-08-16 23:38:36 | Re: Why we lost Uber as a user |
Previous Message | Christopher Browne | 2016-08-16 23:12:31 | Re: [GENERAL] C++ port of Postgres |