Re: proposal: Set effective_cache_size to greater of .conf value, shared_buffers

Le jeudi 5 septembre 2013 17:14:37 Bruce Momjian a écrit :
> On Thu, Sep 5, 2013 at 06:14:33PM +0200, Magnus Hagander wrote:
> > > I have developed the attached patch which implements an auto-tuned
> > > effective_cache_size which is 4x the size of shared buffers. I had to
> > > set effective_cache_size to its old 128MB default so the EXPLAIN
> > > regression tests would pass unchanged.
> >
> > That's not really autotuning though. ISTM that making the *default* 4
> > x shared_buffers might make perfect sense, but do we really need to
> > hijack the value of "-1" for that? That might be useful for some time
> > when we have actual autotuning, that somehow inspects the system and
> > tunes it from there.
> >
> > I also don't think it should be called autotuning, when it's just a
> > "smarter default value".
> >
> > I like the feature, though, just not the packaging.
>
> That "auto-tuning" text came from the wal_buffer documentation, which
> does exactly this based on shared_buffers:
>
> The contents of the WAL buffers are written out to disk at every
> transaction commit, so extremely large values are unlikely to
> provide a significant benefit. However, setting this value to at
> least a few megabytes can improve write performance on a busy
> --> server where many clients are committing at once. The auto-tuning
> -----------
> selected by the default setting of -1 should give reasonable
> results in most cases.
>
> I am fine with rewording and not using -1, but we should change the
> wal_buffer default and documentation too then. I am not sure what other
> value than -1 to use? 0? I figure if we ever get better auto-tuning,
> we would just remove this functionality and make it better.

I'm fine with a -1 for auto-tune or inteligent default: it means (for me) that
you don't need to care about this parameter in most case.

A negative impact of the simpler multiplier might be that if suddendly someone
reduce the shared_buffers size to fix some strange behavior, then he at the same
needs to increase manualy the effective_cache_size (which remain the sum of the
caches on the system, at least on a dedicated to PostgreSQL one).

IMHO it is easy to know exactly how much of the memory is (or can be) used
for/by PostgreSQL, we can compute that and update effective_cache_size at
regular point int time. (just an idea, I know there are arguments against that
too)

Maybe the value for a 4x multiplier instead of 3x, is that the
effective_cache_size usage can be larger than required. It's not a big trouble.
With all things around NUMA we maybe just need to revisit that area (memory
access cost non linear, double-triple caching, ...) .
--
Cédric Villemain +33 (0)6 20 30 22 52
http://2ndQuadrant.fr/
PostgreSQL: Support 24x7 - Développement, Expertise et Formation