Re: MusicBrainz postgres performance issues

On 15.3.2015 13:07, Robert Kaye wrote:
>
>> If the load problem really is being caused by swapping when things
>> really shouldn't be swapping, it could be a matter of adjusting your
>> swappiness - what does cat /proc/sys/vm/swappiness show on your server?
>
> 0
>
> We adjusted that too, but no effect.
>
> (I’ve updated the blog post with these two comments)

IMHO setting swappiness to 0 is way too aggressive. Just set it to
something like 10 - 20, that works better in my experience.

>> There are other linux memory management things that can cause
>> postgres and the server running it to throw fits like THP and zone
>> reclaim. I don't have enough info about your system to say they are
>> the cause either, but check out the many postings here and other
>> places on the detrimental effect that those settings *can* have.
>> That would at least give you another angle to investigate.
>
> If there are specific things you’d like to know, I’ve be happy to be a
> human proxy. :)

I'd start with vm.* configuration, so the output from this:

# sysctl -a | grep '^vm.*'

and possibly /proc/meminfo. I'm especially interested in the overcommit
settings, because per the free output you provided there's ~16GB of free
RAM.

BTW what amounts of data are we talking about? How large is the database
and how large is the active set?

I also noticed you use kernel 3.2 - that's not the best kernel version
for PostgreSQL - see [1] or [2] for example.

[1]
https://medium.com/postgresql-talk/benchmarking-postgresql-with-different-linux-kernel-versions-on-ubuntu-lts-e61d57b70dd4

[2]
http://www.databasesoup.com/2014/09/why-you-need-to-avoid-linux-kernel-32.html

--
Tomas Vondra http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services