Re: PostgreSQL with ZFS on Linux

On Thu, Jan 16, 2014 at 3:22 AM, Sébastien Lorion
<sl(at)thestrangefactory(dot)com>wrote:

> Hello,
>
> Since ZFS on Linux (http://zfsonlinux.org/) has been declared production
> ready last March (v0.6.1), I am curious if anyone is using it with
> PostgreSQL on production servers (either main or backup) and if so, what is
> their experience so far ?
>
> Thank you,
>
> Sébastien
>

I would highly recommend you proceed with caution and lots of testing. If
you look back in the archives of a week or so ago on this list, I posted
with memory issues. After several helpful suggestions, we found the root
cause to be postgresql coupled with ZFS on Linux. Basically, as I
understand it (I may be a bit off), the ZFS Linux port has a middle layer
to translate and emulate the Solaris kernel. Because of this, it isn't as
efficient at memory mangement as a more native port like FreeBSD or Solaris
itself. It is also prone to memory bugs/leaks with large rsync or file
copy operations. Couple this with a memory hungry database and you can
start having issues.

In our case, we went from a server with 4 GB of RAM on a hardware RAID 10
and Postgresql 8.3 to a system with 32 GB of RAM, ZFS and Postgresql 9.3.
Once we started pushing people to the new server and it got under load,
postgres started being unable to allocate new RAM when it needed it. This
was with same number of clients and same databases as the old 4 GB system,
so it seemed crazy. With some great help of some of the people on this
list, we started looking into what was going on and basically found that
ZFS had taken 24 GB of RAM, about 18 of which was not cached/shared RAM but
"real" RAM it was holding onto and wouldn't let go of.

We have since moved to FreeBSD with ZFS and Postgres 9.3 (everything else
the same) and performance has been awesome (as we were expecting out of the
new server). If you really want ZFS, I would highly recommend looking into
FreeBSD (Postgresql works great on it) or if you want to stick with Linux,
look into mdadm with LVM or some other filesystem solution.

A few helpful things we found:

Some notes on ZFS on Linux of things to watch out for:

http://utcc.utoronto.ca/~cks/space/blog/linux/ZFSonLinuxWeakAreas

A great guide to figuring out how much RAM postgresql is actually using
(assuming you run into RAM issues):

http://www.depesz.com/2012/06/09/how-much-ram-is-postgresql-using/

Preston