From: | jesper(at)krogh(dot)cc |
---|---|
To: | "Andres Freund" <andres(at)2ndquadrant(dot)com> |
Cc: | jesper(at)krogh(dot)cc, "Pg Hackers" <pgsql-hackers(at)postgresql(dot)org> |
Subject: | Re: wal-size limited to 16MB - Performance issue for subsequent backup |
Date: | 2014-10-20 19:41:26 |
Message-ID: | 13292d3e12dee87e239bc64576ed2d73.squirrel@shrek.krogh.cc |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
> On 2014-10-20 21:03:59 +0200, jesper(at)krogh(dot)cc wrote:
>> One of our "production issues" is that the system generates lots of
>> wal-files, lots is like 151952 files over the last 24h, which is about
>> 2.4TB worth of WAL files. I wouldn't say that isn't an issue by itself,
>> but the system does indeed work fine. We do subsequently gzip the files
>> to
>> limit actual disk-usage, this makes the files roughly 30-50% in size.
>
> Have you analyzed what the source of that volume is? Which version of
> postgres are you using? What's your checkpoint_timeout/segments
> settings?
Suggestions are surely welcome. I do suspect the majority is from 30
concurrent processes updating an 506GB GIN index, but it would be nice to
confirm that. There is also a message-queue in the DB with a fairly high
turnaround.
Currently PG 9.2 moving to 9.3 hopefully before end-of-year,
checkpoint_timeout = 30min, checkpoint_segments = 4096.
According to logs checkpoints are roughly 15 minutes apart.
Jesper
From | Date | Subject | |
---|---|---|---|
Next Message | Andres Freund | 2014-10-20 19:49:40 | Re: wal-size limited to 16MB - Performance issue for subsequent backup |
Previous Message | Tom Lane | 2014-10-20 19:39:03 | Re: wal-size limited to 16MB - Performance issue for subsequent backup |