Re: Disk latency goes up during certaing pediods

BTW I have all this data ploted using munin, let me know if you are
intrested in looking at the graphs, I send them

On Mon, Jul 29, 2013 at 4:41 PM, German Becker <german(dot)becker(at)gmail(dot)com>wrote:

> Luis, The disk only has the WAL (pg_xlog ) directory. Brett, Here are the
> mount options:
>
> /dev/sdb1 on /storage/sdb1 type ext3
> (rw,noatime,data=writeback,errors=remount-ro)
>
> BTW The original fs was ext4, now I am trying with ext3, with the exact
> same results. No noticeable changes using diferent journal modes.
>
> I also tried disabling the journal altogether, which dramatically reduced
> the disk usage, but nevertheless there was this latency spikes.
>
>
> On Mon, Jul 29, 2013 at 3:01 PM, Brett Stauner <brett(at)mightybs(dot)net> wrote:
>
>> Okay, so it's not happening at the same time of day or anything. What
>> are your mount options for the WAL disk?
>>
>>
>> On Mon, Jul 29, 2013 at 12:58 PM, German Becker <german(dot)becker(at)gmail(dot)com>wrote:
>>
>>>
>>> Brett,
>>>
>>> Yes I'm not impying it is postgres related, perhaps is even a normal
>>> thing of ext3 /ext4 filesystem, but, this behavioiur is only notable when
>>> using postgres and in particular the wal files, as it is very hard disk
>>> intensive, soy maybe somenone has seen this befores. The server only task
>>> is the database. It has 4 disks one for the os, one for the wal and the
>>> other 2 for data. All disks show different access times. The WAL disk is
>>> the only one on which the with constant latency, and in certain ocasions it
>>> goes up.
>>> Plus the ocasions are absolutely random.
>>>
>>> On Mon, Jul 29, 2013 at 1:15 PM, Brett Stauner <brett(at)mightybs(dot)net>wrote:
>>>
>>>> "...like if the disk/filesystem gets slower during one hour or so"
>>>>
>>>> What about a different scheduled task on the system, not necessarily
>>>> Postgres related?
>>>>
>>>>
>>>> On Mon, Jul 29, 2013 at 9:17 AM, German Becker <german(dot)becker(at)gmail(dot)com
>>>> > wrote:
>>>>
>>>>> Alvaro,
>>>>>
>>>>> Thanks for your reply. I believe that the only possibility is
>>>>> autovacum activity I will check that. Anyway what puzzles me is that
>>>>> the throughput does not increase like i might expect if there where high
>>>>> VACUUM activity, only the WAIT TIME and thus the UTILIZATION. Is like if
>>>>> the disk/filesystem gets slower during one hour or so...
>>>>>
>>>>>
>>>>> On Fri, Jul 26, 2013 at 5:27 PM, Alvaro Herrera <
>>>>> alvherre(at)2ndquadrant(dot)com> wrote:
>>>>>
>>>>>> German Becker escribió:
>>>>>> > Hi list,
>>>>>> >
>>>>>> > I am running Postgres 9.1 on Ubuntu 12.04. I have a dedicated disk
>>>>>> for
>>>>>> > pg_xlog, using ext4 filesystem with journaling in writeback mode
>>>>>> > During high load times, the disk usage is arround 40%. The IO write
>>>>>> time is
>>>>>> > constant at about 3ms. On certain occasions roughly once in 15
>>>>>> days, the
>>>>>> > IO write time goes up to about 10ms. This makes the disk usage go
>>>>>> up to
>>>>>> > almost 100%, probably saturation, and the INSERTS DELETES UPDATES
>>>>>> run
>>>>>> > considerable slower than normal.This lasts for about 2 hours and
>>>>>> then the
>>>>>> > latency goes back to 3ms and everything is normal again.
>>>>>> > Has anyone seen this behavior? What could be causing the increase in
>>>>>> > latency?
>>>>>>
>>>>>> Can you correlate these episodes with autovacuum activity? Or perhaps
>>>>>> backups are being taken (maybe a new base backup is taken every 15
>>>>>> days)?
>>>>>>
>>>>>> --
>>>>>> Álvaro Herrera http://www.2ndQuadrant.com/
>>>>>> PostgreSQL Development, 24x7 Support, Training & Services
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>