Re: renice on an I/O bound box

On Tue, Jan 19, 2010 at 3:09 PM, Jean-David Beyer <jeandavid8(at)verizon(dot)net>wrote:

> Willy-Bas Loos wrote:
>
>
>>
>> On Tue, Jan 19, 2010 at 2:28 PM, Jean-David Beyer <jeandavid8(at)verizon(dot)net<mailto:
>> jeandavid8(at)verizon(dot)net>> wrote:
>>
>> It could make sense.
>>
>> I once had a job populating a database. It was I/O bound and ran for a
>> couple of hours. In those days, it was on a machine shared with
>> developers of other software, and running my job interfered a lot with
>> their work. I tried running it "nice" and this slowed my job down a lot
>> so it took something like 16 hours. But it did not help the other users
>> because my job was screwing the disk cache in memory that much longer.
>> It worked out best for everyone to run my job at normal priority, but
>> to
>> start it just before quitting time so it was nearly finished by the
>> time
>> they came back to work.
>>
>> There was a better way to solve the problem (use raw IO for the dbms),
>> but in those days they would not let me do that. I had done it before
>> when I was working in another department and it worked just fine.
>>
>> It is my understanding that postgreSQL does not use raw IO because it
>> does not help all that much. With my current machine with 8 GB RAM (6
>> GB
>> of which is is cache) and a relatively small database, this seems to be
>> true.
>>
>>
>> Did you mean to not send that to the list?
>>
>
> I meant to send it to the list. Most of the lists I am on do that
> automatically when I press Reply. A few do not, and I forget which is
> which.

Ok, back on the list now.

>
>
> Seems like what you are saying is that in your case, it didn't work out.
>> The cache thing is a good one to remember.
>>
>> How could it make sense then?
>>
>
> Because the process screws up the disk cache, and that takes time to
> recover. At which point your process runs again and messes up the cache
> again.
>
> I have a similar problem now. I run a lot of BOINC processes at the
> lowest priority (nice 19) to eat up all the processor cycles. These
> normally do not interfere with anything. But when I run a postgreSQL
> process, it runs slowly because whenever postgreSQL pauses for IO, the
> BOINC stuff runs and invalidates (in this case, the memory L1, L2, and
> L3) cache, so when postgreSQL gets a processor, all the memory caches
> are invalid and the machine runs at RAM speeds (533MHz) instead of cache
> speed (3.06 GHz). So if I turn off the BOINC processes, that do not
> compete for processor cycles, but do compete for cache slots, the
> postgreSQL process runs faster.
>

So to me it *doesn't* make sense, renicing the process in the case that you
describe.
Probably hard to detect too.

Would things be better if the processes would compete for CPU cycles? Or
equally bad, only you can see it reflected in the load?

Is it generally bad to renice postgres processes, unless you are CPU bound?
(and even then..)

Cheers,

WBL

>
>

> --
> .~. Jean-David Beyer Registered Linux User 85642.
> /V\ PGP-Key: 9A2FC99A Registered Machine 241939.
> /( )\ Shrewsbury, New Jersey http://counter.li.org
> ^^-^^ 09:00:01 up 11 days, 10:55, 3 users, load average: 4.34, 4.42, 4.43
>

--
"Patriotism is the conviction that your country is superior to all others
because you were born in it." -- George Bernard Shaw