Re: Shared buffers, db transactions commited, and write IO on Solaris

On Apr 3, 2007, at 11:51 AM, Dimitri wrote:

>> Well, to check if there is a real potential gain all we need is a
>> small comparing test using PgSQL compiled with LOG block size
>> equal to
>> say 1K and direct IO enabled.
>>
>> Rgds,
>> -Dimitri
>>
>>
>> On 3/30/07, Kenneth Marshall <ktm(at)rice(dot)edu> wrote:
>> > On Fri, Mar 30, 2007 at 04:25:16PM +0200, Dimitri wrote:
>> > > The problem is while your goal is to commit as fast as
>> possible - it's
>> > > pity to vast I/O operation speed just keeping common block
>> size...
>> > > Let's say if your transaction modification entering into 512K
>> - you'll
>> > > be able to write much more 512K blocks per second rather 8K
>> per second
>> > > (for the same amount of data)... Even we rewrite probably several
>> > > times the same block with incoming transactions - it still
>> costs on
>> > > traffic, and we will process slower even H/W can do better. Don't
>> > > think it's good, no? ;)
>> > >
>> > > Rgds,
>> > > -Dimitri
>> > >
>> > With block sizes you are always trading off overhead versus space
>> > efficiency. Most OS write only in 4k/8k to the underlying hardware
>> > regardless of the size of the write you issue. Issuing 16 512byte
>> > writes has much more overhead than 1 8k write. On the light
>> transaction
>> > end, there is no real benefit to a small write and it will slow
>> > performance for high throughput environments. It would be better
>> to,
>> > and I think that someone is looking into, batching I/O.
>> >
>> > Ken
>> >
>>
> Folks,
>
> to close topic with "LOG block size=1K" idea - I took a time to test
> it (yes) and in best cases there is only 15% gain comparing to 8K -
> storage protocol is quite heavy itself, so less or more data sent
> within it doesn't reduce service time too much... As well even this
> gain is quickly decreasing with growing workload! So, yes 8K is good
> enough and probably the most optimal choice for LOG (as well data)
> block size.
>
> Rgds,
> -Dimitri

Hey, man, thanks for taking the time to profile that!

erik jones <erik(at)myemma(dot)com>
software developer
615-296-0838
emma(r)