Re: Sustained inserts per sec ... ?

On Apr 4, 2005 11:57 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Greg Stark <gsstark(at)mit(dot)edu> writes:
> > All this is happening within a single transaction too, right? So there hasn't
> > been an fsync the entire time. It's entirely up to the kernel when to decide
> > to start writing data.
>
> No ... there's a commit every 500 records. However, I think Chris said
> he was running with fsync off; so you're right that the kernel is at
> liberty to write stuff to disk when it feels like. It could be that
> those outlier points are transactions that occurred in the middle of
> periodic syncer-driven mass writes. Maybe fsync off is
> counterproductive for this situation?

Looking at preliminary results from running with shared_buffers at
16000, it seems this may be correct. Performance was flatter for a
BIT longer, but slammed right into the wall and started hitting the
3-30 second range per COPY. I've restarted the run, with fsync turned
on (fdatasync), and we'll see.

My fear is that it's some bizarre situation interacting with both
issues, and one that might not be solvable. Does anyone else have
much experience with this sort of sustained COPY?

Chris
--
| Christopher Petrilli
| petrilli(at)gmail(dot)com