Re: Bulk INSERT performance in 7.4.1

Vivek Khera wrote:

> If you've got the time, could you try also doing the full bulk insert
> test with the checkpoint log files on another physical disk? See if
> that's any faster.

We have been doing that for a few weeks, but the performance
improvements are less than what we expected. There is hardly any disk
activity on the log RAID, even during checkpointing.

After I activated the tuned configuration, we are again mostly CPU-bound
(it seems that updating all four indices is quite expensive). The
bulk INSERT process runs single-threaded right now, and if we switched
to multiple processes for that, we could reach some 1,500 INSERTs per
second, I believe. This is more than sufficient for us; our real-time
data collector is tuned to emit about 150 records per second, on the
average. (There is an on-disk queue to compensate temporary problems,
such as spikes in the data rate and database updates gone awry.)

--
Current mail filters: many dial-up/DSL/cable modem hosts, and the
following domains: atlas.cz, bigpond.com, freenet.de, hotmail.com,
libero.it, netscape.net, postino.it, tiscali.co.uk, tiscali.cz,
tiscali.it, voila.fr, wanadoo.fr, yahoo.com.