From: | "Graeme B(dot) Bell" <graeme(dot)bell(at)nibio(dot)no> |
---|---|
To: | Carlo <reg01(at)stonebanks(dot)ca> |
Cc: | "pgsql-performance(at)postgresql(dot)org" <pgsql-performance(at)postgresql(dot)org> |
Subject: | Re: One long transaction or multiple short transactions? |
Date: | 2015-10-08 08:54:55 |
Message-ID: | F545968D-7C4A-41CE-B54D-203F1D4D9FEB@skogoglandskap.no |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-performance |
Sounds like a locking problem, but assuming you aren’t sherlock holmes and simply want to get the thing working as soon as possible:
Stick a fast SSD in there (whether you stay on VM or physical). If you have enough I/O, you may be able to solve the problem with brute force.
SSDs are a lot cheaper than your time.
Suggest you forward this to your operators: a talk I have about optimising multi-threaded work in postgres:
http://graemebell.net/foss4gcomo.pdf (Slides: “Input/Output” in the middle of the talk and also the slides at the end labelled “For Techies")
Graeme Bell
p.s. You mentioned a VM. Consider making the machine physical and not VM. You’ll get a performance boost and remove the risk of DB corruption from untrustworthy VM fsyncs. One day there will be a power cut or O/S crash during these your writes and with a VM you’ve a reasonable chance of nuking your DB because VM virtualised storage often doesn’t honour fsync (for performance reasons), but it’s fundamental to correct operation of PG.
> On 08 Oct 2015, at 01:40, Carlo <reg01(at)stonebanks(dot)ca> wrote:
>
>
> I am told 32 cores on a LINUX VM. The operators have tried limiting the number of threads. They feel that the number of connections is optimal. However, under the same conditions they noticed a sizable boost in performance if the same import was split into two successive imports which had shorter transactions.
>
From | Date | Subject | |
---|---|---|---|
Next Message | Bram Van Steenlandt | 2015-10-08 09:17:49 | large object write performance |
Previous Message | Carlo | 2015-10-07 23:40:31 | Re: One long transaction or multiple short transactions? |