corruption in indexes under heavy load

From: Russell Keane <Russell(dot)Keane(at)inps(dot)co(dot)uk>
To: "pgsql-general(at)postgresql(dot)org" <pgsql-general(at)postgresql(dot)org>
Subject: corruption in indexes under heavy load
Date: 2016-08-25 08:50:54
Message-ID: 8D0E5D045E36124A8F1DDDB463D54855BFD24F36A7@mxsvr1.is.inps.co.uk
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

We're writing a large amount of data to a number of tables in PG 9.3 on Windows Server 2012 R2 and then, immediately after, creating a number of indexes (there are no indexes during the initial data write). The data we're writing exists in files on the same drive as PG's data.
During the index creation we're seeing write latency on the storage up to 1500ms (during the data write it may go up to 200ms, possibly due to reading the data from the same storage). The write latency is being investigated separately as there's no way it should ever get that high.

There is no indication in the PG log that anything is amiss while creating the indexes and there are no clients accessing or updating the data at this time, although there are a number of instances of "pgstat wait timeout" around the time of the high write latency. Therefore we must assume that the indexes were created successfully.

We're fairly convinced the issue lies with the actual storage but I was wondering if there is anything within PG that would be affected by the high latency and result in corrupt indexes.

Regards,
Russell

________________________________
Registered name: In Practice Systems Ltd.
Registered address: The Bread Factory, 1a Broughton Street, London, SW8 3QJ
Registered Number: 1788577
Registered in England
The information in this internet email is confidential and is intended solely for the addressee. Access, copying or re-use of information in it by anyone else is not authorised. Any views or opinions presented are solely those of the author and do not necessarily represent those of INPS or any of its affiliates. If you are not the intended recipient please contact is(dot)helpdesk(at)inps(dot)co(dot)uk

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Simon Riggs 2016-08-25 09:00:11 Re: corruption in indexes under heavy load
Previous Message Francisco Olarte 2016-08-25 07:29:47 Re: ON CONFLICT does not support deferrable unique constraints