Re: Duplicated IDs

On 09 Aug 2014, at 11:38, Alexis Bernard <alexis(at)bernard(dot)io> wrote:

> Hi,
>
> I am having regurlarly duplicated rows with the same primary key.
>
> => select id, created_at, updated_at from tracks where created_at = '2014-07-03 15:09:16.336488';
> id | created_at | updated_at
> --------+----------------------------+----------------------------
> 331804 | 2014-07-03 15:09:16.336488 | 2014-07-03 15:37:55.253729
> 331804 | 2014-07-03 15:09:16.336488 | 2014-07-03 15:37:55.253801
>
> => select id from tracks where id = 331804;
> id
> ----
> (0 rows)
>

First of all, what is the definition of that primary key?
What exact version of PG are you using?: select version();
What exact OS is this on? What kind of storage?

To me it looks like you may have run into transaction wrap-around or a corrupted index.

Before you do anything, make a backup.

Theorising that the issue here indeed is transaction wrap-around, what you’re seeing may be data from older transactions that has become newer because your current transaction txid is lower (due to the wraparound) than the txid of the transactions those rows belong(ed) to. If those transactions were committed, then you’re possibly seeing deleted or updated rows that are still around. TX wraparound can occur if you do not vacuum frequently enough and another thing that vacuum does is mark old rows obsolete so that the DB can reclaim the space they use. Seeing data from rows that are no longer there or that has been modified since seems to fit the bill here.

Hence the question: When did you last (auto-)vacuum this table? Did you perhaps turn autovacuum off? Did it fall behind?

If you have been vacuuming and the issue is a corrupt index: Does it help to reindex that table? You said that you have fsync on; what kind of storage is this database on? Something allowed that index to get corrupted. It is more likely that it’s caused by something in the underlying storage system (including the OS) than that it is a bug in PG.

> => delete from tracks where created_at = '2014-07-03 15:09:16.336488' and updated_at = '2014-07-03 15:37:55.253801';
> ERROR: update or delete on table "tracks" violates foreign key constraint "fk_sources_on_track_id" on table "sources"
> DETAIL: Key (id)=(331804) is still referenced from table "sources”.

Apparently there is a row from another table referencing this one. So either the referenced row does actually exist (corrupt index theory) or it doesn’t and the referencing row is from an older transaction as well (TX wraparound theory).

Considering that you’re seeing this regularly, my bet is on TX wraparound.

Alban Hertroys
--
If you can't see the forest for the trees,
cut the trees and you'll find there is no forest.