Re: pgarchives: Bug report + Patches: loader can't handle message in multiple lists

From: Célestin Matte <celestin(dot)matte(at)cmatte(dot)me>
To: Magnus Hagander <magnus(at)hagander(dot)net>
Cc: PostgreSQL WWW <pgsql-www(at)lists(dot)postgresql(dot)org>
Subject: Re: pgarchives: Bug report + Patches: loader can't handle message in multiple lists
Date: 2023-07-13 13:36:20
Message-ID: 1f5e1598-620d-d543-e647-5916acd0a664@cmatte.me
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-www

>> Traceback (most recent call last):
>> File "/path/pgarchives/local//loader/load_message.py", line 158, in <module>
>> ap.store(conn, listid, opt.overwrite, opt.overwrite)
>> File "/path/pgarchives/local/loader/lib/storage.py", line 216, in store
>> 'listid': listid,
>> psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint "list_threads_pkey"
>> DETAIL: Key (threadid)=(21) already exists.
>
> That looks like your primary key on list_threads is wrong. The primary
> key should be on (threadid, listid) but based on that error message it
> looks like it's on (threadid).

Hmm. This is due to my deployment relying on my patch to merge schema.sql into a Django model (review still pending! [1]).
As Django does not support composite primary keys [2], I had to use unique_together instead:

migrations.CreateModel(
name='ListThreads',
fields=[
('threadid', models.IntegerField(primary_key=True, serialize=False)),
('listid', models.ForeignKey(db_column='listid', on_delete=django.db.models.deletion.CASCADE, to='mailarchives.List')),
],
options={
'db_table': 'list_threads',
'unique_together': {('threadid', 'listid')},
},
),

Which leads to the following indexes being created:
Indexes:
"list_threads_pkey" PRIMARY KEY, btree (threadid)
"list_threads_listid_44de2a94" btree (listid)
"list_threads_listid_idx" btree (listid)
"list_threads_threadid_listid_226e84bf_uniq" UNIQUE CONSTRAINT, btree (threadid, listid)

Not sure how to fix this. Inputs on previously mentioned patch would be helpful!

[1] https://www.postgresql.org/message-id/12eb75f0-3fc2-14f3-0931-4f29e145f182%40cmatte.me
[2] https://code.djangoproject.com/ticket/373
--
Célestin Matte

In response to

Responses

Browse pgsql-www by date

  From Date Subject
Next Message Stephen Frost 2023-07-13 14:16:30 Re: Undeliverable: Re: Backend handling replication slot stuck using 100% cpu, unkillable
Previous Message Célestin Matte 2023-07-13 13:03:14 Re: [PATCH] pglister: auth_receive: Indicate when PGAUTH_KEY is invalid instead of crashing