Re: PostgreSQL as advanced job queuing system

From: Thiemo Kellner <thiemo(at)gelassene-pferde(dot)biz>
To: pgsql-general(at)lists(dot)postgresql(dot)org
Subject: Re: PostgreSQL as advanced job queuing system
Date: 2024-03-22 16:17:10
Message-ID: 0da7da15-9155-4b53-9d6d-d289fc0986c7@gelassene-pferde.biz
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

Am 22.03.2024 um 14:15 schrieb Fred Habash:
> We developed a home-grown queue system using Postgres, but its
> performance was largely hindered by que tables bloating and the need to
> continuously vacuum them. It did not scale whatsoever. With some
> workarounds, we ended up designing three sets of queue tables, switching
> between them based on some queue stats, vacuum the inactive set, and repeat.
> We kept this queue system for low SLA app components. For others, we
> switched to Kafka. Investing in learning and implementing purpose built
> queue systems pays off for the long term.

I wonder whether one should https://youtu.be/VEWXmdjzIpQ&t=543 not to
scale either.

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Fred Habash 2024-03-22 16:25:16 Timing out A Blocker Based on Time or Count of Waiters
Previous Message Alban Hertroys 2024-03-22 16:00:43 Re: could not open file "global/pg_filenode.map": Operation not permitted