Re: FileFallocate misbehaving on XFS

From: Michael Harris <harmic(at)gmail(dot)com>
To: Andres Freund <andres(at)anarazel(dot)de>
Cc: Jakub Wartak <jakub(dot)wartak(at)enterprisedb(dot)com>, Tomas Vondra <tomas(at)vondra(dot)me>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: FileFallocate misbehaving on XFS
Date: 2024-12-12 22:13:26
Message-ID: CADofcAVMSsbhWaRhMFd2kdx9JpeG_yqN+gbpjMMK_8K576sq4g@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi Andres

On Fri, 13 Dec 2024 at 08:38, Andres Freund <andres(at)anarazel(dot)de> wrote:
> > Another interesting snippet: the application has a number of ETL
> > workers going at once. The actual number varies depending on a number
> > of factors but might be somewhere from 10 - 150. Each worker will have
> > a single postgres backend that they are feeding data to.
>
> Are they all inserting into distinct tables/partitions or into shared tables?

The set of tables they are writing into is the same, but we do take
some effort to randomize the order of the tables that we each worker
is writing into so as to reduce contention. Even so it is quite likely
that multiple processes will be writing into a table at a time.
Also worth noting that I have only seen this error triggered by COPY
statements (other than the upgrade case). There are some other cases
in our code that use INSERT but so far I have not seen that end in an
out of space error.

> When you say that they're not "all striking it at once", do you mean that some
> of them aren't interacting with the database at the time, or that they're not
> erroring out?

Sorry, I meant erroring out.

Thanks for the patch!

Cheers
Mike

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Nathan Bossart 2024-12-12 22:14:33 Re: Proposal for Updating CRC32C with AVX-512 Algorithm.
Previous Message Nathan Bossart 2024-12-12 22:00:19 Re: connection establishment versus parallel workers