From: | Tomas Vondra <tomas(dot)vondra(at)enterprisedb(dot)com> |
---|---|
To: | Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> |
Cc: | Bharath Rupireddy <bharath(dot)rupireddyforpostgres(at)gmail(dot)com>, "houzj(dot)fnst(at)fujitsu(dot)com" <houzj(dot)fnst(at)fujitsu(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org> |
Subject: | Re: Fdw batch insert error out when set batch_size > 65535 |
Date: | 2021-06-09 10:23:45 |
Message-ID: | 0b3fb49c-fc3a-ad6c-4a98-1cc64936d4f8@enterprisedb.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
On 6/9/21 12:22 PM, Tomas Vondra wrote:
>
>
> On 6/9/21 8:28 AM, Tom Lane wrote:
>> I wrote:
>>> Bharath Rupireddy <bharath(dot)rupireddyforpostgres(at)gmail(dot)com> writes:
>>>>> I've added a simple regression test to postgres_fdw, testing that
>>>>> batch
>>>>> sizes > 65535 work fine, and pushed the fix.
>>
>>>> I was earlier thinking of adding one, but stopped because it might
>>>> increase the regression test execution time. It looks like that's true
>>>> - with and without the test case it takes 17 sec and 4 sec
>>>> respectively on my dev system which is 4X slower. I'm not sure if this
>>>> is okay.
>>
>>> The cost, versus the odds of ever detecting a problem, doesn't
>>> seem like a good tradeoff.
>>
>> I took a quick look and noted that on buildfarm member longfin
>> (to take a random example that's sitting a few feet from me),
>> the time for contrib-install-check went from 34 seconds before
>> this patch to 40 seconds after. I find that completely
>> unacceptable compared to the likely value of this test case.
>>
>
> Note that the problem here is [1] - we're creating a lot of slots
> referencing the same tuple descriptor, which inflates the duration.
> There's a fix in the other thread, which eliminates ~99% of the
> overhead. I plan to push that fix soon (a day or two).
>
Forgot to add the link:
[1]
https://www.postgresql.org/message-id/ebbbcc7d-4286-8c28-0272-61b4753af761%40enterprisedb.com
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
From | Date | Subject | |
---|---|---|---|
Next Message | Hassan Camacho Cadre | 2021-06-09 10:30:15 | How to pass a parameter in a query to postgreSQL 11 |
Previous Message | Tomas Vondra | 2021-06-09 10:22:15 | Re: Fdw batch insert error out when set batch_size > 65535 |