From: | Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com> |
---|---|
To: | Tomas Vondra <tomas(dot)vondra(at)enterprisedb(dot)com> |
Cc: | John Naylor <john(dot)naylor(at)enterprisedb(dot)com>, jitesh tiwari <jitesh120(at)gmail(dot)com>, pgsql-bugs(at)postgresql(dot)org |
Subject: | Re: fetching bytea (blob) data of 850 MB from psql client failed |
Date: | 2022-08-31 15:40:22 |
Message-ID: | CAFj8pRBdHYSh5cGxs1NWd5yGAkb+E9gdBW3002DEs04y1VEGMg@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-bugs |
st 31. 8. 2022 v 15:20 odesílatel Tomas Vondra <
tomas(dot)vondra(at)enterprisedb(dot)com> napsal:
>
>
> On 8/31/22 09:27, John Naylor wrote:
> > On Mon, Aug 29, 2022 at 1:02 PM jitesh tiwari <jitesh120(at)gmail(dot)com>
> wrote:
> >>
> >> Hi John,
> >> Thanks for those details.
> >> The database table for bytea columns has binary data data and some are
> not text. Hence, when it tried to fetch by changing the bytea_output to
> 'escape' it failed again with the below error -
> >> VERROR; result of bytea output conversion is too large(File varlena.c;
> Line 415; Routine byteaout; ).
> >> BTW, is there any plan to fix this issue in the PostgreSQL backend code
> in near future?
> >
> > Unfortunately, there is no plan as there is no consensus on how to
> > approach the issue. It seems the possible actions are (easiest to
> > hardest):
> >
> > 1. Document that bytea is unsafe -- you can insert anything you like,
> > but you may be unable to retrieve it again.
> > 2. Determine some maximum size that guarantees accessibility and
> > invent a GUC that by default prevents inserting larger values than
> > that.
> > 3. Fix the issue properly.
> >
> > #2 and #3 are challenging for reasons given in the thread I linked to
> above.
> >
>
> I haven't tried, but wouldn't it be enough to fetch the data in smaller
> chunks? The application would have to re-assemble that, of course.
>
>
yes - there are possible both direction conversions to LO. So anybody can
convert any bytea to temp LO, and then it can download to any client
Regards
Pavel
>
> regards
>
> --
> Tomas Vondra
> EnterpriseDB: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>
>
>
From | Date | Subject | |
---|---|---|---|
Next Message | Bruce Momjian | 2022-09-01 02:05:46 | Re: BUG #17349: Function gen_random_uuid duplicates in pgcrypto and core |
Previous Message | Tomas Vondra | 2022-08-31 13:20:29 | Re: fetching bytea (blob) data of 850 MB from psql client failed |