Re: inserting huge file into bytea cause out of memory

From: Chris Travers <chris(dot)travers(at)gmail(dot)com>
To: liuyuanyuan <liuyuanyuan(at)highgo(dot)com(dot)cn>
Cc: Michael Paquier <michael(dot)paquier(at)gmail(dot)com>, pgsql-general <pgsql-general(at)postgresql(dot)org>, "laurenz(dot)albe" <laurenz(dot)albe(at)wien(dot)gv(dot)at>
Subject: Re: inserting huge file into bytea cause out of memory
Date: 2013-08-08 02:41:58
Message-ID: CAKt_Zfs5kGJ85KMb-iAYwsv2z-OZN-WYikCn_AefUYpU5Rmb-Q@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

On Wed, Aug 7, 2013 at 6:41 PM, liuyuanyuan <liuyuanyuan(at)highgo(dot)com(dot)cn>wrote:

> **
>
>
> Thanks for your last reply!
> I've test Large Object ( oid type ), and it seems better on out of
> memory.
> But, for the out of memory problem of bytea, we really have no idea
> to
> solve it ? Why there's no way to solve it ? Is this a problem of JDBC ,or
> the type itself ?
>

I think the big difficulty efficiency-wise is in the fact that everything
is exchanged in a textual representation. This means you have likely at
least two representations in memory on the client and the server, and maybe
more depending on the client framework, and the textual representation is
around twice as large as the binary one. Add to this the fact that it must
all be handled at once and you have difficulties which are inherent to the
implementation. In general, I do not recommend byteas for large amounts of
binary data for that reason. If your files are big, use lobs.

Best Wishes,
Chris Travers

>
> Yours,
> Liu Yuanyuan
>

--
Best Wishes,
Chris Travers

Efficito: Hosted Accounting and ERP. Robust and Flexible. No vendor
lock-in.
http://www.efficito.com/learn_more.shtml

In response to

Browse pgsql-general by date

  From Date Subject
Next Message Tom Lane 2013-08-08 04:06:42 Re: Adding ip4r to Postgresql core?
Previous Message Sergey Konoplev 2013-08-08 02:13:08 Re: Self referencing composite datatype