Re: storing large files in database - performance

From: "Peter J(dot) Holzer" <hjp-pgsql(at)hjp(dot)at>
To: pgsql-general(at)postgresql(dot)org
Subject: Re: storing large files in database - performance
Date: 2017-05-16 16:14:40
Message-ID: 20170516161440.GE20574@hjp.at
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

On 2017-05-16 12:25:03 +0000, Eric Hill wrote:
> I searched and found a few discussions of storing large files in the database
> in the archives, but none that specifically address performance and how large
> of files can realistically be stored in the database.
>
>
>
> I have a node.js application using PostgreSQL to store uploaded files. The
> column in which I am storing the file contents is of type “bytea” with
> “Storage” type set to “EXTENDED”.

I have mentioned this little experiment before, but I finally put the
results on my web site: https://www.hjp.at/databases/blob-bench/

(Please note that so far I have run this only on one system.
Generalizing to other systems might be premature).

> Storing a 12.5 MB file is taking 10 seconds, and storing a 25MB file
> is taking 37 seconds. Two notable things about those numbers: It
> seems like a long time, and the time seems to grow exponentially with
> file size rather than linearly.
>
>
>
> Do these numbers surprise you?

Yes. on my system, storing a 25 MB bytea value takes well under 1 second.

hp

--
_ | Peter J. Holzer | we build much bigger, better disasters now
|_|_) | | because we have much more sophisticated
| | | hjp(at)hjp(dot)at | management tools.
__/ | http://www.hjp.at/ | -- Ross Anderson <https://www.edge.org/>

In response to

Browse pgsql-general by date

  From Date Subject
Next Message Martin Goodson 2017-05-16 16:31:41 Re: EnterpriseDB installed PostgreSQL 9.6 vs. REPMGR. Round 4 - compilation issues on RHEL 7.2
Previous Message Tom Lane 2017-05-16 15:51:58 Re: Different query plan used for the same query depending on how parameters are passed