Re: storing large files in database - performance

From: Thomas Kellerer <spam_eater(at)gmx(dot)net>
To: pgsql-general(at)postgresql(dot)org
Subject: Re: storing large files in database - performance
Date: 2017-05-16 14:35:01
Message-ID: off2mg$ies$1@blaine.gmane.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

Eric Hill schrieb am 16.05.2017 um 14:25:
> I have a node.js application using PostgreSQL to store uploaded
> files. The column in which I am storing the file contents is of type
> “bytea” with “Storage” type set to “EXTENDED”. Storing a 12.5 MB file
> is taking 10 seconds, and storing a 25MB file is taking 37 seconds.
> Two notable things about those numbers: It seems like a long time,
> and the time seems to grow exponentially with file size rather than
> linearly.
>
> Do these numbers surprise you?

When my (JDBC based) SQL client and the database server are on the same computer, I can store a 45MB file about 4 seconds, a 240MB file in about 20seconds
When I do the same with a server on the (same) network, the 45MB take about 9 seconds, the 240MB take about 60 seconds.

So yes, these numbers sound a bit surprising.

> Could there be something about my methodology that is slowing things down?

How far are your app server and the database server apart?
Maybe it's just a case of a slow network

Thomas

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Adrian Klaver 2017-05-16 14:36:41 Re: storing large files in database - performance
Previous Message Martin Goodson 2017-05-16 14:22:14 Re: EnterpriseDB installed PostgreSQL 9.6 vs. REPMGR. Round 4 - compilation issues on RHEL 7.2