Re: storing large files in database - performance

On 05/16/2017 05:25 AM, Eric Hill wrote:
> Hey,
>
> I searched and found a few discussions of storing large files in the
> database in the archives, but none that specifically address performance
> and how large of files can realistically be stored in the database.
>
> I have a node.js application using PostgreSQL to store uploaded files.
> The column in which I am storing the file contents is of type “bytea”
> with “Storage” type set to “EXTENDED”. Storing a 12.5 MB file is taking
> 10 seconds, and storing a 25MB file is taking 37 seconds. Two notable
> things about those numbers: It seems like a long time, and the time
> seems to grow exponentially with file size rather than linearly.
>
> Do these numbers surprise you? Are these files just too large for
> storage in PostgreSQL to be practical? Could there be something about
> my methodology that is slowing things down?

Yes, it does surprise me. I just tested inserting an 11MB file using
psycopg2(Python) and it was less then a second.

>
> I do have the Sequelize ORM and the pg driver in between my code and the
> database.
>
> Thanks,
>
> Eric
>

--
Adrian Klaver
adrian(dot)klaver(at)aklaver(dot)com