From: | Rod <cckramer(at)gmail(dot)com> |
---|---|
To: | pgsql-general(at)postgresql(dot)org |
Subject: | Storing many big files in database- should I do it? |
Date: | 2010-04-27 08:42:40 |
Message-ID: | l2kb4c00a111004270142uecd400b9r1a202ea9d154eca@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-general |
Hello,
I have a web application where users upload/share files.
After file is uploaded it is copied to S3 and all subsequent downloads
are done from there.
So in a file's lifetime it's accessed only twice- when created and
when copied to S3.
Files are documents, of different size from few kilobytes to 200
Megabytes. Number of files: thousands to hundreds of thousands.
My dilemma is - Should I store files in PGSQL database or store in
filesystem and keep only metadata in database?
I see the possible cons of using PGSQL as storage:
- more network bandwidth required comparing to access NFS-mounted filesystem ?
- if database becomes corrupt you can't recover individual files
- you can't backup live database unless you install complicated
replication add-ons
- more CPU required to store/retrieve files (comparing to filesystem access)
- size overhead, e.g. storing 1000 bytes will take 1000 bytes in
database + 100 bytes for db metadata, index, etc. with lot of files
this will be a lot of overhead.
Are these concerns valid?
Anyone had this kind of design problem and how did you solve it?
Thanks.
From | Date | Subject | |
---|---|---|---|
Next Message | Magnus Hagander | 2010-04-27 08:48:43 | Re: PostgreSQL Performance issue |
Previous Message | Nikhil G. Daddikar | 2010-04-27 08:39:16 | Re: PostgreSQL Performance issue |