Re: Database versus filesystem for storing images

Andrew Chernow <pg-job(at)esilo(dot)com> writes:

>> And how do you guarantee that after a failure? You're restoring two
>> different sets of data here:
>
>> How do you link them together on that specific operation? Or even on a daily
>> basis, if you get corrupted data...
>
> I answered that already.

I'm sorry. It must be the flu, the pain or something else, but I really don't
remember reading your message about how you can be 100% sure that all
references to the filesystem have their corresponding files present and also
all present files have their respective database entry.

I've seen HA measures (I don't imagine anyone sacrificing their customers
copying 3 billion files and a few hundred terabytes while still maintaining an
adequate service rate with part of its infra-structure down, just to use your
example to that answer...), ideas about requiring an answer from the
filesystem before considering the transaction done DB-wise (who grants you
that the image really went to the disk and is not on cache when the machine
has a power failure and shuts down abruptly?)...

I might have missed your message, though. Would you be gentle to quote that
again, please?

> Another nice feature is the database and images can be handled spearately.

What might be bad.

> Some people have seen this as a disadvantage on this thread, I personally
> don't see it that why.

I am questioning two points that show two situations where it is bad.
Specially if those images are important to the records (e.g. product failure
images, prize winning images, product specs, prototype images, blueprints --
after all, we don't need to restrict our files to images, right? --,
agreements, spreadsheets with the last years of company account movements,
documents received from lawyers, etc.).

> I guess it depends on access needs, many files and how much data you have.
> What if you had 3 billion files across a few hundred terabytes? Can you say
> with experience how the database would hold up in this situation?

I'd have partitioning if I had a case like that. Part of those would be
delegated to one machine, part to another and so on. Even if that solution --
partitioning -- makes the overall MTBF lower...

And I still can't imagine how you guarantee that all 3 billion files have
their corresponding entries on the database. Couting them is not enough since
I can have one file with the wrong "name" present on the filesystem or some
duplicate record on the DB...

--
Jorge Godoy <jgodoy(at)gmail(dot)com>