Re: Storing images in PostgreSQL databases (again)

Michelle Konzack <linux4michelle(at)freenet(dot)de> wrote:

> I do this already but have problems since I have
> stored arround 130 million files on a server...
>
> ...
>
> MD5 hashes are 32 Bytes long, maybe they change
> it to 64 Bytes?
>
> I have already over 2000 collisions and checked
> it, that the files are NOT the same.

You mean you have 2000 collisions out of the checksums of 130 million
different files? That can't be right.

An MD5 hash is 128 bits, and using the values found in
<http://en.wikipedia.org/wiki/Birthday_attack>, you don't reach a 50%
chance of a single collision until you've checksummed 2.2 x 10^19
different inputs. That's, ummm, 22,000,000,000,000,000,000, I think,
which is much larger than 130,000,000.

In other words, you should not expect even a single collision until you
have 169,230,769,231 times as many files as you currently have, which
should not be a issue before the end of the useful life of the solar system.

If you have 2000 collisions after 130 million different files (or even
if you have two collisions), something is almost certainly wrong with
your code, unfortunately.

--
Robert L Mathews

"The trouble with doing something right the first time is
that nobody appreciates how difficult it was."