Re: used for large media files

On Jun 17, 2009, at 8:43 AM, Mike Kay wrote:

> Now that's an interesting way of doing this I never thought about
> before.
> Using a fileserver though, how would I categorize and index the files?
>
> I was planning on using multiple databases to hold the data - one
> for each
> client and a separate database for each file type. Yes, they would be
> hosted on the same server. I see the bottleneck.
>
> I suppose that instead of saving the files, indexes and categories
> all in
> the same database, I could simply reference the location and file
> names in
> the database - and index and categorize in this manner. Does this make
> sense?

Storing all the metadata in the database and the content on the
filesystem
of the webserver lets both do what they're good at.

Serving static files from the filesystem of the webserver is
ridiculously
cheap compared with retrieving the data from the database, as it's
something that everything from the kernel up is optimized to do.
Backups are much simpler too.

All categorization and suchlike is done via the database, with the
only time you hit the main filesystem being when you want to serve
the media file itself to the user.

The only structure you really need for the static files is something
that
makes accessing them reasonably cheap, which is mostly driven by
limiting the number of files or subdirectories in each directory to a
reasonable number. The names of the files and the directory structure
don't really need to be meaningful.

I've done something similar and added some level of transaction
safety to removing files by having any deletion handled through
the database, using triggers to add the filename of any object that
was removed to a queue table. Then an external process polls that
queue table and removes the actual file.

Cheers,
Steve