Re: Caching Websites

On Friday 09 May 2003 9:48 pm, Adam Kessel wrote:
> I am wondering whether it would be better to store each website in a
> record in a table, or instead have a table which links URLs to filenames
> (the file would contain the pickled website). The sites will of course
> vary greatly in size, but typically be between 1k and 200k (I probably
> won't store anything bigger than that).
>
> This seems like a simple question, and I suspect there's an obvious
> answer for which data storage method makes more sense, I just don't know
> how to go about researching that. What would be the considerations for
> using one method of data storage vs. the other?
>
> Any suggestions for me?

Not that simple a question - look back through the archives for plenty of
discussions (usually regarding images).

My personal approach is to ask myself whether I'm going to access/process the
data in any way. Basically if I want to do any of:
1. query the large data
2. summarise it
3. have transaction-based update control
then I'll store it in the database. If not, I'll store a path to the file.

--
Richard Huxton