Re: Caching and Blobs in PG? Was: Can PG replace redis, amqp, s3 in the future?

On Wed, May 3, 2017 at 3:57 AM, Thomas Güttler <guettliml(at)thomas-guettler(dot)de
> wrote:

> Am 02.05.2017 um 05:43 schrieb Jeff Janes:
>
>> On Sun, Apr 30, 2017 at 4:37 AM, Thomas Güttler <
>> guettliml(at)thomas-guettler(dot)de <mailto:guettliml(at)thomas-guettler(dot)de>>
>> wrote:
>>
>> Is is possible that PostgreSQL will replace these building blocks in
>> the future?
>>
>> - redis (Caching)
>>
>>
>> PostgreSQL has its own caching. It might not be quite as effective as
>> redis', but you can us it if you are willing to
>> take those trade offs.
>>
>
> What kind of caching does PG offer?
>

It has shared_buffers to cache the data it needs frequently (not query
results, but the data needed to produce the results), and also uses the
file systems cache. This is what I am referring to. I wouldn't recommend
using PostgreSQL simply as a cache for something else, if you don't want
any other features of the database. But if you want to throw Redis up as a
layer of cache in front of PostgreSQL, maybe you should first see if that
RAM, and a bit of tuning, can be used to make PostgreSQL fast enough to not
need the Redis cache.

>
>>
>>
>> - s3 (Blob storage)
>>
>>
>>
>
> No. You can certainly use PostgreSQL to store blobs. But then, you need
>> to store the PostgreSQL data **someplace**.
>> If you don't store it in S3, you have to store it somewhere else.
>>
>
> I don't understand what you mean here. AFAIK storing blobs in PG is not
> recommended since it is not very efficient.
>

If the metadata is stored in PG and the blobs themselves are stored
individually S3, you have a transaction atomicity problem. Solving that is
not likely to be very efficient, either. You have to pick your poison.

Cheers,

Jeff