Re: PG vs ElasticSearch for Logs

From: Thomas Güttler <guettliml(at)thomas-guettler(dot)de>
To: pgsql-general(at)postgresql(dot)org
Subject: Re: PG vs ElasticSearch for Logs
Date: 2016-08-19 08:57:51
Message-ID: 9de601b1-1e51-0207-45a4-743be00574b7@thomas-guettler.de
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

Am 19.08.2016 um 09:42 schrieb John R Pierce:
> On 8/19/2016 12:32 AM, Thomas Güttler wrote:
>> What do you think?
>
> I store most of my logs in flat textfiles syslog style, and use grep for adhoc querying.
>
> 200K rows/day, thats 1.4 million/week, 6 million/month, pretty soon you're talking big tables.
>
> in fact thats several rows/second on a 24/7 basis

There is no need to store them more then 6 weeks in my current use case.

I think indexing in postgres is much faster than grep.

And queries including json data are not possible with grep (or at least very hard to type)

My concern is which DB (or indexing) to use ...

Regards,
Thomas

--
Thomas Guettler http://www.thomas-guettler.de/

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Sameer Kumar 2016-08-19 09:16:42 Re: Limit Heap Fetches / Rows Removed by Filter in Index Scans
Previous Message John R Pierce 2016-08-19 07:42:09 Re: PG vs ElasticSearch for Logs