Re: PG vs ElasticSearch for Logs

On Sat, 20 Aug 2016, 2:00 a.m. Andy Colson, <andy(at)squeakycode(dot)net> wrote:

> On 8/19/2016 2:32 AM, Thomas Güttler wrote:
> > I want to store logs in a simple table.
> >
> > Here my columns:
> >
> > Primary-key (auto generated)
> > timestamp
> > host
> > service-on-host
> > loglevel
> > msg
> > json (optional)
> >
> > I am unsure which DB to choose: Postgres, ElasticSearch or ...?
> >
> > We don't have high traffic. About 200k rows per day.
> >
> > My heart beats for postgres. We use it since several years.
> >
> > On the other hand, the sentence "Don't store logs in a DB" is
> > somewhere in my head.....
> >
> > What do you think?
> >
> >
> >
>
> I played with ElasticSearch a little, mostly because I wanted to use
> Kibana which looks really pretty. I dumped a ton of logs into it, and
> made a pretty dashboard ... but in the end it didn't really help me, and
> wasn't that useful. My problem is, I don't want to have to go look at
> it. If something goes bad, then I want an email alert, at which point
> I'm going to go run top, and tail the logs.
>

There are tools from Elastic Stack which could have helped you achieve
email alerts and gather top or tailing of logfile

> Another problem I had with kibana/ES is the syntax to search stuff is
> different than I'm used to. It made it hard to find stuff in kibana.
>
> Right now, I have a perl script that reads apache logs and fires off
> updates into PG to keep stats. But its an hourly summary, which the
> website turns around and queries the stats to show pretty usage graphs.
>
> In the end, PG or ES, all depends on what you want.
>
> -Andy
>
>
> --
> Sent via pgsql-general mailing list (pgsql-general(at)postgresql(dot)org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-general
>
--
--
Best Regards
Sameer Kumar | DB Solution Architect
*ASHNIK PTE. LTD.*

101 Cecil Street, #11-11 Tong Eng Building, Singapore 069 533

T: +65 6438 3504 | M: +65 8110 0350

Skype: sameer.ashnik | www.ashnik.com