Re: 42.0.0.jre7, driver logging takes 30-40% of my server's time

From: Dave Cramer <pg(at)fastcrypt(dot)com>
To: Konstantinos Kougios <kostas(dot)kougios(at)googlemail(dot)com>
Cc: Daniel Migowski <dmigowski(at)ikoffice(dot)de>, Jorge Solórzano <jorsol(at)gmail(dot)com>, List <pgsql-jdbc(at)postgresql(dot)org>
Subject: Re: 42.0.0.jre7, driver logging takes 30-40% of my server's time
Date: 2017-03-23 13:23:05
Message-ID: CADK3HHKKLKaT7q_szUoDkGZmgW3MDWy7FbZjMYFP0_rfh_Yn5g@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-jdbc

So as someone who doesn't use logging in this way. Is having
java.util.logging useful ? The old logging mechanism didn't have these
problems

Dave Cramer

davec(at)postgresintl(dot)com
www.postgresintl.com

On 23 March 2017 at 09:21, Konstantinos Kougios <
kostas(dot)kougios(at)googlemail(dot)com> wrote:

> Hi, after configuring propagator, the time it takes to run some heavy
> queries dropped from 5s to 1.7s !
>
> Indeed a boolean will even make things faster, so I will update as soon as
> it is ready, thanks
>
>
> On 23/03/17 13:19, Daniel Migowski wrote:
>
> Hello,
>
>
>
> At first: Did you configure the Level Change Propagator?
> https://logback.qos.ch/manual/configuration.html
>
>
>
> Also: I am currently working on a patch that does exactly that for
> ResultSets, because not only can evaluating the log level be very slow when
> not using the same logging framework than the intented one, but also most
> of the log statements in PgConnection and PgResultSet have hidden object
> creation issues because primitive types are passed to calls that expect
> objects.
>
>
>
> I would implement that in a way where the log level is only evaluated at
> query start, and then cached until the next query. Should be good enough
> for most scenarios and make the driver 5% faster out of the box. Expect
> some substantial performance improvements from this.
>
>
>
> I will need a few days to be able to have this ready for a pull request,
> please standby.
>
>
>
> Regards,
>
> Daniel Migowski
>
>
>
> *Von:* pgsql-jdbc-owner(at)postgresql(dot)org [mailto:pgsql-jdbc-owner@
> postgresql.org <pgsql-jdbc-owner(at)postgresql(dot)org>] *Im Auftrag von *Konstantinos
> Kougios
> *Gesendet:* Mittwoch, 22. März 2017 18:35
> *An:* Dave Cramer <pg(at)fastcrypt(dot)com> <pg(at)fastcrypt(dot)com>
> *Cc:* Jorge Solórzano <jorsol(at)gmail(dot)com> <jorsol(at)gmail(dot)com>; List
> <pgsql-jdbc(at)postgresql(dot)org> <pgsql-jdbc(at)postgresql(dot)org>
> *Betreff:* Re: [JDBC] 42.0.0.jre7, driver logging takes 30-40% of my
> server's time
>
>
>
> It continues. It must be a misconfig of logback+common logging
>
> On 22/03/17 17:32, Dave Cramer wrote:
>
> I'm confused when you say it passes this, does it return or continue after
> this if statement ?
>
>
> Dave Cramer
>
> davec(at)postgresintl(dot)com
>
> www.postgresintl.com
>
>
>
> On 22 March 2017 at 13:24, Konstantinos Kougios <
> kostas(dot)kougios(at)googlemail(dot)com> wrote:
>
> I am using logback and I don't see any postgres-driver related logging, so
> I assume it should be off
>
> Looking at the stacktrace of the profile though, it seems it passes this
> code:
>
> if (!isLoggable(level)) {
>
> return;
>
> }
>
>
> So it seems as if it is ON though it doesn't log anything.
>
>
>
> On 22/03/17 17:21, Jorge Solórzano wrote:
>
> Have you enabled the Logger with a FINEST (TRACE) level?
>
>
>
> On Wed, Mar 22, 2017 at 11:13 AM, Konstantinos Kougios <
> kostas(dot)kougios(at)googlemail(dot)com> wrote:
>
> I tried to paste an image with a profile, but my email was rejected.
>
> It seems logging is called in places like row.getInt(), row.getString()
> etc without a check for the logging level. Then it takes a lot of time for
> those to be processed, especially for queries that get a lot of data. It
> goes down to parsing strings and locale and it just is slow. I see postgres
> processing a query for 1 sec but on the java side it takes 5 secs to get
> the data back to my code. Part of this slowdown is due to the logging. Can
> it be optimized, i.e. having a boolean to true/false if logging is enabled
> and then do the calls?
>
> Thanks
>
>
>
>
> --
> Sent via pgsql-jdbc mailing list (pgsql-jdbc(at)postgresql(dot)org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-jdbc
>
>
>
>
>
>
>
>
>
>
>

In response to

Responses

Browse pgsql-jdbc by date

  From Date Subject
Next Message Daniel Migowski 2017-03-23 13:41:37 Re: 42.0.0.jre7, driver logging takes 30-40% of my server's time
Previous Message Konstantinos Kougios 2017-03-23 13:21:25 Re: 42.0.0.jre7, driver logging takes 30-40% of my server's time