RE: Moving from pgFouine to pgBadger Issue with Total query duration metric

Hi,

It could be a case of pgFouine not picking up/reporting all the queries when it calculates those stats. I see all the metrics matching closely in either reports except for the total query duration and query peak times. pgFouine thinks that there has been a peak of 6 queries/second at 2018-01-11 14:26:18 while pgBadger thinks there has been a peak of 6 queries/second at 2018-01-11 17:41:30, which could be right but it may be a different set of queries.

Best Regards,

Nawaz Ahmed
Software Development Engineer

Fujitsu Australia Software Technology Pty Ltd
14 Rodborough Road, Frenchs Forest NSW 2086, Australia
T +61 2 9452 9027
Nawaz(at)fast(dot)au(dot)fujitsu(dot)com<mailto:Nawaz(at)fast(dot)au(dot)fujitsu(dot)com>
fastware.com.au<http://fastware.com.au/>

[cid:image001(dot)jpg(at)01D38EF9(dot)5CCDFA00]
[cid:image002(dot)jpg(at)01D38EF9(dot)5CCDFA00]

From: Sreekanth Palluru [mailto:sree4pg(at)gmail(dot)com]
Sent: Friday, 12 January 2018 4:02 PM
To: PostgreSQL mailing lists <pgsql-general(at)postgresql(dot)org>
Subject: Moving from pgFouine to pgBadger Issue with Total query duration metric

Hi ,
I am moving from pgFouine to pgBadger. When I compare the reports I see that there is major difference in time reported by metric Total query duration for given same postgres server log file.( excuse me for not sharing the postgres server logfile )
pgfouine reports around 2hrs where as pgBadger reports around 4hrs.

I am using below command options to execute the report. I am mainly comparing pgBadger report with queries.html report

Appreciate your help on this ? wanted to know why there so much difference in time ?

pgFouine

/home/postgres/fouine/pgfouine-1.2/pgfouine.php -logtype stderr -file $filename -top 30 \
-report /tmp/queries.html=overall,bytype,slowest,n-mosttime,n-mostfrequent,n-slowestaverage \
-report /tmp/hourly.html=overall,hourly \
-report /tmp/errors.html=overall,n-mostfrequenterrors \
-database $1

pgBadger

perl pgbadger -t 30 --disable-session --disable-connection --disable-lock --disable-temporary --disable-checkpoint --disable-autovacuum -o pgbadger_out_3.html -d <DBNAME> /home/postgres/postgresql-2018-01-11_000000.log

pgFouine => Overall statistics

• Overall statistics ^

* Number of unique normalized queries: 208
* Number of queries: 2,111
* Total query duration: 2h36m45s
* First query: 2018-01-11 00:01:06
* Last query: 2018-01-11 23:20:07
* Query peak: 6 queries/s at 2018-01-11 14:26:18
* Number of errors: 70,043
* Number of unique normalized errors: 19

pgBadger => Global Stats

Global Stats

* Queries
* Events
* Vacuums
* Temporary files
* Sessions
* Connections

* 207 Number of unique normalized queries
* 2,113 Number of queries
* 4h41m48s Total query duration
* 2018-01-11 00:01:06 First query
* 2018-01-11 23:20:07 Last query
* 6 queries/s at 2018-01-11 17:41:30 Query peak

--
Regards
Sreekanth
Disclaimer

The information in this e-mail is confidential and may contain content that is subject to copyright and/or is commercial-in-confidence and is intended only for the use of the above named addressee. If you are not the intended recipient, you are hereby notified that dissemination, copying or use of the information is strictly prohibited. If you have received this e-mail in error, please telephone Fujitsu Australia Software Technology Pty Ltd on + 61 2 9452 9000 or by reply e-mail to the sender and delete the document and all copies thereof.

Whereas Fujitsu Australia Software Technology Pty Ltd would not knowingly transmit a virus within an email communication, it is the receiver’s responsibility to scan all communication and any files attached for computer viruses and other defects. Fujitsu Australia Software Technology Pty Ltd does not accept liability for any loss or damage (whether direct, indirect, consequential or economic) however caused, and whether by negligence or otherwise, which may result directly or indirectly from this communication or any files attached.

If you do not wish to receive commercial and/or marketing email messages from Fujitsu Australia Software Technology Pty Ltd, please email unsubscribe(at)fast(dot)au(dot)fujitsu(dot)com