pgCluu 3.0 has been published

Ubud, Bali - September 18th, 2019

**pgCluu 3.0 released**

PostgreSQL Cluster Utilization
----------------------------------------

pgCluu is a Perl program used to perform a full audit of a PostgreSQL
Cluster performances. It is divided in two parts, a collector used to
grab statistics on the PostgreSQL server using psql and sar, a reports
builder that will generate all HTML and charts output.

This release is a major release that adds lot of improvement and
new reports. It also improve performances and fully support a CGI
mode with dynamic content for realtime reports.

New or enhanced reports/features:

* The CGI mode to provide dynamic reports on time selection is now
production ready and has left the beta stage. See documentation
on how to enable it. Debian packaging enable this mode by default.
* Installation and configuration is fully managed from Makefile.PL
script. This will help distribution packaging with additional
configuration directives to control the behavior of pgcluu scripts
with generic systemd service and timer files.
* Add incremental mode to pgcluu. That mean that you don't have to
remove the content of the report directory anymore before running
pgcluu again and when pgcluu detect that you are running in this
mode. The incremental mode is detected from statistic directory
when you have used the --rotate-daily or --rotate-hourly option
to pgcluu_collectd.
* Rewrite Stats I/O reports to display read/hit per table or indexes
during the audit period reported.
* Move Xlog (Wal bytes written) report under replication menu as
data comes from pg_stat_replication.
* Explode System and Cluster reports into several reports dedicated
files to limit the size of the HTML files in incremental mode.
* Add collect and report of vm.nr_overcommit_hugepages.
* Add report on pages scanned to report number of pages scanned by
the kswapd daemon and number of pages scanned directly per second
(pgscank/s and pgscand/s). It also reports the number of pages
the system has reclaimed from cache (pagecache and swapcache) per
second to satisfy its memory demands (pgsteal/s). On a second
axis %vmeffto reports the the efficiency of page reclaim.
* All pages reports have been moved as sub menu of the Page menu and
pgfree has been removed from the page fault report.
* Limit system cache statistics report to pgpgin/s and pgpgout/s and
a new report is dedicated to Page fault statistics with majflt/s,
minflt/s and pgfree/s dataset. Dataset minflt/s is calculated from
sar output as result of "fault/s - majflt/s".
* All statistics about pg_stat_user_* and pg_stat_xact_user_* views
are now stored as a snapshot at start and at end of the audit
period and each hour if you are running in incremental mode. The
old behavior was to append the snapshot at each running interval
to the data file but this is not necessary and this can use huge
disk space if you have lot of table and indexes in your database.
Backward compatibility is preserved.
* New report of json versus jsonb columns in each database.
* Add keepalive kernel configuration in SysInfo report.
* Separate bgwriter "allocated" buffers in a dedicated report for
better reading.
* Add new report about estimated memory workload based on the value
of Committed_AS from /proc/meminfo. Committed_AS is the total
amount of memory estimated to complete the workload. This value
represents the worst case scenario value, and also includes swap
memory. The report show the memory workload aver the time.

New command line options:

* Add option -x, --external-menu to pgcluu to save the common menu
in menu.html and load it into each report using w3-include-html
attribute from w3.js. This will only work if access to HTML report
is through a Web server, not using the file:// protocol.
* Add new option --retention to pgcluu to set number of rolling days
to keep in report directory. Default is to store indefinitely.
* Add option --retention NDAYS to pgcluu_collectd to set the number
of rolling days to keep in data directory in incremental mode.
Default is to store indefinitely.

New configuration directives :

* Add configuration directive STATS_REPORT_CACHING to pgcluu.conf
file. This directive must be enabled in CGI mode. In this case the
pgcluu script must be run periodically at least each 5 minutes to
build and update cached statistics. In cache mode pgcluu generate
binary file in the statistics directory and nothing in the report
directory. The pgcluu.service and pgcluu.timer can be use, pgcluu
will read the configuration file and switch automatically to cache
mode if STATS_REPORT_CACHING is enabled.
* Add MAX_RENDERED_DAYS configuration directive to set the maximum
number of days in a graph. Using default pgcluu_collectd 1 minute
interval each, daily graph will have 1440 points. Having too much
data to render is not a good point for performances with the
current code. The user can set is own graph period, default is to
limit to the first seven days of the selected period. This mean a
maximum of 10080 points for one week, this seems a safe limit.

It also fixes some issues reported by users since last ten months.

--
Gilles Darold
http://www.darold.net/