Re: too much pgbench init output

Hi,

attached is a new version of the patch that

(a) converts the 'log_step_seconds' variable to a constant (and does
not allow changing it using a command-line option etc.)

(b) keeps the current logging as a default

(c) adds a "-q" switch that enables the new logging with a 5-second
interval

I'm still not convinced there should be yet another know for tuning the
log interval - opinions?

Tomas

On 11.12.2012 10:23, Jeevan Chalke wrote:
>
>
>
> On Sun, Dec 9, 2012 at 8:11 AM, Tomas Vondra <tv(at)fuzzy(dot)cz
> <mailto:tv(at)fuzzy(dot)cz>> wrote:
>
> On 20.11.2012 08:22, Jeevan Chalke wrote:
> > Hi,
> >
> >
> > On Tue, Nov 20, 2012 at 12:08 AM, Tomas Vondra <tv(at)fuzzy(dot)cz
> <mailto:tv(at)fuzzy(dot)cz>
> > <mailto:tv(at)fuzzy(dot)cz <mailto:tv(at)fuzzy(dot)cz>>> wrote:
> >
> > On 19.11.2012 11:59, Jeevan Chalke wrote:
> > > Hi,
> > >
> > > I gone through the discussion for this patch and here is my
> review:
> > >
> > > The main aim of this patch is to reduce the number of log
> lines. It is
> > > also suggested to use an options to provide the interval but
> few of us
> > > are not much agree on it. So final discussion ended at
> keeping 5 sec
> > > interval between each log line.
> > >
> > > However, I see, there are two types of users here:
> > > 1. Who likes these log lines, so that they can troubleshoot some
> > > slowness and all
> > > 2. Who do not like these log lines.
> >
> > Who likes these lines / needs them for something useful?
> >
> >
> > No idea. I fall in second category.
> >
> > But from the discussion, I believe some people may need detailed
> (or lot
> > more) output.
>
> I've read the thread again and my impression is that no one really needs
> or likes those lines, but
>
> (1) it's rather pointless to print a message every 100k rows, as it
> usually fills the console with garbabe
>
> (2) it's handy to have regular updates of the progress
>
> I don't think there're people (in the thread) that require to keep the
> current amount of log messages.
>
> But there might be users who actually use the current logs to do
> something (although I can't imagine what). If we want to do this in a
> backwards compatible way, we should probably use a new option (e.g.
> "-q") to enable the new (less verbose) logging.
>
> Do we want to allow both types of logging, or shall we keep only the new
> one? If both, which one should be the default one?
>
>
> Both the options are fine with me, but the default should be the current
> behaviour.
>
>
> > > So keeping these in mind, I rather go for an option which
> will control
> > > this. People falling in category one can set this option to
> very low
> > > where as users falling under second category can keep it high.
> >
> > So what option(s) would you expect? Something that tunes the
> interval
> > length or something else?
> >
> >
> > Interval length.
>
> Well, I can surely imagine something like "--interval N".
>
>
> +1
>
>
>
> > A switch that'd choose between the old and new behavior might
> be a good
> > idea, but I'd strongly vote against "automagic" heuristics. It
> makes the
> > behavior very difficult to predict and I really don't want to
> force the
> > users to wonder whether the long delay is due to general
> slowness of the
> > machine or some "clever" logic that causes long delays between log
> > messages.
> >
> > That's why I choose a very simple approach with constant time
> interval.
> > It does what I was aiming for (less logs) and it's easy to
> predict.
> > Sure, we could choose different interval (or make it an option).
> >
> >
> > I am preferring an option for choosing an interval, say from 1
> second to
> > 10 seconds.
>
> Ummmm, why not to allow arbitrary integer? Why saying 1 to 10 seconds?
>
>
> Hmm.. actually, I have no issues with any number there. Just put 1..10
> as we hard-coded it 5. No particular reason as such.
>
>
>
> > BTW, what if, we put one log message every 10% (or 5%) with time taken
> > (time taken for last 10% (or 5%) and cumulative) over 5 seconds ?
> > This will have only 10 (or 20) lines per pgbench initialisation.
> > And since we are showing time taken for each block, if any slowness
> > happens, one can easily find a block by looking at the timings and
> > troubleshoot it.
> > Though 10% or 5% is again a debatable number, but keeping it constant
> > will eliminate the requirement of an option.
>
> That's what I originally proposed in September (see the messages from
> 17/9), and Alvaro was not relly excited about this.
>
> Attached is a patch with fixed whitespace / indentation errors etc.
> Otherwise it's the same as the previous version.
>
>
> OK. Looks good now.
>
> Any other views / suggestions are welcome.
>
> Thanks
>
>
> Tomas
>
>
> --
> Sent via pgsql-hackers mailing list (pgsql-hackers(at)postgresql(dot)org
> <mailto:pgsql-hackers(at)postgresql(dot)org>)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-hackers
>
>
>
>
> --
> Jeevan B Chalke
> Senior Software Engineer, R&D
> EnterpriseDB Corporation
> The Enterprise PostgreSQL Company
>
> Phone: +91 20 30589500
>
> Website: www.enterprisedb.com <http://www.enterprisedb.com>
> EnterpriseDB Blog: http://blogs.enterprisedb.com/
> Follow us on Twitter: http://www.twitter.com/enterprisedb
>
> This e-mail message (and any attachment) is intended for the use of the
> individual or entity to whom it is addressed. This message contains
> information from EnterpriseDB Corporation that may be privileged,
> confidential, or exempt from disclosure under applicable law. If you are
> not the intended recipient or authorized to receive this for the
> intended recipient, any use, dissemination, distribution, retention,
> archiving, or copying of this communication is strictly prohibited. If
> you have received this e-mail in error, please notify the sender
> immediately by reply e-mail and delete this message.