Re: pgbench--new transaction type

On 06/11/2011 03:21 PM, Jeff Janes wrote:
> I wouldn't expect IPC chatter to show up in profiling, because it
> costs wall time, but not CPU time. The time spent might be attributed
> to the kernel, or to pgbench, or to nothing at all.
>

Profilers aren't necessarily just accumulating raw CPU time though. If
the approach includes sampling "what code is active right now?"
periodically, you might be able to separate this out even though it's
not using CPU time in the normal fashion. I think you might just need
to use a better profiler.

Anyway, the sort of breakdown this helps produce is valuable
regardless. I highlighted the statement you made because I reflexively
challenge theorizing about code hotspots on general principle. The
measurement-based breakdown you provided was more what I look for.

> But there is no
> repository of such "useful for developer testing" patches, other than
> this mailing list. That being the case, would it even be worthwhile
> to fix it up more and submit it to commitfest?
>

The activity around profiling pgbench and trying to crack one of the
bottlenecks has been picking up a lot of momentum lately, and if we're
lucky that will continue throughout 9.2 development. As such, now seems
a good time as any to consider adding something like this. We may end
up reskinng lots of pgbench before this is over. I added your patch to
the CommitFest.

> So at a loop of 512, you would have overhead of 59.0/512=0.115 out of
> total time of 17.4, or 0.7% overhead. So that should be large enough.
>

That I think is workable. If the split was a compile time constant
fixed at 512 unless you specifically changed it, even the worst typical
cases shouldn't suffer much from batch overhang. If you create a
database so large that you only get 50 TPS, which is unusual but not
that rare, having a 512 execution batch size would mean you might get
your "-T" set end time lagging 10 seconds behind its original plan.
Unlike the 10K you started with, that is reasonable; that does sound
like the sweet spot where overhead is low but time overrun isn't too
terrible.

--
Greg Smith 2ndQuadrant US greg(at)2ndQuadrant(dot)com Baltimore, MD
PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us