Re: Show lossy heap block info in EXPLAIN ANALYZE for bitmap heap scan

Robert Haas wrote:
> On Fri, Dec 6, 2013 at 5:02 AM, Etsuro Fujita
<fujita(dot)etsuro(at)lab(dot)ntt(dot)co(dot)jp>
> wrote:
> > Though at first I agreed on this, while working on this I start to
> > think information about (2) is enough for tuning work_mem. Here are
> > examples using a version under development, where "Bitmap Memory
> > Usage" means (peak) memory space used by a TIDBitmap, and "Desired"
> > means the memory required to guarantee non-lossy storage of a TID set,
> > which is shown only when the TIDBitmap has been lossified. (work_mem
> > = 1MB.)

> I'd be wary of showing a desired value unless it's highly likely to be
> accurate.

Thank you for the comments!

The desired value is accurately estimated based on (a) the total number of
exact/lossy pages stored in the TIDBitmap and (b) the following equation in
tbm_create(), except for the GIN case where lossy pages are added to the
TIDBitmap by tbm_add_page().

/*
* Estimate number of hashtable entries we can have within maxbytes. ...
*/
nbuckets = maxbytes /
(MAXALIGN(sizeof(HASHELEMENT)) + MAXALIGN(sizeof(PagetableEntry))
+ sizeof(Pointer) + sizeof(Pointer));

In the GIN case, however, the version under development has a risk of the
overestimation. (And in that case, in my understanding, we can't guarantee
non-lossy storage of the TIDBitmap any more.) So, for that case, I think to
change the message for the desired value a bit. I'll submit the patch
later.

Thanks,

Best regards,
Etsuro Fujita