Re: Spilling hashed SetOps and aggregates to disk

On 06/05/2018 03:41 PM, serge(at)rielau(dot)com wrote:
> In our code base we have added a WithStats-Flavor for creating memory
> contexts.
> This api accepts a pointer to metric for accounting and it is inherited
> by all subcontexts unless overridden.
> So we only needed to change context creation API where we wanted (such
> as TopTansactionContext, Message Context, ..)
> That's quite trivial, actually.

I think that's pretty much what we tried when this patch was first
discussed in 2015, but it added measurable overhead (1-3%) even when the
accounting was disabled. Which is not quite desirable, of course.

> Also we have fixed all those missing hash spills - albeit based on the
> 9.6 hash table design I think.

I don't think this part of the code changed all that much.

> If there is interest by the community we are very willing to share.

Absolutely! I think it'd be great to share both the code and the
reasoning behind the design.

> Cheers
> Serge
> Salesforce

cheers

--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services