From: | serge(at)rielau(dot)com |
---|---|
To: | "Tomas Vondra" <tomas(dot)vondra(at)2ndquadrant(dot)com>, "David Rowley" <david(dot)rowley(at)2ndquadrant(dot)com>, "Andres Freund" <andres(at)anarazel(dot)de> |
Cc: | "Heikki Linnakangas" <hlinnaka(at)iki(dot)fi>, "pgsql-hackers" <pgsql-hackers(at)postgresql(dot)org>, "Jeff Davis" <pgsql(at)j-davis(dot)com> |
Subject: | RE: Re: Spilling hashed SetOps and aggregates to disk |
Date: | 2018-06-05 15:39:02 |
Message-ID: | 20180605083902.9cf3801d03770ada01bb39dc8f52321d.8e74dafc00.mailapi@email18.godaddy.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
Strange. We don't see this overeahd and measure a lot more than just a single metric:
Of the grab below only the context->stats += size is need.
Could easily be a macro.
We also track overall backend size to cap it, and track memory contexts in shared memory (plan cache, function cache, message and transaction contexts).
Each with high watermarks.
/*
* NOTE: Shared contexts stats are global and shared by
* all engines, so, never add the size directly, instead
* use stats API which uses Atomic.*() calls to ensure
* mutual exclusion.
*/
static void MaintainAllocStats(MemoryContext context, Size size)
{
if (context->stats)
{
if (!context->isShared)
{
sMonAdd(memory_context_stats, TotalBackendAllocated_Size, size);
sMonHWM(memory_context_stats, TotalBackendAllocated_HWM,
sMonGet(memory_context_stats, TotalBackendAllocated_Size));
context->allocatedSize += size;
context->stats[0] += size;
context->stats[1] = (context->stats[1] >= context->stats[0] ?
context->stats[1] : context->stats[0]);
}
else
{
if (!context->isNested)
size += SHM_BLOCK_OVERHEAD;
sAtomicAdd(&context->allocatedSize, size);
if (!context->isNested ||
!context->clusterrep ||
context->clusterrep->stats != context->stats)
sAtomicAdd(context->stats, size);
}
}
}
I'll try myself on peeling out a patch for community for the stats and ask he developer responsible for hash spilling to engage.
Cheers
Serge
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
From | Date | Subject | |
---|---|---|---|
Next Message | Lutz Horn | 2018-06-05 15:39:25 | Re: Code of Conduct plan |
Previous Message | Peter Geoghegan | 2018-06-05 15:37:11 | Re: Code of Conduct plan |