Re: Sync Rep v17

On Wed, Feb 23, 2011 at 10:39 PM, Daniel Farina <daniel(at)heroku(dot)com> wrote:
> On Fri, Feb 18, 2011 at 4:06 PM, Simon Riggs <simon(at)2ndquadrant(dot)com> wrote:
>>
>> Well, good news all round.
>>
>> v17 implements what I believe to be the final set of features for sync
>> rep. This one I'm actually fairly happy with. It can be enjoyed best at
>> DEBUG3.
>
> I've been messing with this patch and am wondering if this behavior is expected:
>
> I've been frobbing the server around (I was messing around with the
> syncrep feature, but do not know if this is related just yet), and
> came upon a case I do not expect: it would appear that prior to
> establishing a connection to do streaming replication, the "startup
> process" (which is recovering) is very slowly catching up (or so it
> would be indicated by txid_current_snapshot()) and eating up enormous
> amounts of memory, such as 6GB at a time in RES,  monotonically
> increasing. Furthermore, the incrementation of the txid_snapshot is
> very slow, and it doesn't seem like I'm coming close to making full
> use of my resources: cpu and block devices are not very busy.  There
> may have been a brief spurt of pgbench activity that would generate
> such WAL traffic to replay.
>
> I have not done a hard shutdown to my knowledge, and the server does
> allow me to query relatively quickly as a standby.

Oh, yes, this reproduces past shutdowns/startups, and there's quite a
few txids before I catch up. I'm also comfortable poking around with
gdb (I have already recompiled with debugging symbols and
optimizations off and was poking around, especially at
MemoryContextStats(TopMemoryContext), but was not rewarded.

--
fdr