Re: checkpointer continuous flushing

On Wed, Aug 19, 2015 at 12:13 PM, Fabien COELHO <coelho(at)cri(dot)ensmp(dot)fr> wrote:

>
> Sure, I think what can help here is a testcase/'s (in form of script file
>> or some other form, to test this behaviour of patch) which you can write
>> and post here, so that others can use that to get the data and share it.
>>
>
> Sure... note that I already did that on this thread, without any echo...
> but I can do it again...
>
>
Thanks.

I have tried your scripts and found some problem while using avg.py
script.
grep 'progress:' test_medium4_FW_off.out | cut -d' ' -f4 | ./avg.py
--limit=10 --length=300
: No such file or directory

I didn't get chance to poke into avg.py script (the command without
avg.py works fine). Python version on the m/c, I planned to test is
Python 2.7.5.

Today while reading the first patch (checkpoint-continuous-flush-10-a),
I have given some thought to below part of patch which I would like
to share with you.

+static int

+NextBufferToWrite(

+ TableSpaceCheckpointStatus *spcStatus, int nb_spaces,

+ int *pspace, int num_to_write, int num_written)

+{

+ int space = *pspace, buf_id = -1, index;

+

+ /*

+ * Select a tablespace depending on the current overall progress.

+ *

+ * The progress ratio of each unfinished tablespace is compared to

+ * the overall progress ratio to find one with is not in advance

+ * (i.e. overall ratio > tablespace ratio,

+ * i.e. tablespace written/to_write > overall written/to_write

Here, I think above calculation can go for toss if backend or bgwriter
starts writing buffers when checkpoint is in progress. The tablespace
written parameter won't be able to consider the one's written by backends
or bgwriter. Now it may not big thing to worry but I find Heikki's version
worth considering, he has not changed the overall idea of this patch, but
the calculations are somewhat simpler and hence less chance of going
wrong.

With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com