Re: checkpointer continuous flushing

From: Fabien COELHO <coelho(at)cri(dot)ensmp(dot)fr>
To: Andres Freund <andres(at)anarazel(dot)de>
Cc: PostgreSQL Developers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: checkpointer continuous flushing
Date: 2015-07-26 15:01:50
Message-ID: alpine.DEB.2.10.1507261656380.25082@sto
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers


Hello,

Attached is very minor v5 update which does a rebase & completes the
cleanup of doing a full sort instead of a chuncked sort.

>> Attached is an updated version of the patch which turns the sort option
>> into a boolean, and also include the sort time in the checkpoint log.
>>
>> There is still an open question about whether the sorting buffer allocation
>> is lost on some signals and should be reallocated in such event.
>
> In such case, probably the allocation should be managed from
> CheckpointerMain, and the lazy allocation could remain for other callers (I
> guess just "initdb").
>
>
> More open questions:
>
> - best name for the flush option (checkpoint_flush_to_disk,
> checkpoint_flush_on_write, checkpoint_flush, ...)
>
> - best name for the sort option (checkpoint_sort,
> checkpoint_sort_buffers, checkpoint_sort_ios, ...)
>
>
> Other nice-to-have inputs:
>
> - tests on a non-linux system with posix_fadvise
> (FreeBSD? others?)
>
> - tests on a large dedicated box
>
>
> Attached are some scripts to help with testing, if someone's feels like that:
>
> - cp_test.sh: run some tests, to adapt to one's setup...
>
> - cp_test_count.pl: show percent of late transactions
>
> - avg.py: show stats about stuff
>
> sh> grep 'progress: ' OUTPUT_FILE | cut -d' ' -f4 | avg.py
>
> *BEWARE* that if pgbench got stuck some "0" data are missing,
> look for the actual tps in the output file and for the line
> count to check whether it is the case... some currently submitted
> patch on pgbench helps, see https://commitfest.postgresql.org/5/199/

As this pgbench patch is now in master, pgbench is less likely to get
stuck, but check nevertheless that the number of progress line matches the
expected number.

--
Fabien.

Attachment Content-Type Size
checkpoint-continuous-flush-5.patch text/x-diff 42.5 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Andrew Gierth 2015-07-26 15:06:30 Re: [HACKERS] Gsets: ROW expression semantic broken between 9.4 and 9.5
Previous Message Andres Freund 2015-07-26 15:00:11 Re: Buildfarm failure from overly noisy warning message