From: | wenhui qiu <qiuwenhuifx(at)gmail(dot)com> |
---|---|
To: | PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org> |
Subject: | bgwrite process is too lazy |
Date: | 2024-10-02 08:48:01 |
Message-ID: | CAGjGUALHnmQFXmBYaFCupXQu7nx7HZ79xN29+HoE5s-USqprUg@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
Hi hackers
Whenever I check the checkpoint information in a log, most dirty pages
are written by the checkpoint process,I tried to read the source code to
find out why, I think reducing this value will make bgwrite process write
more dirty pages (min_scan_buffers = (int) (NBuffers /
(scan_whole_pool_milliseconds / BgWriterDelay));) ,Here's how I tested
it, Originally, I wanted to write a patch, and I listened to the expert
community leaders first
src/backend/postmaster/bgwriter.c
[image: image.png]
I see a line of comments in (src/backend/storage/buffer/bufmgr.c )
[image: image.png]
This is a simple test I did myself
```
test methods
pgbench -i -s 10000 dbname -U postgres
pgbench -U postgres dbname -c 96 -j 96 -P 1 -T 900
test env
Intel(R) Xeon(R) Platinum 8336C CPU @ 2.30GHz 128 threads 64 cores
512G
ssd:1.8 T
postgresql version 17 GA
postgresql parameters
cat postgresql.auto.conf
listen_addresses = '*'
port = '5430'
max_connections = '1000'
unix_socket_directories = '/tmp,.'
unix_socket_permissions = '0700'
password_encryption = 'md5'
shared_buffers = '32GB'
maintenance_work_mem = '2GB'
autovacuum_work_mem = '2GB'
vacuum_buffer_usage_limit = '16GB'
max_files_per_process = '60000'
vacuum_cost_limit = '10000'
bgwriter_delay = '10ms'
bgwriter_lru_maxpages = '8192'
bgwriter_lru_multiplier = '10.0'
max_worker_processes = '32'
max_parallel_workers_per_gather = '8'
max_parallel_maintenance_workers = '8'
max_parallel_workers = '32'
wal_buffers = '64MB'
checkpoint_completion_target = '0.999'
max_wal_size = '32GB'
min_wal_size = '16GB'
archive_mode = 'on'
archive_command = '/bin/date'
wal_keep_size = '32GB'
effective_cache_size = '512GB'
log_destination = 'csvlog'
logging_collector = 'on'
log_filename = 'postgresql-%a_%H.log'
log_truncate_on_rotation = 'on'
log_lock_waits = 'on'
autovacuum_max_workers = '10'
autovacuum_naptime = '1min'
autovacuum_vacuum_scale_factor = '0.02'
autovacuum_vacuum_insert_scale_factor = '0.02'
autovacuum_analyze_scale_factor = '0.01'
checkpoint_timeout = '600'
test report
original value
scan_whole_pool_milliseconds = 60000.0
scan_whole_pool_milliseconds = 30000.0
scan_whole_pool_milliseconds = 10000.0
buffers_clean
2731311
7247872
13271584
17406338
num_requested
35
27
28
24
write_time
1110084
855959
1176642
870992
sync_time
1285
1110
746
419
buffers_written
49208974
42617458
35153293
31586820
tps
33153.438460
41242.271814
34969.554408
37177.879583
```
Thanks
From | Date | Subject | |
---|---|---|---|
Next Message | Peter Smith | 2024-10-02 08:54:32 | Re: Conflict Detection and Resolution |
Previous Message | jian he | 2024-10-02 08:41:35 | Re: not null constraints, again |