Re: AIO v2.5

From: Andres Freund <andres(at)anarazel(dot)de>
To: Antonin Houska <ah(at)cybertec(dot)at>
Cc: Noah Misch <noah(at)leadboat(dot)com>, pgsql-hackers(at)postgresql(dot)org, Thomas Munro <thomas(dot)munro(at)gmail(dot)com>, Heikki Linnakangas <hlinnaka(at)iki(dot)fi>, Robert Haas <robertmhaas(at)gmail(dot)com>, Jakub Wartak <jakub(dot)wartak(at)enterprisedb(dot)com>, Jelte Fennema-Nio <postgres(at)jeltef(dot)nl>
Subject: Re: AIO v2.5
Date: 2025-03-17 20:00:53
Message-ID: oav52ytakcvfnk52w5o5wvxdjavxth2tgptxt25gnfmh2pdcpk@omp3xukdu5if
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

Attached is v2.9 with the following, fairly small, changes:
- Rebased ontop of the additional committed read stream patches

- Committed the localbuf: refactorings (but not the change to expand
refcounting of local buffers, that seems a bit more dependent on the rest)

- Committed the temp table test after some annoying debugging
https://postgr.es/m/w5wr26ijzp7xz2qrxkt6dzvmmn2tn6tn5fp64y6gq5iuvg43hw%40v4guo6x776dq

- Some rephrasing and moving of comments in the first two commits

- There was a small bug in the smgr reopen call I found when reviewing, the
PgAioOpData->read.fd field was referenced for both reads and writes, which
failed to fail because both read/write use a compatible struct layout.

Unless I hear otherwise, I plan to commit the first two patches fairly soon,
followed by the worker support patches a few buildfarm cycles later.

I'm sure there's a bunch of stuff worth improving in the AIO infrastructure
and I can't imagine a project of this size not having bugs. But I think it's
in a state where that's better worked out in tree, with broader exposure and
testing.

Greetings,

Andres Freund

Attachment Content-Type Size
v2.9-0001-aio-Basic-subsystem-initialization.patch text/x-diff 25.1 KB
v2.9-0002-aio-Add-core-asynchronous-I-O-infrastructure.patch text/x-diff 90.3 KB
v2.9-0003-aio-Infrastructure-for-io_method-worker.patch text/x-diff 24.9 KB
v2.9-0004-aio-Add-io_method-worker.patch text/x-diff 20.2 KB
v2.9-0005-aio-Add-liburing-dependency.patch text/x-diff 13.4 KB
v2.9-0006-aio-Add-io_method-io_uring.patch text/x-diff 20.2 KB
v2.9-0007-smgr-Hold-interrupts-in-most-smgr-functions.patch text/x-diff 9.4 KB
v2.9-0008-aio-Implement-support-for-reads-in-smgr-md-fd.patch text/x-diff 22.0 KB
v2.9-0009-aio-Add-README.md-explaining-higher-level-desig.patch text/x-diff 19.0 KB
v2.9-0010-aio-Add-pg_aios-view.patch text/x-diff 9.7 KB
v2.9-0011-Support-buffer-forwarding-in-read_stream.c.patch text/x-diff 8.7 KB
v2.9-0012-Support-buffer-forwarding-in-StartReadBuffers.patch text/x-diff 10.5 KB
v2.9-0013-localbuf-Track-pincount-in-BufferDesc-as-well.patch text/x-diff 4.5 KB
v2.9-0014-bufmgr-Implement-AIO-read-support.patch text/x-diff 25.4 KB
v2.9-0015-bufmgr-Improve-stats-when-buffer-was-read-in-co.patch text/x-diff 3.0 KB
v2.9-0016-bufmgr-Use-AIO-in-StartReadBuffers.patch text/x-diff 26.9 KB
v2.9-0017-docs-Reframe-track_io_timing-related-docs-as-wa.patch text/x-diff 5.3 KB
v2.9-0018-aio-Basic-read_stream-adjustments-for-real-AIO.patch text/x-diff 3.8 KB
v2.9-0019-aio-Experimental-heuristics-to-increase-batchin.patch text/x-diff 1.6 KB
v2.9-0020-aio-Add-test_aio-module.patch text/x-diff 42.4 KB
v2.9-0021-aio-Implement-smgr-md-fd-write-support.patch text/x-diff 12.7 KB
v2.9-0022-aio-Add-bounce-buffers.patch text/x-diff 23.1 KB
v2.9-0023-bufmgr-Implement-AIO-write-support.patch text/x-diff 9.9 KB
v2.9-0024-aio-Add-IO-queue-helper.patch text/x-diff 7.4 KB
v2.9-0025-bufmgr-use-AIO-in-checkpointer-bgwriter.patch text/x-diff 31.1 KB
v2.9-0026-Ensure-a-resowner-exists-for-all-paths-that-may.patch text/x-diff 2.6 KB
v2.9-0027-Temporary-Increase-BAS_BULKREAD-size.patch text/x-diff 1.3 KB
v2.9-0028-WIP-Use-MAP_POPULATE.patch text/x-diff 1.1 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Robert Haas 2025-03-17 20:02:54 Re: ZStandard (with dictionaries) compression support for TOAST compression
Previous Message David G. Johnston 2025-03-17 19:36:04 Re: Forbid to DROP temp tables of other sessions