pgsql: bufmgr: Implement AIO read support

bufmgr: Implement AIO read support

This commit implements the infrastructure to perform asynchronous reads into
the buffer pool.

To do so, it:

- Adds readv AIO callbacks for shared and local buffers

It may be worth calling out that shared buffer completions may be run in a
different backend than where the IO started.

- Adds an AIO wait reference to BufferDesc, to allow backends to wait for
in-progress asynchronous IOs

- Adapts StartBufferIO(), WaitIO(), TerminateBufferIO(), and their localbuf.c
equivalents, to be able to deal with AIO

- Moves the code to handle BM_PIN_COUNT_WAITER into a helper function, as it
now also needs to be called on IO completion

As of this commit, nothing issues AIO on shared/local buffers. A future commit
will update StartReadBuffers() to do so.

Buffer reads executed through this infrastructure will report invalid page /
checksum errors / warnings differently than before:

In the error case the error message will cover all the blocks that were
included in the read, rather than just the reporting the first invalid
block. If more than one block is invalid, the error will include information
about the range of the read, the first invalid block and the number of invalid
pages, with a HINT towards the server log for per-block details.

For the warning case (i.e. zero_damaged_buffers) we would previously emit one
warning message for each buffer in a multi-block read. Now there is only a
single warning message for the entire read, again referring to the server log
for more details in case of multiple checksum failures within a single larger
read.

Reviewed-by: Noah Misch <noah(at)leadboat(dot)com>
Reviewed-by: Melanie Plageman <melanieplageman(at)gmail(dot)com>
Reviewed-by: Nazir Bilal Yavuz <byavuz81(at)gmail(dot)com>
Discussion: https://postgr.es/m/uvrtrknj4kdytuboidbhwclo4gxhswwcpgadptsjvjqcluzmah%40brqs62irg4dt
Discussion: https://postgr.es/m/20210223100344.llw5an2aklengrmn@alap3.anarazel.de
Discussion: https://postgr.es/m/stj36ea6yyhoxtqkhpieia2z4krnam7qyetc57rfezgk4zgapf@gcnactj4z56m

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/047cba7fa0f8c6930b0dd1d93d98c736ef1e4a5a

Modified Files
--------------
src/backend/storage/aio/aio_callback.c | 5 +
src/backend/storage/buffer/README | 9 +-
src/backend/storage/buffer/buf_init.c | 3 +
src/backend/storage/buffer/bufmgr.c | 841 +++++++++++++++++++++++++++++++--
src/backend/storage/buffer/localbuf.c | 61 ++-
src/backend/storage/page/bufpage.c | 12 +-
src/include/storage/aio.h | 6 +-
src/include/storage/buf_internals.h | 7 +-
src/include/storage/bufmgr.h | 6 +
src/include/storage/bufpage.h | 1 +
10 files changed, 885 insertions(+), 66 deletions(-)