Re: Extended Prefetching using Asynchronous IO - proposal and patch

Thanks for looking at it!

> Date: Thu, 29 May 2014 00:19:33 +0300
> From: hlinnakangas(at)vmware(dot)com
> To: johnlumby(at)hotmail(dot)com; pgsql-hackers(at)postgresql(dot)org
> CC: klaussfreire(at)gmail(dot)com
> Subject: Re: [HACKERS] Extended Prefetching using Asynchronous IO - proposal and patch
>
> On 05/28/2014 11:52 PM, John Lumby wrote:
> >
>
> The patch seems to assume that you can put the aiocb struct in shared
> memory, initiate an asynchronous I/O request from one process, and wait
> for its completion from another process. I'm pretty surprised if that
> works on any platform.

It works on linux. Actually this ability allows the asyncio implementation to
reduce complexity in one respect (yes I know it looks complex enough) :
it makes waiting for completion of an in-progress IO simpler than for
the existing synchronous IO case,. since librt takes care of the waiting.
specifically, no need for extra wait-for-io control blocks
such as in bufmgr's WaitIO()

This also plays very nicely with the syncscan where the cohorts run close together.

If anyone would like to advise whether this also works on MacOS/BSD , windows,
that would be good, as I can't verify it myself.

>
> How portable is POSIX aio nowadays? Googling around, it still seems that
> on Linux, it's implemented using threads. Does the thread-emulation
> implementation cause problems with the rest of the backend, which
> assumes that there is only a single thread? In any case, I think we'll

Good question, I am not aware of any dependency on a backend having only
a single thread. Can you please point me to such dependencies?

> want to encapsulate the AIO implementation behind some kind of an API,
> to allow other implementations to co-exist.

It is already - it follows the smgr(stg mgr) -> md (mag disk) -> fd (filesystem)
layering used for the existing filesystem ops including posix-fadvise.

>
> Benchmarks?

I will see if I can package mine up somehow.
Would be great if anyone else would like to benchmark it on theirs ...

> - Heikki