Re: Raw device on PostgreSQL

On Tue, Apr 28, 2020 at 8:10 AM Andreas Karlsson <andreas(at)proxel(dot)se> wrote:

> It would require quite a bit of work since 1) PostgreSQL stores its data
> in multiple files and 2) PostgreSQL currently supports only synchronous
> buffered IO.
>
> To get the performance benefits from using raw devices I think you would
> want to add support for asynchronous IO to PostgreSQL rather than
> implementing your own layer to emulate the kernel's buffered IO.
>
> Andres Freund did a talk on aync IO in PostgreSQL earlier this year. It
> was not recorded but the slides are available.
>
>
> https://www.postgresql.eu/events/fosdem2020/schedule/session/2959-asynchronous-io-for-postgresql/

FWIW, in 2007/2008, when I was at EnterpriseDB, Inaam Rana and I
implemented a benchmarkable proof-of-concept patch for direct I/O and
asynchronous I/O (for libaio and POSIX). We made that patch public, so it
should be on the list somewhere. But, we began to run into performance
issues related to buffer manager scaling in terms of locking and,
specifically, replacement. We began prototyping alternate buffer managers
(going back to the old MRU/LRU model with midpoint insertion and testing a
2Q variant) but that wasn't public. I had also prototyped raw device
support, which is a good amount of work and required implementing a custom
filesystem (similar to Oracle's ASM) within the storage manager. It's
probably a bit harder now than it was then, given the number of different
types of file access.

--
Jonah H. Harris