Unified File API

Background
==========
PostgreSQL has an amazing variety of routines for accessing files. Consider just the “open file” routines.
PathNameOpenFile, OpenTemporaryFile, BasicOpenFile, open, fopen, BufFileCreateFileSet,
BufFileOpenFileSet, AllocateFile, OpenTransientFile, FileSetCreate, FileSetOpen, mdcreate, mdopen,
Smgr_open,

On the downside, “amazing variety” also means somewhat confusing and difficult to add new features.
Someday, we’d like to add encryption or compression to the various PostgreSql files.
To do that, we need to bring all the relevant files into a common file API where we can implement
the new features.

Goals of Patch
=============
1)Unify file access so most of “the other” files can go through a common interface, allowing new features
like checksums, encryption or compression to be added transparently. 2) Do it in a way which doesn’t
change the logic of current code. 3)Convert a reasonable set of callers to use the new interface.

Note the focus is on the “other” files. The buffer cache and the WAL have similar needs,
but they are being done in a separate project. (yes, the two projects are coordinating)

Patch 0001. Create a common file API.
===============================
Currrently, PostgreSQL files feed into three funnels. 1) system file descriptors (read/write/open),
2) C library buffered files (fread/fwri;te/fopn), and 3) virtual file descriptors (FileRead/FileWrite/PathNameOpenFile).
Of these three, virtual file descriptors (VFDs) are the most common. They are also the
only funnel which is implemented by PostgresSql.

Decision: Choose VFDs as the common interface.

Problem: VFDs are random access only.
Solution: Add sequential read/write code on top of VFDs. (FileReadSeq, FileWriteSeq, FileSeek, FileTell, O_APPEND)

Problem: VFDs have minimal error handling (based on errno.)
Solution: Add an “ferror” style interface (FileError, FileEof, FileErrorCode, FileErrorMsg)

Problem: Must maintain compatibility with existing error handling code.
Solution: save and restore errno to minimize changes to existing code.

Patch 0002. Update code to use the common file API
===========================================
The second patch alters callers so they use VFDs rather than system or C library files.
It doesn’t modify all callers, but it does capture many of the files which need
to be encrypted or compressed. This is definitely WIP.

Future (not too far away)
=====================
Looking ahead, there will be another set of patches which inject buffering and encryption into
the VFD interface. The future patches will build on the current work and introduce new “oflags”
to enable encryption and buffering.

Compression is also a possibility, but currently lower priority and a bit tricky for random access files.
Let us know if you have a use case.