Re: Modest proposal to extend TableAM API for controlling cluster commands

> On Jun 16, 2022, at 12:28 AM, David G. Johnston <david(dot)g(dot)johnston(at)gmail(dot)com> wrote:
>
> But you are basically proposing a reworking of the existing system into one that makes pretty much any SQL Command something that a TAM can treat as being an optional request by the user;

Yes, and I think I'm perfectly correct in asking for that. If the standard you are proposing (albeit as Devil's Advocate) were applied to filesystems, nobody could ever implement /dev/null, on the argument that users have a reasonable expectation that data they write to a file will be there for them to read later. Yet Michael Paquier wrote a blackhole TAM, and although I don't find it terribly useful, I do think it's a reasonable thing for somebody to be able to write.

> whereas today the system presumes that the implementations will respond to these commands.

That depends on what you mean by "respond to". A TAM which implements a tamper resistant audit log responds to update and delete commands with a "permission denied" error. A TAM which implements running aggregates implements insert commands by computing and inserting a new running aggregate value and reclaiming space from old running aggregate values when no transaction could any longer see them. You can do this stuff at a higher level with hooks, functions, triggers, and rules, inserting into a heap, and having to periodically vacuum, by why would you want to? That's almost guaranteed to be slower, maybe even orders of magnitude slower.

> And to make this change without any core code having such a need.

The core code won't have any such need, because the core code is content with heap, and the API already accommodates heap. It seems Andres moved the project in the direction of allowing custom TAMs when he created the Table AM interface, and I'm quite pleased that he did so, but it doesn't allow nearly enough flexibility to do all the interesting things a TAM could otherwise do. Consider for example that the multi_insert hook uses a BulkInsertStateData parameter, defined as:

typedef struct BulkInsertStateData
{
BufferAccessStrategy strategy; /* our BULKWRITE strategy object */
Buffer current_buf; /* current insertion target page */
} BulkInsertStateData;

which is just the structure heap would want, but what about a TAM that wants to route different tuples to different pages? The "current_buf" isn't enough information, and there's no void *extra field, so you're just sunk.

—
Mark Dilger
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company