Re: Extensibility of the PostgreSQL wire protocol

Attached are a first patch and a functioning extension that implements a
telnet protocol server.

The extension needs to be loaded via shared_preload_libraries and
configured for a port number and listen_addresses as follows:

shared_preload_libraries = 'telnet_srv'

telnet_srv.listen_addresses = '*'
telnet_srv.port = 54323

It is incomplete in that it doesn't address things like the COPY protocol.
But it is enough to give a more detailed idea of what this interface will
look like and what someone would do to implement their own protocol or
extend an existing one.

The overall idea here is to route all functions, that communicate with the
frontend, through function pointers that hang off of MyProcPort. Since we
are performing socket communication in them I believe one extra function
pointer indirection is unlikely to have significant performance impact.

Best Regards, Jan
On behalf of Amazon Web Services

On Sun, Feb 14, 2021 at 12:36 PM Dave Cramer <davecramer(at)postgres(dot)rocks>
wrote:

>
>
> On Thu, 11 Feb 2021 at 09:28, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
>
>> On Wed, Feb 10, 2021 at 2:04 PM Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>> > That is a spot-on definition of where I do NOT want to end up. Hooks
>> > everywhere and enormous extensions that break anytime we change anything
>> > in the core. It's not really clear that anybody is going to find that
>> > more maintainable than a straight fork, except to the extent that it
>> > enables the erstwhile forkers to shove some of their work onto the PG
>> > community.
>>
>> +1.
>>
>> Making the lexer and parser extensible seems desirable to me. It would
>> be beneficial not only for companies like EDB and Amazon that might
>> want to extend the grammar in various ways, but also for extension
>> authors. However, it's vastly harder than Jan's proposal to make the
>> wire protocol pluggable. The wire protocol is pretty well-isolated
>> from the rest of the system. As long as you can get queries out of the
>> packets the client sends and package up the results to send back, it's
>> all good.
>
>
> I would have to disagree that the wire protocol is well-isolated. Sending
> and receiving are not in a single file
> The codes are not even named constants so trying to find a specific one is
> difficult.
>
> Anything that would clean this up would be a benefit
>
>
> That being said, I'm not in favor of transferring maintenance work to
>> the community for this set of hooks any more than I am for something
>> on the parsing side. In general, I'm in favor of as much extensibility
>> as we can reasonably create, but with a complicated proposal like this
>> one, the community should expect to be able to get something out of
>> it. And so far what I hear Jan saying is that these hooks could in
>> theory be used for things other than Amazon's proprietary efforts and
>> those things could in theory bring benefits to the community, but
>> there are no actual plans to do anything with this that would benefit
>> anyone other than Amazon. Which seems to bring us right back to
>> expecting the community to maintain things for the benefit of
>> third-party forks.
>>
>
> if this proposal brought us the ability stream results that would be a
> huge plus!
>
> Dave Cramer
> www.postgres.rocks
>
>>
>>

--
Jan Wieck