questions about possible enhancing protocol of communication between psql and pager

Hi

I am starting to think about the next generation of pspg (
https://github.com/okbob/pspg).

Now, the communication between psql and pspg is very simple. psql reads all
data, does formatting to pretty table format, and sends it through pipe to
pspg. pspg stores all data and tries to detect header lines that are used
for identification of column widths. The information about number of header
rows and about width of columns are used for printing fixed or moveable
data.

It is working surprisingly well, but there are limits.

1. In some cases it can be slow - you can try \x and select * from pg_proc.
The formatted table can be full of spaces, and the formatting can be slow,
and passing via pipe too. The difference is in 2 versus 20 seconds.

2. It cannot to work when FETCH_COUNT is non zero

Passing data in csv format to pager can be very significantly faster.
Processing csv data can be much more robust than processing tabular format
that depends on a lot of pset settings. Unfortunately, psql doesn't send in
csv all information. There is not any information about used data types,
there is no title. Unfortunately, there is not any info about wanted
formatting settings from psql - so the user's comfort is less than could be.

Can be nice (from my perspective) if pspg can read some metadata about the
result. The question is - how to do it?

There are three possibilities:

a) psql sends some control data through a pipe. Currently we use only text
protocol, but we can use some ascii control chars, so it is not a problem
to detect start of header, and detect start of data, and possibly we can
detect end of data.

b) psql can send data like now, but before the start of the pager can fill
some environment variables. A pager can read these variables - like
PSQL_PAGER_SETTING, PSQL_PAGER_DATADESC, ...

c) we can introduce a new custom format (can be named "pspg")- it can be
based on csv or tsv, where the first part is data description, following
data, and it can be ended by some special flag.

What do you think about described possibilities?

regards

Pavel