Re: FPGA optimization ...

Hi,

On 2019-11-06 22:54:48 +0100, Tomas Vondra wrote:
> If we're only talking about FPGA I/O acceleration, essentially FPGA
> between the database and storage, it's likely possible to get that
> working without any extensive executor changes. Essentially create an
> FPGA-aware variant of SeqScan and you're done. Or an FPGA-aware
> tuplesort, or something like that. Neither of this should require
> significant planner/executor changes, except for costing.

I doubt that that is true. For one, you either need to teach the FPGA
to understand at least enough about the intricacies of postgres storage
format, to be able to make enough sense of visibility information to
know when it safe to look at a tuple (you can't evaluate qual's before
visibility information). It also needs to be fed a lot of information
about the layout of the table, involved operators etc. And even if you
define those away somehow, you still need to make sure that the on-disk
state is coherent with the in-memory state - which definitely requires
reaching outside of just a replacement seqscan node.

I've a hard time believing that, even though some storage vendors are
pushing this model heavily, the approach of performing qual evaluation
on the storage level is actually useful for anything close to a general
purpose database, especially a row store.

It's more realistic to have a model where the fpga is fed pre-processed
data, and it streams out the processed results. That way there are no
problems with coherency, one can can transparently handle parts of
reading the data that the FPGA can't, etc.

But I admit I'm sceptical even the above model is relevant for
postgres. The potential market seems likely to stay small, and there's
so much more performance work that's applicable to everyone using PG,
even without access to special purpose hardware.

Greetings,

Andres Freund