From: | Gunther <raj(at)gusw(dot)net> |
---|---|
To: | "pgsql-performance(at)lists(dot)postgresql(dot)org" <pgsql-performance(at)lists(dot)postgresql(dot)org> |
Subject: | FPGA optimization ... |
Date: | 2019-11-04 23:33:15 |
Message-ID: | e41deea2-9594-36dc-be76-47c39368911e@gusw.net |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-performance |
The time has come.
FPGA optimization is in the palm of our hands (literally a 2 TB 40 GB/s
IO PostgreSQL server fits into less than a shoe box), and on Amazon AWS
F1 instances.
Some demos are beginning to exist: https://github.com/Xilinx/data-analytics.
<https://github.com/Xilinx/data-analytics>
But a lot more could be done. How about linear sort performance at O(N)?
https://hackaday.com/2016/01/20/a-linear-time-sorting-algorithm-for-fpgas/.
And how about https://people.csail.mit.edu/wjun/papers/fccm2017.pdf,
the following four sorting accelerators are used:
* Tuple Sorter : Sorts an N-tuple using a sorting network.
* Page Sorter : Sorts an 8KB (a flash page) chunk of sorted N-tuples
in on-chip memory.
* Super-Page Sorter : Sorts 16 8K-32MB sorted chunks in DRAM.
* Storage-to-Storage Sorter: Sorts 16 512MB or larger sorted chunks in
flash.
Order of magnitude speed improvements? Better than Hadoop clusters on a
single chip? 40 GB/s I/O throughput massive full table scan, blazing
fast sort-merge joins? Here it is. Anybody working more on that? Should
be an ideal project for a student or a group of students.
Is there a PostgreSQL foundation I could donate to, 501(c)(3) tax
exempt? I can donate and possibly find some people at Purdue University
who might take this on. Interest?
regards,
-Gunther
From | Date | Subject | |
---|---|---|---|
Next Message | Tomas Vondra | 2019-11-05 00:19:15 | Re: FPGA optimization ... |
Previous Message | Jeff Janes | 2019-11-04 21:05:12 | Re: Huge shared hit for small table |