Re: On columnar storage

On 6/14/15 10:22 AM, Alvaro Herrera wrote:
>> >To me, it feels like there are two different features here that would
>> >be better separated. First, there's the idea of having a table that
>> >gets auto-joined to other tables whenever you access it, so that the
>> >user sees one really wide table but really the data is segregated by
>> >column groups under the hood. That's a neat idea.
> Thanks. (It also seems pretty tricky to implement.)

I look at it as a form of vertical partitioning; the big difference
being whether you normalize the columns out or not (or to use data
warehouse parlance, slow vs fast changing dimensions).

Perhaps it would be useful to vet this out as a userspace extension
first since that would presumably be much easier. I believe we actually
have all the backend infrastructure that would be needed for this now
that views are smart enough to exclude tables that aren't referenced at
all. I suspect that even a 'dumb userspace' approach would still expose
a lot of the planner problems we'll run into (join explosion and
filtering through the join come to mind).

Related to idea of an 'auto join', I do wish we had the ability to
access columns in a referenced FK table from a referring key; something
like SELECT customer_id.first_name FROM invoice (which would be
translated to SELECT first_name FROM invoice JOIN customer USING(
customer_id )).
--
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Data in Trouble? Get it in Treble! http://BlueTreble.com