Re: Hadoop backend?

From: Josh Berkus <josh(at)agliodbs(dot)com>
To: Paul Sheer <paulsheer(at)gmail(dot)com>
Cc: pgsql-hackers(at)postgresql(dot)org
Subject: Re: Hadoop backend?
Date: 2009-02-24 19:30:12
Message-ID: 49A44AC4.8040002@agliodbs.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers


> With a distributed data store, the data would become a logical
> object - no adding or removal of machines would affect the data.
> This is an ideal that would remove a tremendous maintenance
> burden from many sites ---- well, at least the one's I have worked
> at as far as I can see.

Two things:

1) Hadoop is the wrong technology. It's not designed to support
transactional operations.

2) Transactional operations are, in general, your Big Obstacle for doing
anything in the way of a distributed storage manager.

It's possible you could make both of the above "go away" if you were
planning for a DW platform in which transactions weren't important.
However, that would have to become an incompatible fork of PostgreSQL.

AFAIK, the Yahoo platform does not involve Hadoop at all.

--Josh

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Joshua D. Drake 2009-02-24 19:30:48 Re: Synchronous replication & Hot standby patches
Previous Message Jeff Davis 2009-02-24 19:28:31 Re: GIN fast insert