Re: RFC: Async query processing

On Fri, Jan 03, 2014 at 03:06:11PM -0200, Claudio Freire wrote:
> On Fri, Jan 3, 2014 at 12:20 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> > Claudio Freire <klaussfreire(at)gmail(dot)com> writes:
> >> On Fri, Jan 3, 2014 at 10:22 AM, Florian Weimer <fweimer(at)redhat(dot)com> wrote:
> >>> Loading data into the database isn't such an uncommon task. Not everything
> >>> is OLTP.
> >
> >> Truly, but a sustained insert stream of 10 Mbps is certainly way
> >> beyond common non-OLTP loads. This is far more specific than non-OLTP.
> >
> > I think Florian has a good point there, and the reason is this: what
> > you are talking about will be of exactly zero use to applications that
> > want to see the results of one query before launching the next. Which
> > eliminates a whole lot of apps. I suspect that almost the *only*
> > common use case in which a stream of queries can be launched without
> > feedback is going to be bulk data loading. It's not clear at all
> > that pipelining the PQexec code path is the way to better performance
> > for that --- why not use COPY, instead?
>
> You're forgetting ORM workloads.
>
> ORMs can usually plan the inserts to be in a sequence that both don't
> require feedback (except the knowledge that they were successful), and
> that do not violate constraints.
>
> Flushing a whole object hierarchy for instance, can be done without
> feedback. Not even serial columns need feedback, since many ORMs
> (SQLAlchemy, Hibernate) support allocation of ID sequences in batches
> (by issuing a proper select nextval).

This is already doable using data-modifying WITH clauses. Whether
ORMs are (or should be made) smart enough to take advantage of this is
probably out of the scope of this discussion, though. For what it's
worth, I'm not in favor of orthogonality in languages, so +1 from me.
Cheers,
David.
--
David Fetter <david(at)fetter(dot)org> http://fetter.org/
Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter
Skype: davidfetter XMPP: david(dot)fetter(at)gmail(dot)com
iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate