large query by offset and limt

From: finecur <finecur(at)yahoo(dot)com>
To: pgsql-general(at)postgresql(dot)org
Subject: large query by offset and limt
Date: 2008-05-02 21:01:43
Message-ID: 69019638-6acd-4e14-af1b-dea697a62620@p25g2000pri.googlegroups.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

Hi, I am ruuning a database behind a webserver and there is a table
which is huge. I need to pull data from this table and send to user
through http. If I use

select * from huge_table where userid = 100

It will return millions of records which exhuasts my server's memory.
So I do this:

select * from huge_table where userid = 100 limit 1000 offset 0
and then send the results to user, then

select * from huge_table where userid = 100 limit 1000 offset 1000
and then send the results to user, then

select * from huge_table where userid = 100 limit 1000 offset 2000
and then send the results to user,

Continue this until there is no records available

It runs great but it is kind of slow. I think it is because even I
need only 1000 records, the query search the whole table every time.

Is there a better way to do this?

Thank you.

ff

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Andreas 'ads' Scherbaum 2008-05-02 21:06:15 Re: How to modify ENUM datatypes?
Previous Message Andrew Sullivan 2008-05-02 20:41:01 Re: How to modify ENUM datatypes?