| From: | Daniele Varrazzo <daniele(dot)varrazzo(at)gmail(dot)com> |
|---|---|
| To: | "psycopg(at)postgresql(dot)org" <psycopg(at)postgresql(dot)org> |
| Subject: | Re: speed concerns with executemany() |
| Date: | 2017-01-02 16:24:47 |
| Message-ID: | CA+mi_8aFtNS3jN6OTn00kcLD8tGBcN61dMDxj3pAaP5xvG1Svg@mail.gmail.com |
| Views: | Raw Message | Whole Thread | Download mbox | Resend email |
| Thread: | |
| Lists: | psycopg |
On Mon, Jan 2, 2017 at 5:16 PM, Karsten Hilbert <Karsten(dot)Hilbert(at)gmx(dot)net> wrote:
> On Mon, Jan 02, 2017 at 05:10:10PM +0100, Daniele Varrazzo wrote:
>
>> > Any chance 'l' can be supported to be a generator expression ?
>>
>> The function would consume only page_size records at time. This is to
>> avoid creating a humongous query in memory and then on the server, but
>> has the nice effect of requesting only as many records from the
>> sequence.
>
> Nice.
>
> Can said sequence be a generator (IOW not needing to support
> len() upfront) ?
Correct: see the implementation of paginate() in the gist at
<https://gist.github.com/dvarrazzo/4204cca5d1cd7c9c95bc814d81e6b83e>:
the sequence is only consumed via next(). Final implementation may
change but accessing the sequence only as a generator is a desired
feature for me too.
-- Daniele
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Karsten Hilbert | 2017-01-02 16:27:44 | Re: speed concerns with executemany() |
| Previous Message | Daniele Varrazzo | 2017-01-02 16:21:31 | Re: Solving the SQL composition problem |