Re: performance problem on big tables

From: Mariel Cherkassky <mariel(dot)cherkassky(at)gmail(dot)com>
To: Claudio Freire <klaussfreire(at)gmail(dot)com>
Cc: "pgsql-performance(at)postgresql(dot)org" <pgsql-performance(at)postgresql(dot)org>
Subject: Re: performance problem on big tables
Date: 2017-08-24 07:51:11
Message-ID: CA+t6e1kUotu9EWU7qRe2DsrrvLzH-CzaRbKqPUnp-3J=XqUxyQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-performance

Hi Claudio, how can I do that ? Can you explain me what is this option ?

2017-08-24 2:15 GMT+03:00 Claudio Freire <klaussfreire(at)gmail(dot)com>:

> On Mon, Aug 21, 2017 at 5:00 AM, Mariel Cherkassky
> <mariel(dot)cherkassky(at)gmail(dot)com> wrote:
> > To summarize, I still have performance problems. My current situation :
> >
> > I'm trying to copy the data of many tables in the oracle database into my
> > postgresql tables. I'm doing so by running insert into
> local_postgresql_temp
> > select * from remote_oracle_table. The performance of this operation are
> > very slow and I tried to check the reason for that and mybe choose a
> > different alternative.
> >
> > 1)First method - Insert into local_postgresql_table select * from
> > remote_oracle_table this generated total disk write of 7 M/s and actual
> disk
> > write of 4 M/s(iotop). For 32G table it took me 2 hours and 30 minutes.
> >
> > 2)second method - copy (select * from oracle_remote_table) to /tmp/dump
> > generates total disk write of 4 M/s and actuval disk write of 100 K/s.
> The
> > copy utility suppose to be very fast but it seems very slow.
>
> Have you tried increasing the prefetch option in the remote table?
>
> If you left it in its default, latency could be hurting your ability
> to saturate the network.
>

In response to

Responses

Browse pgsql-performance by date

  From Date Subject
Next Message Mariel Cherkassky 2017-08-24 13:15:19 printing results of query to file in different times
Previous Message Claudio Freire 2017-08-23 23:15:44 Re: performance problem on big tables