Re: [pgAdmin4][patch][runtime]: RM#2829, RM#2491 - pgAdmin4 crashes while saving CSV data from Query tool

On Fri, Nov 17, 2017 at 8:45 AM, Neel Patel <neel(dot)patel(at)enterprisedb(dot)com>
wrote:

> Hi Dave,
>
> On Thu, Nov 16, 2017 at 8:13 PM, Dave Page <dpage(at)pgadmin(dot)org> wrote:
>
>> Hi
>>
>> On Thu, Nov 16, 2017 at 1:47 PM, Neel Patel <neel(dot)patel(at)enterprisedb(dot)com>
>> wrote:
>>
>>> Hi,
>>>
>>> Adding information.
>>>
>>> With this patch, RM#2715 should also be resolved.
>>>
>>> Thanks,
>>> Neel Patel
>>>
>>> On Thu, Nov 16, 2017 at 7:01 PM, Neel Patel <neel(dot)patel(at)enterprisedb(dot)com
>>> > wrote:
>>>
>>>> Hi,
>>>>
>>>> I am able to reproduce the crash while downloading and save data in CSV
>>>> file from query tool.
>>>>
>>>> Please find attached updated patch with below changes after reading Qt
>>>> documentation.
>>>>
>>>> - Added new signal "readyRead". As per the Qt documentation, this
>>>> signal will be emitted when data is ready from IO channel for large amount
>>>> of data transfer between server and client.
>>>> - Ready read and DownloadInProgress signal is very quick in call so
>>>> we should not do large operation inside that slot because for downloading
>>>> big data it may possible of frequent call of those signals which may cause
>>>> the crash or missing data to write inside the file so removed unnecessary
>>>> logic from that slot.
>>>> - Fixed the crash while opening IODevice with NULL handle.
>>>>
>>>> With above changes, I have tested with same data as earlier and it is
>>>> working as expected without crashing the application.
>>>>
>>>> Do review it and let me know for comments.
>>>>
>>>
>> My first test was on Mac using Qt 5.8 with webkit, and when I attempted
>> to download the results from a 300K row query, it basically hung showing
>> the Downloading File progress indicator. It let me cancel it and carried on
>> working, but hung again the next time I tried the CSV download. Each time
>> it seems to download some data - from the same query I've seen 2.8MB, 5.1MB
>> and 1.5MB.
>>
>
> We are using "*downloadProgress*" signal to update the progress bar and
> here we are getting "readData" as bytes but "totalData" is unknown so we
> received "totalData" as -1 because Qt doesn't know the total size of the
> file we are getting from server but when we open "pgAdmin4" website in
> another link and try to download some binary or source at that time we are
> getting "totalData" as known size and we are displaying the progress bar
> correctly.
>
> Here, Qt sending the signal "downloadProgress" with unknown totalData size
> as -1 so we update the progress bar with 100% so from user point of view it
> looks like hang but it is downloading and writing to file at back ground.
> if we apply the same patch in windows then it display running progress bar
> so user can know something happening at back end side.
>
> I think it is same as browser - if we download big size file from server
> then it keep on downloading the file without displaying total size. What
> should we do here ? Should we create one custom progress bar dialog with
> moving cursor or something other so that user can know that activity is
> doing at backend ? Thoughts ?
>

It is misleading, but it *is* also hanging. 10+ minutes to download a few
megs of data from a server on the same machine is not just a misleading
progress indicator :-)

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company