Re: Fixed RM #1356

Hi Dave

On Fri, Jun 17, 2016 at 9:00 PM, Dave Page <dpage(at)pgadmin(dot)org> wrote:

> On Fri, Jun 17, 2016 at 4:25 PM, Ashesh Vashi
> <ashesh(dot)vashi(at)enterprisedb(dot)com> wrote:
> >
> > On Jun 17, 2016 20:53, "Dave Page" <dpage(at)pgadmin(dot)org> wrote:
> >>
> >>
> >>
> >> On Fri, Jun 17, 2016 at 4:04 PM, Akshay Joshi
> >> <akshay(dot)joshi(at)enterprisedb(dot)com> wrote:
> >>>
> >>> Hi Dave
> >>>
> >>> On Thu, Jun 16, 2016 at 6:48 PM, Dave Page <dpage(at)pgadmin(dot)org> wrote:
> >>>>
> >>>>
> >>>>
> >>>> On Thu, Jun 16, 2016 at 1:43 PM, Akshay Joshi
> >>>> <akshay(dot)joshi(at)enterprisedb(dot)com> wrote:
> >>>>>
> >>>>>
> >>>>>
> >>>>> On Thu, Jun 16, 2016 at 6:09 PM, Dave Page <dpage(at)pgadmin(dot)org>
> wrote:
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> On Thu, Jun 16, 2016 at 1:34 PM, Akshay Joshi
> >>>>>> <akshay(dot)joshi(at)enterprisedb(dot)com> wrote:
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> On Thu, Jun 16, 2016 at 5:47 PM, Khushboo Vashi
> >>>>>>> <khushboo(dot)vashi(at)enterprisedb(dot)com> wrote:
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Thu, Jun 16, 2016 at 5:07 PM, Dave Page <dpage(at)pgadmin(dot)org>
> >>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Thu, Jun 16, 2016 at 12:19 PM, Khushboo Vashi
> >>>>>>>>> <khushboo(dot)vashi(at)enterprisedb(dot)com> wrote:
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On Thu, Jun 16, 2016 at 4:42 PM, Dave Page <dpage(at)pgadmin(dot)org>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Thu, Jun 16, 2016 at 12:04 PM, Akshay Joshi
> >>>>>>>>>>> <akshay(dot)joshi(at)enterprisedb(dot)com> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>> Hi Dave
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Thu, Jun 16, 2016 at 2:42 PM, Akshay Joshi
> >>>>>>>>>>>> <akshay(dot)joshi(at)enterprisedb(dot)com> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Thu, Jun 16, 2016 at 2:35 PM, Dave Page <
> dpage(at)pgadmin(dot)org>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Thanks, patch applied.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> However, whilst I was testing, I saw just how slow the tool
> >>>>>>>>>>>>>> is:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> SELECT * FROM pg_attribute
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> In a PEM database, returns 8150 rows. In pgAdmin 3, this is
> >>>>>>>>>>>>>> timed at 676ms on my laptop. In pgAdmin 4, the busy spinner
> runs for approx
> >>>>>>>>>>>>>> 5 seconds, then the whole UI freezes. I then have to wait a
> further 3
> >>>>>>>>>>>>>> minutes and 46 seconds(!!!!) for the operation to complete.
> Once loaded,
> >>>>>>>>>>>>>> scrolling is very sluggish.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Please make this your top priority - and if you have
> >>>>>>>>>>>>>> incremental improvements, send them as you have them.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Sure.
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> Below is my initial finding while running "SELECT * FROM
> >>>>>>>>>>>> pg_attribute" on PEM database, returns 8498 rows:
> >>>>>>>>>>>> Fetching data from the server side took consistent time and it
> >>>>>>>>>>>> took 3-4 secs.
> >>>>>>>>>>>> Create/Render Backgrid without pagination : 1 minute
> >>>>>>>>>>>> Create/Render Backgrid with pagination (50 items per page):
> >>>>>>>>>>>> 469ms
> >>>>>>>>>>>> Create/Render Backgrid with pagination (500 items per page): 3
> >>>>>>>>>>>> secs
> >>>>>>>>>>>> Create/Render Backgrid with pagination (1000 items per page):
> 6
> >>>>>>>>>>>> secs
> >>>>>>>>>>>> Create/Render Backgrid with pagination (3000 items per page):
> 22
> >>>>>>>>>>>> secs
> >>>>>>>>>>>> Create/Render Backgrid with pagination (5000 items per page):
> 36
> >>>>>>>>>>>> secs
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> OK, so I guess diving into Backgrid is the next step. Are there
> >>>>>>>>>>> any profiling tools that could be used?
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> Can we use infinity scrolling in case of no pagination?
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> How would add row work then?
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> Yeah, in this case user has to wait till the last record to load.
> :(
> >>>>>>>> Btw, I was thinking of
> >>>>>>>> https://github.com/bhvaleri/backgrid-infinator
> >>>>>>>
> >>>>>>>
> >>>>>>> This seems to be the good option.
> >>>>>>
> >>>>>>
> >>>>>> No - please see my previous comment.
> >>>>>
> >>>>>
> >>>>> We can add/paste row as top row of the backgrid. In that case
> will
> >>>>> it be fine?
> >>>>
> >>>>
> >>>> It's a hack, it doesn't solve the underlying problem. The fact that it
> >>>> took 4 minutes to load 8000 rows on my top-of-the-range MacBook Pro
> is not
> >>>> good.
> >>>
> >>>
> >>> I have tried to fix the issue, but unable to figure out any way to
> do
> >>> it . I have tried following options
> >>> Same issue found here https://github.com/wyuenho/backgrid/issues/126
> >>> Which will be fixed https://github.com/wyuenho/backgrid/pull/444. I
> have
> >>> copied the backgrid.js and backgrid.css from "perf" branch replace it
> in our
> >>> code, but faced so many error's and warning, fixed those but some how
> data
> >>> is not rendered.
> >>
> >> Hmm, that's so old I'm not holding my breath about it being included in
> a
> >> hurry.
> >>
> >>>
> >>> Another approach is instead of adding all the records to the backbone
> >>> collection at once, I have added them in chunk of 500 records at a
> time and
> >>> sleep for 500ms, in this case busy spinner won't run for a longer time
> as
> >>> first 500 records gets rendered, but browser again goes into
> unresponsive
> >>> state as CPU goes up to 98-99%.
> >>
> >> Urgh. I wonder if we need a different grid control for this, something
> >> like SlickGrid which is extremely fast with large data sets.
> >>
> >> https://github.com/6pac/SlickGrid
> > That's what I was researching about.
> > But - we need to research on adding row, copy rows, etc.
>
> The examples (http://mleibman.github.io/SlickGrid/examples/) certainly
> look promising - it can add/edit rows, copy/paste arbitrary selections
> etc. It can even display graphs in cells, or use templates for
> rendering them.
>
> There are also a couple of shims to backbone out there, though I'm not
> sure if that would just slow things down again.
>

Should we start working on SlickGrid. If we are planning to use this,
first we will have to identify the feature's given in the Query Tool would
be possible with SlickGrid and also It will take time to implement. If not
then we shouldn't allow user to set items per page = 0 (no pagination), we
should set the minimum(50 or 100) and maximum(2000-2500) value for the time
being to avoid the issue.

>
> --
> Dave Page
> Blog: http://pgsnake.blogspot.com
> Twitter: @pgsnake
>
> EnterpriseDB UK: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>

--
*Akshay Joshi*
*Principal Software Engineer *

*Phone: +91 20-3058-9517Mobile: +91 976-788-8246*