Re: [pgAdmin4]: Webpacking of static JS/CSS

Thanks, applied.

On Tue, Jul 18, 2017 at 4:12 PM, Surinder Kumar <
surinder(dot)kumar(at)enterprisedb(dot)com> wrote:

> Hi
>
> 1. As Slickgrid has dependency of `jQuery-ui`, it was missed. now added.
> 2. Column sorting for collection nodes sometimes failing when clicked on
> different collection nodes.
>
> Please find attached patch.
>
> Thanks
> Surinder
>
> On Tue, Jul 18, 2017 at 8:20 PM, Khushboo Vashi <
> khushboo(dot)vashi(at)enterprisedb(dot)com> wrote:
>
>>
>>
>> On Tue, Jul 18, 2017 at 7:46 PM, Dave Page <dpage(at)pgadmin(dot)org> wrote:
>>
>>> Thanks - applied!
>>>
>>> Awesome work - on an average of 3 tests on my Mac, load time reduced
>>> from 11.55s with v1.6 to 5.53s with GIT Head.
>>>
>> ​Thanks to all​
>
>>
>>> Surinder, great work...
>>
>>
>>> On Mon, Jul 17, 2017 at 5:57 PM, Surinder Kumar <
>>> surinder(dot)kumar(at)enterprisedb(dot)com> wrote:
>>>
>>>> Hi
>>>>
>>>> Now all test cases are executing.
>>>> Please find updated patch.
>>>>
>>>> Thanks
>>>> Surinder
>>>>
>>>> On Mon, Jul 17, 2017 at 6:57 PM, Surinder Kumar <
>>>> surinder(dot)kumar(at)enterprisedb(dot)com> wrote:
>>>>
>>>>> On Mon, Jul 17, 2017 at 4:52 PM, Dave Page <dpage(at)pgadmin(dot)org> wrote:
>>>>>
>>>>>> Hi
>>>>>>
>>>>>> No errors now, but do you know why JS tests are being skipped?
>>>>>>
>>>>> ​No errors/warning in console even after settings `logLevel:
>>>>> config.LOG_DEBUG`. I am still debugging.
>>>>>
>>>>>>
>>>>>> PhantomJS 2.1.1 (Mac OS X 0.0.0): Executed 4 of 216 (skipped 212)
>>>>>> SUCCESS (0.085 secs / 0.046 secs)
>>>>>>
>>>>>> Thanks!
>>>>>>
>>>>>> On Mon, Jul 17, 2017 at 12:07 PM, Surinder Kumar <
>>>>>> surinder(dot)kumar(at)enterprisedb(dot)com> wrote:
>>>>>>
>>>>>>> ​Hi Dave,
>>>>>>>
>>>>>>> I didn't removed the vendor modules when i ran regression test
>>>>>>> cases, so modules were being referenced from vendor dir and passed for me.
>>>>>>> Now I have fixed path references and test cases are working.
>>>>>>>
>>>>>>> Please find attached patch.
>>>>>>>
>>>>>>> Thanks
>>>>>>> Surinder​
>>>>>>>
>>>>>>> On Mon, Jul 17, 2017 at 3:18 PM, Surinder Kumar <
>>>>>>> surinder(dot)kumar(at)enterprisedb(dot)com> wrote:
>>>>>>>
>>>>>>>> Hi
>>>>>>>>
>>>>>>>> I'm currently working on first TODO: "Automatically handle static
>>>>>>>> and template JS files"
>>>>>>>>
>>>>>>>> As discussed with Ashesh, currently the paths to module id are
>>>>>>>> written manually in webpack.config.js, instead the path defined in moudle's
>>>>>>>> `def get_own_javascript()` should be used.
>>>>>>>>
>>>>>>>> So, we will be generating a paths.json file which will contain:
>>>>>>>>
>>>>>>>> 1. resolve > alias - path with reference to module id.(Static files)
>>>>>>>>
>>>>>>>> 2. externals - list of modules to be loaded dynamically on
>>>>>>>> demand(Template files)
>>>>>>>>
>>>>>>>> 3. Shim module dependency
>>>>>>>>
>>>>>>>> 4. List of JS modules to be loaded in specified order.
>>>>>>>>
>>>>>>>> *Implementation:*
>>>>>>>>
>>>>>>>> To generate `paths.json` file, we will be using `Flask's
>>>>>>>> test_client` to make an http request internally within the app context so
>>>>>>>> we can call `current_app.javascripts` property and return the list of JS
>>>>>>>> paths and write those into paths.json file and then use it in
>>>>>>>> webpack.shim.js before the execution of `yarn run bundle` in
>>>>>>>> `javascript_bundler.py`
>>>>>>>>
>>>>>>>> *For example:*
>>>>>>>>
>>>>>>>> @app.route('/get_script_paths')
>>>>>>>> def get_script_paths():
>>>>>>>> from flask import current_app
>>>>>>>> from pgadmin.utils.ajax import make_json_response
>>>>>>>>
>>>>>>>> return make_json_response(data=current_app.javascripts)
>>>>>>>>
>>>>>>>> if config.DEBUG:
>>>>>>>> with app.test_client() as client:
>>>>>>>> import simplejson as json
>>>>>>>> list_scripts = client.get('/get_script_paths')
>>>>>>>> scripts = json.loads(list_scripts.data)
>>>>>>>>
>>>>>>>> javascriptBundler = JavascriptBundler()
>>>>>>>> javascriptBundler.bundle(scripts['data'])
>>>>>>>>
>>>>>>>>
>>>>>>>> This also needs little change in module dependency we defined using
>>>>>>>> 'When': 'node_name' in `def get_own_javascripts(...)` method
>>>>>>>> the module specified(name: module_name) is loaded when module given
>>>>>>>> in `When` is expanded in node. Since we are using Webpack in which
>>>>>>>> behaviour to load module is little different.
>>>>>>>>
>>>>>>>> Now in webpack we are using `imports-loader` to load specific
>>>>>>>> modules. So this is how it should work.
>>>>>>>>
>>>>>>>> 1. First load all modules which do not have dependency on any node
>>>>>>>> like 'about', 'dashboard', 'server-group', 'server' etc.
>>>>>>>>
>>>>>>>> 2. Load module such as `Databases` node first before its child
>>>>>>>> nodes are loaded.
>>>>>>>> Similarly load `Schemas` node before its child nodes are loaded as
>>>>>>>> they are dependent on parent node.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Surinder
>>>>>>>> On Wed, Jul 5, 2017 at 8:22 PM, Sarah McAlear <smcalear(at)pivotal(dot)io>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hello,
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> *​Things to discuss:*
>>>>>>>>>>
>>>>>>>>>> How to differentiate between a static and template JS
>>>>>>>>>> ​​
>>>>>>>>>> .
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> What is the advantage of webpacking templated JS? It seems as
>>>>>>>>> though this creates a system in which the bundled dependencies have to
>>>>>>>>> refer back to the backend to load the templates.
>>>>>>>>>
>>>>>>>> ​Templated JS will not be part of generated bundle JS, they will
>>>>>>>> load externally( an extra request will be made to server For example:
>>>>>>>> translations.js)
>>>>>>>>
>>>>>>>>>
>>>>>>>>> If there is a performance win in packing templated JS then looking
>>>>>>>>> at it makes sense. Otherwise it may make sense to put off until it is
>>>>>>>>> clear that the templated files should be dealt with by either de-templating
>>>>>>>>> them or bundling them where there is a clear reason.
>>>>>>>>>
>>>>>>>> ​Template JS cannot be bundled, so i extract the <Jinja> code from
>>>>>>>> template files and put into a separate file - ABC.js (also moved template
>>>>>>>> files to static directory) and then load ABC.js dynamically as dependency
>>>>>>>> of other modules.
>>>>>>>>
>>>>>>>>>
>>>>>>>>> However, we're wondering about possible performance penalties with
>>>>>>>>> templating larger files (as opposed to templating on-demand.) Since jinja
>>>>>>>>> templates can execute arbitrary python, this could get time expensive and
>>>>>>>>> further slow things like initial page-load.
>>>>>>>>> Another concern is: what happens when a template gets out of date
>>>>>>>>> (e.g. if browser.js had previously filled in the content for
>>>>>>>>> 'panel_item.content' and had been cached, would it render a new version
>>>>>>>>> with the new values when needed? Or is it possible that we would get old
>>>>>>>>> content?)
>>>>>>>>>
>>>>>>>> ​That file will always gets new content when loaded dynamically,
>>>>>>>> the content is not cached.​
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> *Taks remaining:*
>>>>>>>>>>
>>>>>>>>>> ​1. ​
>>>>>>>>>> Fix local variables which are declared without using var, have to
>>>>>>>>>> check in each file
>>>>>>>>>> ​ by​
>>>>>>>>>> running eslint (For now, i will fix only errors which are giving
>>>>>>>>>> error in browser).
>>>>>>>>>>
>>>>>>>>>> ​2. ​
>>>>>>>>>> Move non-template files from ’templates’ to ’static’ directory.
>>>>>>>>>> List of
>>>>>>>>>> ​ pending​
>>>>>>>>>> modules is here:
>>>>>>>>>>
>>>>>>>>>> - Tools (mostly all modules - 9 modules)
>>>>>>>>>> - Browser nodes - 3 modules(resource group, roles, tablespace)
>>>>>>>>>> - ​About
>>>>>>>>>> ​​
>>>>>>>>>>
>>>>>>>>>> Also can we move
>>>>>>>>>> ​'​
>>>>>>>>>> dashboard, statistic
>>>>>>>>>> ​s​
>>>>>>>>>> , preferences and help
>>>>>>>>>> ​'​
>>>>>>>>>> modules inside misc to preserve modularity as pgAdmin is modular
>>>>>>>>>> ​ ?​
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Is there anything from a organization stance you discussed in the
>>>>>>>>> previous email that needs to be done to make this usable and consistent?
>>>>>>>>>
>>>>>>>> ​No​
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>>
>>>>>>>>> George & Sarah
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Dave Page
>>>>>> Blog: http://pgsnake.blogspot.com
>>>>>> Twitter: @pgsnake
>>>>>>
>>>>>> EnterpriseDB UK: http://www.enterprisedb.com
>>>>>> The Enterprise PostgreSQL Company
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>>
>>> --
>>> Dave Page
>>> Blog: http://pgsnake.blogspot.com
>>> Twitter: @pgsnake
>>>
>>> EnterpriseDB UK: http://www.enterprisedb.com
>>> The Enterprise PostgreSQL Company
>>>
>>
>>
>

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company