Re: [pgAdmin4]: Webpacking of static JS/CSS

On Tue, Jul 18, 2017 at 7:46 PM, Dave Page <dpage(at)pgadmin(dot)org> wrote:

> Thanks - applied!
>
> Awesome work - on an average of 3 tests on my Mac, load time reduced from
> 11.55s with v1.6 to 5.53s with GIT Head.
>
> Surinder, great work...

> On Mon, Jul 17, 2017 at 5:57 PM, Surinder Kumar <
> surinder(dot)kumar(at)enterprisedb(dot)com> wrote:
>
>> Hi
>>
>> Now all test cases are executing.
>> Please find updated patch.
>>
>> Thanks
>> Surinder
>>
>> On Mon, Jul 17, 2017 at 6:57 PM, Surinder Kumar <
>> surinder(dot)kumar(at)enterprisedb(dot)com> wrote:
>>
>>> On Mon, Jul 17, 2017 at 4:52 PM, Dave Page <dpage(at)pgadmin(dot)org> wrote:
>>>
>>>> Hi
>>>>
>>>> No errors now, but do you know why JS tests are being skipped?
>>>>
>>> ​No errors/warning in console even after settings `logLevel:
>>> config.LOG_DEBUG`. I am still debugging.
>>>
>>>>
>>>> PhantomJS 2.1.1 (Mac OS X 0.0.0): Executed 4 of 216 (skipped 212)
>>>> SUCCESS (0.085 secs / 0.046 secs)
>>>>
>>>> Thanks!
>>>>
>>>> On Mon, Jul 17, 2017 at 12:07 PM, Surinder Kumar <
>>>> surinder(dot)kumar(at)enterprisedb(dot)com> wrote:
>>>>
>>>>> ​Hi Dave,
>>>>>
>>>>> I didn't removed the vendor modules when i ran regression test cases,
>>>>> so modules were being referenced from vendor dir and passed for me.
>>>>> Now I have fixed path references and test cases are working.
>>>>>
>>>>> Please find attached patch.
>>>>>
>>>>> Thanks
>>>>> Surinder​
>>>>>
>>>>> On Mon, Jul 17, 2017 at 3:18 PM, Surinder Kumar <
>>>>> surinder(dot)kumar(at)enterprisedb(dot)com> wrote:
>>>>>
>>>>>> Hi
>>>>>>
>>>>>> I'm currently working on first TODO: "Automatically handle static and
>>>>>> template JS files"
>>>>>>
>>>>>> As discussed with Ashesh, currently the paths to module id are
>>>>>> written manually in webpack.config.js, instead the path defined in moudle's
>>>>>> `def get_own_javascript()` should be used.
>>>>>>
>>>>>> So, we will be generating a paths.json file which will contain:
>>>>>>
>>>>>> 1. resolve > alias - path with reference to module id.(Static files)
>>>>>>
>>>>>> 2. externals - list of modules to be loaded dynamically on
>>>>>> demand(Template files)
>>>>>>
>>>>>> 3. Shim module dependency
>>>>>>
>>>>>> 4. List of JS modules to be loaded in specified order.
>>>>>>
>>>>>> *Implementation:*
>>>>>>
>>>>>> To generate `paths.json` file, we will be using `Flask's test_client`
>>>>>> to make an http request internally within the app context so we can call
>>>>>> `current_app.javascripts` property and return the list of JS paths and
>>>>>> write those into paths.json file and then use it in webpack.shim.js before
>>>>>> the execution of `yarn run bundle` in `javascript_bundler.py`
>>>>>>
>>>>>> *For example:*
>>>>>>
>>>>>> @app.route('/get_script_paths')
>>>>>> def get_script_paths():
>>>>>> from flask import current_app
>>>>>> from pgadmin.utils.ajax import make_json_response
>>>>>>
>>>>>> return make_json_response(data=current_app.javascripts)
>>>>>>
>>>>>> if config.DEBUG:
>>>>>> with app.test_client() as client:
>>>>>> import simplejson as json
>>>>>> list_scripts = client.get('/get_script_paths')
>>>>>> scripts = json.loads(list_scripts.data)
>>>>>>
>>>>>> javascriptBundler = JavascriptBundler()
>>>>>> javascriptBundler.bundle(scripts['data'])
>>>>>>
>>>>>>
>>>>>> This also needs little change in module dependency we defined using
>>>>>> 'When': 'node_name' in `def get_own_javascripts(...)` method
>>>>>> the module specified(name: module_name) is loaded when module given
>>>>>> in `When` is expanded in node. Since we are using Webpack in which
>>>>>> behaviour to load module is little different.
>>>>>>
>>>>>> Now in webpack we are using `imports-loader` to load specific
>>>>>> modules. So this is how it should work.
>>>>>>
>>>>>> 1. First load all modules which do not have dependency on any node
>>>>>> like 'about', 'dashboard', 'server-group', 'server' etc.
>>>>>>
>>>>>> 2. Load module such as `Databases` node first before its child nodes
>>>>>> are loaded.
>>>>>> Similarly load `Schemas` node before its child nodes are loaded as
>>>>>> they are dependent on parent node.
>>>>>>
>>>>>> Thanks,
>>>>>> Surinder
>>>>>> On Wed, Jul 5, 2017 at 8:22 PM, Sarah McAlear <smcalear(at)pivotal(dot)io>
>>>>>> wrote:
>>>>>>
>>>>>>> Hello,
>>>>>>>
>>>>>>>
>>>>>>>> *​Things to discuss:*
>>>>>>>>
>>>>>>>> How to differentiate between a static and template JS
>>>>>>>> ​​
>>>>>>>> .
>>>>>>>>
>>>>>>>
>>>>>>> What is the advantage of webpacking templated JS? It seems as though
>>>>>>> this creates a system in which the bundled dependencies have to refer back
>>>>>>> to the backend to load the templates.
>>>>>>>
>>>>>> ​Templated JS will not be part of generated bundle JS, they will load
>>>>>> externally( an extra request will be made to server For example:
>>>>>> translations.js)
>>>>>>
>>>>>>>
>>>>>>> If there is a performance win in packing templated JS then looking
>>>>>>> at it makes sense. Otherwise it may make sense to put off until it is
>>>>>>> clear that the templated files should be dealt with by either de-templating
>>>>>>> them or bundling them where there is a clear reason.
>>>>>>>
>>>>>> ​Template JS cannot be bundled, so i extract the <Jinja> code from
>>>>>> template files and put into a separate file - ABC.js (also moved template
>>>>>> files to static directory) and then load ABC.js dynamically as dependency
>>>>>> of other modules.
>>>>>>
>>>>>>>
>>>>>>> However, we're wondering about possible performance penalties with
>>>>>>> templating larger files (as opposed to templating on-demand.) Since jinja
>>>>>>> templates can execute arbitrary python, this could get time expensive and
>>>>>>> further slow things like initial page-load.
>>>>>>> Another concern is: what happens when a template gets out of date
>>>>>>> (e.g. if browser.js had previously filled in the content for
>>>>>>> 'panel_item.content' and had been cached, would it render a new version
>>>>>>> with the new values when needed? Or is it possible that we would get old
>>>>>>> content?)
>>>>>>>
>>>>>> ​That file will always gets new content when loaded dynamically, the
>>>>>> content is not cached.​
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> *Taks remaining:*
>>>>>>>>
>>>>>>>> ​1. ​
>>>>>>>> Fix local variables which are declared without using var, have to
>>>>>>>> check in each file
>>>>>>>> ​ by​
>>>>>>>> running eslint (For now, i will fix only errors which are giving
>>>>>>>> error in browser).
>>>>>>>>
>>>>>>>> ​2. ​
>>>>>>>> Move non-template files from ’templates’ to ’static’ directory.
>>>>>>>> List of
>>>>>>>> ​ pending​
>>>>>>>> modules is here:
>>>>>>>>
>>>>>>>> - Tools (mostly all modules - 9 modules)
>>>>>>>> - Browser nodes - 3 modules(resource group, roles, tablespace)
>>>>>>>> - ​About
>>>>>>>> ​​
>>>>>>>>
>>>>>>>> Also can we move
>>>>>>>> ​'​
>>>>>>>> dashboard, statistic
>>>>>>>> ​s​
>>>>>>>> , preferences and help
>>>>>>>> ​'​
>>>>>>>> modules inside misc to preserve modularity as pgAdmin is modular
>>>>>>>> ​ ?​
>>>>>>>>
>>>>>>>
>>>>>>> Is there anything from a organization stance you discussed in the
>>>>>>> previous email that needs to be done to make this usable and consistent?
>>>>>>>
>>>>>> ​No​
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> George & Sarah
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Dave Page
>>>> Blog: http://pgsnake.blogspot.com
>>>> Twitter: @pgsnake
>>>>
>>>> EnterpriseDB UK: http://www.enterprisedb.com
>>>> The Enterprise PostgreSQL Company
>>>>
>>>
>>>
>>
>
>
> --
> Dave Page
> Blog: http://pgsnake.blogspot.com
> Twitter: @pgsnake
>
> EnterpriseDB UK: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>