Re: [pgAdmin4]: Webpacking of static JS/CSS

Hi All,

*​Things to discuss:*

How to differentiate between a static and template JS
​​
.

​In Webpack we have to resolve paths to the modules same as in
requirejs.config(base.html). but previously we were using
'current_app.javascripts' which holds the the path references for modules
registered. Now in webpack.config.js we don't have such variable. so the
path to module js has to give manually.

However, there are ​t
wo ways:

​1. ​
The convention should be that static files should go in ‘static/‘ dir and
templates in ‘templates/‘ dir.
​ If this convention is followed, we can search through directories and
pull out static and template file paths.​

​2. ​
As per Ashesh,
​'​
static/
'​
dir may also have templates file
​ or vice versa​
​.​

​I​
n this case, the convention to identify template file
​can be
,
​templates ​
file must ends with ‘.template.js’ extension
​.
​
The flag ‘is_template’ added in every module’s __init__.py doesn’t help
here because webpack works on files and directories.
​ so there is no way to use this flag.​

*For debug/release mode:*

We’ll get rid of snippets in
​'​
get_own_javascripts
​'​
and
​'​
get_own_stylesheets
​'​
defined in modules __init__.py as they are
​ of​
no use for now as CSS and JS are
​ going to​
bundl
​e​
. They
​were
used to load specified CSS and JS files
​ at runtime.​

Introduce a flag ‘-p’ in
​'​
yarn run bundle
​ -p'
command to tell webpack config to run in development or production mode.
-p is for production.

Few
​W​
ebpack plugins such as UglifyPlugin or other plugins will run only in
production mode to minify JS otherwise load the generated bundle
JS(unminified) in debug mode to help with debugging code.

*Taks remaining:*

​1. ​
Fix local variables which are declared without using var, have to check in
each file
​ by​
running eslint (For now, i will fix only errors which are giving error in
browser).

​2. ​
Move non-template files from ’templates’ to ’static’ directory. List of
​ pending​
modules is here:

- Tools (mostly all modules - 9 modules)
- Browser nodes - 3 modules(resource group, roles, tablespace)
- ​About
​​

Also can we move
​'​
dashboard, statistic
​s​
, preferences and help
​'​
modules inside misc to preserve modularity as pgAdmin is modular
​ ?​

​3. Client side implementation of 'url_for' for several modules.

​4. ​
Integrating CSS bundling patch, also pull out inline CSS
​ and unvendor the CSS.​
​Today i worked on it but it is not completed yet, has some effect on UI, i
am looking into it.​

​5. ​
FileManager module is not working
​
as it is loaded via html file
​ and the modules it calls are unavailable at that time.
​I will​
look into it
​.​
​​

​There are other minor issues that has to take care.​

​6. ​
Caching:

I haven’t read much in detail on caching, but it can used to help browser
know when to cache static resources and when to not. Basically, it give
names to generated bundled files with unique hash and that hash is changed
when a new change in file is detected on running ‘yarn run bundle’.
Also we have to use HTML Webpack plugin to regenerate
​'​
base.html
​'
again with new hash in bundled files
​.

​Thanks,
Surinder​

On Fri, Jun 30, 2017 at 4:40 PM, Dave Page <dpage(at)pgadmin(dot)org> wrote:

> Awesome work. Do you have an ETA for a feature complete test patch?
>
> On Fri, Jun 30, 2017 at 11:41 AM, Surinder Kumar <
> surinder(dot)kumar(at)enterprisedb(dot)com> wrote:
>
>> Hi
>>
>> So far, I have been able to generate bundled
>> 'sources/generated/browser.js' which contains only three browser modules
>> (Server-group, Server, Database) and load it from
>> 'browser/index.html'(entry point) and it works(except few JS error present
>> in file). so will fix them.
>>
>> Then I will add one by one dashboard module, nodes(cast, schema etc.),
>> tools which i earlier removed(because their JS had conflicts with vendor
>> JS) and make changes accordingly to webpack config and make other modules
>> to load properly.
>>
>> So far the JS files that won't be the part of bundled JS and will load
>> separately according to dependency defined in externals in webpack config:
>>
>> 1. browser/utils.js - This file is extracted from browser.js which
>> contains Jinja syntax related code for menus, panels and preference related
>> settings from browser.js
>> 2. endpoints.js
>> 3. messages.js
>> 4. supported_servers.js
>> 5. current_user.js
>> 6. gettext.js
>> 7. translations.js
>>
>> All other JS files will go into bundled JS.
>>
>> Currently replacing '{{ url_for }}' in dashboard JS with client side
>> 'url_for' and load with bundled JS.
>>
>> Thanks,
>> Surinder
>>
>> On Fri, Jun 30, 2017 at 3:34 PM, Surinder Kumar <
>> surinder(dot)kumar(at)enterprisedb(dot)com> wrote:
>>
>>> Hi
>>>
>>> On Fri, Jun 30, 2017 at 2:29 AM, George Gelashvili <
>>> ggelashvili(at)pivotal(dot)io> wrote:
>>>
>>>> Hey Surinder,
>>>>
>>>>
>>>> ​1. ​
>>>>> Tools
>>>>> ​(tools.js)​ - It will contain all JS modules under tools directory
>>>>>
>>>>>
>>>>> ​2. ​
>>>>> Browser
>>>>> ​(browser.js)​ - It will contain all JS modules under browser directory
>>>>>
>>>>
>>>> By under, do you mean every javascript file recursively under the
>>>> browser directory?
>>>>
>>> ​No, I mean a single bundled javascript ​file of various nodes static JS
>>> files.
>>>
>>>>
>>>>
>>>>> ​4. Common.js and/or vendor.js - Common.js will contains all common js
>>>>> files/libraries shared by modules like backform, backbone, underscore etc.
>>>>>
>>>>
>>>> We're okay with bundling vendorized code for a first pass to webpack
>>>> everything. The goal should be to pull code out of the shared javascript
>>>> bundle as we unvendor dependencies.
>>>>
>>> Yes, bundled vendor.js will have all unvendor dependencies and files
>>> will refer from node_modules.
>>>
>>>> For naming: Common.js is an overloaded term (
>>>> http://requirejs.org/docs/commonjs.html) We prefer vendor.js. Will
>>>> there be non-vendor code put into this bundle?
>>>> ​I think vendor.js should only contain vendor files. We can create
>>>> another file(xyz) that will contains JS files shared across the modules.
>>>>
>>>> ​What do you think ?​
>>>>
>>>>
>>> ​
>>>>
>>>>> Entry points JS: main.js - It will load above modules ascynchronusly
>>>>> or it will be minified version of all.
>>>>> define('', ['common', 'tools', 'browser', 'grid'], function(...) {});
>>>>>
>>>> or these modules can also be loaded using dependency; For example, load
>>>>> grid.js on database node expand.
>>>>>
>>>>
>>>> Could you go into some more detail on this decision? How would main.js
>>>> be used in the app? Is the idea to still use define from require.js in
>>>> javascript being webpacked?
>>>>
>>> ​In pgAdmin4 there is no main.js, but entry point is browser/index.html.​
>>> This is how I think bundled JS will load:
>>>
>>> Load vendor.js using <script></script> in base.html
>>>
>>> *​browser/​index.html*
>>>
>>> require(
>>> ​ ​
>>> ['pgadmin',
>>> ​'sources/generated/common', ​
>>> 'sources/generated/bundled'],
>>> ​ ​
>>> function() {
>>> }
>>> ​)​
>>>
>>> *sources/generated/bundled​.js​*
>>>
>>> ​define
>>> (
>>> ​ ​
>>> ['
>>> ​sources/generated/browser
>>> ', 'sources/generated/
>>> ​tools
>>> '
>>> ​, 'sources/generated/grid'​
>>> ],
>>> ​ ​
>>> function
>>> ​(​
>>> pgBrowser) {
>>> ​ pgBrowser.init();​
>>> }
>>> )​
>>> ​
>>>
>>>
>>> ​or we can use 'import'​ instead of 'require' for files that will be
>>> bundled like 'bundled.js'
>>> I keep require/define call untouched in module JS, and webpack converts
>>> 'define' calls to __webpack_require__(module_id) to load dependent modules.
>>>
>>>>
>>>> ​Currently I am working on converting 'requirejs config shim
>>>>> dependency into webpack' using imports-loader and exports-loader which
>>>>> is taking time and will look for alternative if it doesn't works. Once
>>>>> dependency is properly defined, the task to generate other module JS will
>>>>> become quite easy.
>>>>>
>>>>
>>>> We used imports-loader and exports-loader for tests (see
>>>> web/webpack.test.config.js)
>>>>
>>> ​Hmm. it is useful.​ but I am using shim-loader
>>> <https://github.com/zinserjan/shim-loader> which is compatible with
>>> requirejs shim format.
>>>
>>>>
>>>>
>>>>> I also found, the CSS imported using import
>>>>> 'slickgrid/slick.grid.css'; statement is put into <style></style> tags
>>>>> ​ in html​
>>>>> , instead must be loaded separately as a file so overrides.css can
>>>>> work.
>>>>>
>>>>
>>>> Those three css files from slickgrid need to be brought into the
>>>> application somehow because they won't be available after building the app,
>>>> since SlickGrid has been un-vendored. We opted to webpack the css together
>>>> with the required slickgrid js files. Can the overrides.css files be
>>>> modified to work with this bundle? Which overrides.css file is not
>>>> working now?
>>>>
>>> ​This affects the CSS font styles of edit grid header or might affect
>>> other changes as well. we'll have to look how to override vendor slick grid
>>> css which is now rendered in HTML markup.
>>>
>>>
>>>> Cheers,
>>>> Matt and George
>>>>
>>>
>>>
>>
>
>
> --
> Dave Page
> Blog: http://pgsnake.blogspot.com
> Twitter: @pgsnake
>
> EnterpriseDB UK: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>