How should we manage dependencies?

[wilzbach]

If you haven't done so, please read this excellent article as an intro to the problem of dependency management in web components by @tjvantoll:

http://tjvantoll.com/2014/08/12/the-problem-with-using-html-imports-for-dependency-management/

@tjvantoll discusses the following options - for convenience shortly summarized here:

Option 1: Use a CDN

<script src="//cdnjs.cloudflare.com/ajax/libs/moment.js/2.7.0/moment.min.js"></script>

• only works if exactly the same URL (same CDN provider, same protocol, same version) has been choosen
• prevents script concatention
• prevents offline development

Option 2: Enforcing a folder structure on the end user + relative imports

• The Polymer project uses this strategy
• Could be done with either npm or bower
• Requires strong convention (naming, structure)
• The developer would need to upload two files to /dist (1 dev file with relative imports, 1 minimized with all dependencies)
• Can't deal with version conflicts (e.g. two versions of jquery)

The Polymer project is using this approach - here is the distribution guide for 0.5 written by @addyosmani.

Option 3: Feature detection

Summary: using a tiny module loader to detect whether the script has already been loaded

• two modules can still request the same module with a different URL
• has to be done asynchronously in dev environment
• no/harder build (vulcanize) process

Option 4: Not using HTML imports for external dependencies

Summary: no reference in the component's code - the dependency is only referenced in the documentation

---

This also has been discussed at the Polymer project.

Here is my opinion to start the discussion: Option 2 (enforcing a common folder structure) seems to be the lesser evil.

However for dependency conflicts:

• bower's conflict resolution warns you about a conflict and you can choose which version you want to load, but it doesn't help if two components really depend on two different versions (e.g. Angular 1 and Angular 2)
• npm3 (currently in beta) is awesome at detecting whether there will be a semantic conflict (as long as the author uses correct semantic versioning) - however if there is a conflict it will put the conflicting file in the components node_modules folder (depending on what module was installed first). As an ugly workaround an author could just include both paths (node_modules/<lib>/dist/<lib>.html and ../<lib>/dist/<lib>.html)

I am very much looking forward to your thoughts :)

[herkulano]

HTML Imports are also being contested by Mozilla and they will not implement them. They propose the use of ES6 modules instead.

"Mozilla will not ship an implementation of HTML Imports. We expect that once JavaScript modules — a feature derived from JavaScript libraries written by the developer community — is shipped, the way we look at this problem will have changed."
https://hacks.mozilla.org/2014/12/mozilla-and-web-components/

ECMAScript 6 provides the following ways of importing:

    // Default exports and named exports
    import theDefault, { named1, named2 } from 'src/mylib';
    import theDefault from 'src/mylib';
    import { named1, named2 } from 'src/mylib';

    // Renaming: import named1 as myNamed1
    import { named1 as myNamed1, named2 } from 'src/mylib';

    // Importing the module as an object
    // (with one property per named export)
    import * as mylib from 'src/mylib';

    // Only load the module, don’t import anything
    import 'src/mylib';

ES6 Modules Final - Article

[Hypercubed]

I'm hoping for an ES6 module solution. Currently SystemJS and Web Components are very tangential.

[Hypercubed]

@greenify, If I understand correctly the four options above are regarding html imports. But html imports is not a necessity of web components. From https://hacks.mozilla.org/2015/06/the-state-of-web-components/:

We’ve been working with Web Components in Firefox OS for over a year and have found using existing module syntax (AMD or Common JS) to resolve a dependency tree, registering elements, loaded using a normal <script> tag seems to be enough to get stuff done.

Using SystemJS might be something to look into.

[herkulano]

SystemJS as a module loader

Added a new branch for this test:
https://github.com/herkulano/biojs3-webcomponent-example/tree/dependency-systemjs

Loading module in .js

System.import('../../d3/d3.min.js').then(function() {
  Polymer({
  ...

https://github.com/herkulano/biojs3-webcomponent-example/blob/dependency-systemjs/biojs-component.js#L1

Loading SystemJS in index.html

<script src="../../systemjs/dist/system.js"></script>

https://github.com/herkulano/biojs3-webcomponent-example/blob/dependency-systemjs/demo/index.html#L10

Dependencies listed in bower.json

  "dependencies": {
    "polymer": "Polymer/polymer#^1.0.0",
    "d3": "~3.5.6",
    "systemjs": "https://github.com/systemjs/systemjs.git#0.18.4"
  },

https://github.com/herkulano/biojs3-webcomponent-example/blob/dependency-systemjs/bower.json#L23

It still depends on bower to install all dependencies, i.e., the end-user still has to use bower to install the dependencies.

Any thoughts?

[wilzbach]

I'm hoping for an ES6 module solution. Currently SystemJS and Web Components are very tangential.

I also find the ES6 module specification quite tempting - especially if we have a loader like SystemJS that supports AMD, CommonJS and ES6 modules. However we still would require a common structure to be able to avoid duplications when loading modules?

It still depends on bower to install all dependencies, i.e., the end-user still has to use bower to install the dependencies.

I have created an example that gets all internal dependencies and Polymer from SystemJS.

https://github.com/greenify/polymer-systemjs-loading

Unfortunately it also doesn't solve the problem that we have if someone imports third-party dependencies like System.import('biojs-core'); as SystemJS would need to have a way to figure out where the module is.
So maybe the Polymer approach of using a common structure isn't that bad at all?

[Hypercubed]

@greenify we are working on the same thing.

SystemJS knows where the modules are based on the System.config.js. Using jspm this is automatic.

jspm install d3=github:github:mbostock/[email protected]

then in the module you can:

import d3 from 'd3';

[Hypercubed]

It is additionally complicated if you want to use bundling. SystemJS builder won't bundle into html files.

[Hypercubed]

Here is one way to bootstrap a Web Component using SystemJS: http://plnkr.co/edit/w3L3FB

I used commonjs because I couldn't get traceur working in plunker.

[herkulano]

@Hypercubed That's a really cool example! 👍

It also shows that developers can use whatever flavour they want and still comply with Polymer > Web Components (in the near future).

[Hypercubed]

Thanks. Here is one more demo before I call it a night: http://plnkr.co/edit/JJp6jp

Here I made a dumb systemjs plugin to do the HTML importing. This allows you to use mappings in the system.config.js (and all module loading goodness). I haven't tested it much so I don't know how robust it is.

I can imagine that it could be a lot smarter and process inline css and js for you. I'm surprised it hasn't been done before.

[Hypercubed]

FYI... https://github.com/Hypercubed/systemjs-plugin-html

[herkulano]

@Hypercubed Amazing! 👍

[wilzbach]

@Hypercubed I like your approach - it is more secure than mine ;-)

Here a few remarks that I have by looking through your code:

Injecting html via systemjs-plugin-html

AFAIK using this approach is problematic when you bundle everything, as address won't be available them - that is the advantage of the systemjs-text plugin.

exports.fetch = function(load) {
  return importHref(load.address);
};

Injecting css via relative links

We probably can't bundle bar-char-component.css easily if we just use a relative link - or can we?

<link rel="import" type="css" href="bar-chart-component.css">

using systemjs plugins

I guess that we need to agree on all the custom plugins of systemjs that we want to use as for a development setup as we probably want to avoid requiring a bundled version in other packages - we would have hard time to filter out duplicates.

In summary

The biggest difference between our two approaches is that you (1) dynamically add <link> tags for the html and css on the document, whereas I (2) use innerHTML. Your approach (1) is probably faster and more secure, whereas (2) requiring ressources probably is better for a bundling process. I will have a look at best practices about the bundling process - see you soon :)

For convenience I have put my current version onto plunkr, too: http://plnkr.co/edit/CbZXA1?p=preview

[Hypercubed]

You are correct. Bundling wont work. The css plugin has it's own mechanism for bundling that I did not implement. The inline css and js are NOT processed at all by systemjs. At this point, as shown in my last plunker, I'm using the plugin as a promise generator. But maybe with some work...

[herkulano]

@greenify Also amazing! 👍

[Hypercubed]

Found this... https://github.com/nevir/html-exports/tree/master/dist/sysjs-plugin

Haven't had a chance to look in detail.

[herkulano]

@greenify nice

I've forked @greenify's plunk and added some abstraction as a bio.js library to simplify the module loaders for developers: http://plnkr.co/edit/mEKZ2r?p=preview

Some problems of this solution:

• it's starting to get convoluted
• how does the end-user install local dependencies of a component?
• loss of the css scope by Polymer. @Hypercubed's version solves this and the css as a link tag inside the element doesn't seem to be a problem and it would also be easier to package for distribution.
• do we lose the template goodness of Polymer templates [needs testing]
• do we lose any Polymer goodness? [needs testing]

Are we planning to have several versions: ES5, ES6, HTML?
Should we have a recommended way and then add flavors for other developers?

UPDATE: I've merged your ideas in this plunker http://plnkr.co/edit/g54ESw?p=preview

[Hypercubed]

For me, as a potential BioJS3 module consumer, I am happy using jspm/SystemJS/ES6. I think, however, npm/Browserify/CommonJS has more traction currently and may be more familiar to BioJS2 users and developers. The Web Components/Polymer way seams to be to forget modules, throw everything into bower and reference via relative path. That approach of course limits what modules you can consume. Would it even allow you to import an existing BioJS2 io module from npm?

That said, I think it is more than possible to create a BioJS3 template that provides all three. Write your code in ES6 and html, generate an ES5/UMD module, generate a HTML Import file that uses relative paths. Provide a package.json file for npm that points to the UMD/CommonJS module, include jspm additions that point to the ES6 module, and a bower.json file that points to the HTML import file.

I think that would be amazing.

[Hypercubed]

@herkulano I don't think we want to be using the System.import directly, this wont work for bundling.

I took your plunker and added and es6 module, a generated (by hand) es5 commonjs file, and a pure web component html import file. Only the ES5 module is being used but should show a way we can provide all three. http://plnkr.co/edit/gwnVq9

[Hypercubed]

Relevant discussion: http://www.codequora.com/questions/25005623/web-components-polymer-and-systemjs

Wondering if we need/want Polymer/HTML imports at all.

[wilzbach]

It is absolutely great to work with you on this :)

For me, as a potential BioJS3 module consumer, I am happy using jspm/SystemJS/ES6. I think, however, npm/Browserify/CommonJS has more traction currently and may be more familiar to BioJS2 users and developers.

Have you seen this great talk by @guybredford (the author of jspm) on how jspm can be used to manage {es6,es5} dependencies from {github,npm} - he also showcases a cool demo with web components!

how does the end-user install local dependencies of a component?

I do imagine two ways:

1. script-tags for the end-user: bundled (optionally minimized) version in /dist for independent inclusion
2. bower/npm components for a developer: unfortunately we can't expect everyone to use es6 and neither to use SystemJS as @Hypercubed pointed out

That said, I think it is more than possible to create a BioJS3 template that provides all three. Write your code in ES6 and html, generate an ES5/UMD module, generate a HTML Import file that uses relative paths. Provide a package.json file for npm that points to the UMD/CommonJS module, include jspm additions that point to the ES6 module, and a bower.json file that points to the HTML import file.

Wow 👍 - I am only a bit afraid that

• this might create redundancies as e.g. a developer would need to (a) update his dependencies for bower and npm and he might forget about one of the config files, (b) it would start to get confusing if the bower and npm packages don't share the same name - AFAIK only jspm provides a convenient way to map package names.
• how a dev will be able to include dependencies if he uses the html import way. One requires a relative path name to an html file and the other just a require with the package name

loss of the css scope by Polymer. @Hypercubed's version solves this and the css as a link tag inside the element doesn't seem to be a problem and it would also be easier to package for distribution.
[...] moving the add html outside of the javascript file

(: , but if we decide to go for html imports we would need to find a way to run vulcanize on top of the systemjs bundle. That is why I personally would prefer if the JavaScript version would require the html and css -I need to find a better way than innerHTML tough ;-)

Wondering if we need/want HTML imports at all.

👍 - wondering about the same!

[Hypercubed]

Yeah, keeping the version numbers in sync between bower.json and package.json is already a hassle. I imagine someone could write a tool that maps jspm package names down to bower relative paths. Maybe some fancy gulp scripts? I think I saw someone made a bower -> jspm upconverter... I'll have to find it again. I would guess jspm -> bower -> html imports would be easier than the reverse. Not sure if it is worth all the tooling. Vulcanizing a SystemJS bundle sounds better. I wonder what @guybedford thinks of all this.

[herkulano]

@Hypercubed looks great!

[herkulano]

Some findings on using JSPM with Polymer

I've added another branch, wanted to try jspm, so couldn't use plunker for this.
https://github.com/herkulano/biojs3-webcomponent-example/tree/jspm-test

Polymer Elements
It works well as long as you don't use polymer elements.
When you want to add a polymer element, for instance, a simple paper-button element, because of the structure they have for elements and reliance on bower, you can't use them with jspm anymore.

From paper-button.html

<link rel="import" href="../polymer/polymer.html">
<link rel="import" href="../paper-material/paper-material.html">
<link rel="import" href="../paper-ripple/paper-ripple.html">
<link rel="import" href="../paper-behaviors/paper-button-behavior.html">

Bower
I managed to use bower with jspm with a jspm registry endpoint for bower

Some problems:

• When installing the paper-button the sub-dependencies were not resolved. Didn't spend much time with it though.
• Polymer element's HTML Imports structure rely on bower's flat structure which is very different from jspm's structure, so it breaks polymer elements.

[wilzbach]

Hey,

I used our work @herkulano & @Hypercubed to create a jspm example app that uses the following web components as dependencies:

• bio-element.
• bio-container

In particular bio-container depends on bio-element, other javascript dependencies like Polymer or lodash (as example) are also injected via the SystemJS loader. For simplicity I avoided ES6 code, but feel free to add it. Also the web component polyfill is now conditionally loaded (needs still some work)

Currently

• HTML is loaded via @Hypercubed html plugin
• CSS is loaded via an HTML Import inside the HTML

I guess we only

• have to figure out how we can bundle the app when using the HTML/CSS imports - maybe a combination with vulcanize?
• whether we actually want to (or can) support existing Polymer plugins
• one can depend on BioJS web components without jspm and SystemJS or whether that is important to us (in any case they will always be able to include the bundled version)

Feel free to modify the bio-element, bio-container and the example app.

Happy hacking!

[Hypercubed]

Perhaps, but I don't see the API in there yet. What I mean is BioJS web components can use a component constructor that looks like Polymer:

  <script type="module">
    BioJSComponent({
      is: "dom-element",
      properties: {
        greeting: {
          type: String,
          value: "I\'m a DOM element. This is my local DOM!"
        }
      },
      created: function() {
        this.createdRan = true;
      }
    });
  </script>

[Hypercubed]

If it is exactly the same... might as well use polymer. i.e. # 3

[herkulano]

I don't know @greenify's plans on this.

I agree if the library is the same there's no point in duplicating work.

Although, i still have the concerns i mentioned before with using Polymer JS, unless the Polymer team embraces the idea of Polymer JS, which from @addyosmani's comments it's highly unlikely at least short-term.

[Hypercubed]

Polymer JS, if I understand what you mean, could be as simple as:

jspm install polymer=github:Polymer/polymer

and

require('polymer/polymer.html!html');
module.exports = Polymer;

[herkulano]

AFAIK, it doesn't work like that, because of Polymer's HTML Imports. That's why @greenify had to do a Polymer JS, extracting all of the html and importing polymer-mini.html and polymer-micro.html into one .js file.

It would be good if you could test it so that we are sure of what we are saying. I'm not, so don't know if it works or not. 😄

[wilzbach]

Extracting the Polymer HTML to Javascript is already automated and do think that if we ask nicely they (Polymer) will accept a pull request because it is nothing they have to maintain.

My opinion is that Angular 2 will have support for Web components. It will definitely come in other frameworks like React too or there will be new frameworks. So I don't think we should build our own polymer. We probably lack the resources to maintain it. However if a developer doesn't want to include the 150kb for Polymer he shouldn't need to.

Thinking especially about the parsers it could make sense if we maintain our own lightweight library here because simply they don't need all the fancy features from polymer and all parsers are supposed to inherit from a common parent that we have to and want to maintain anyways.

On July 13, 2015 2:02:12 PM GMT+02:00, Herculano Campos [email protected] wrote:

I don't know @greenify's plans on this.

I agree if the library is the same there's no point in duplicating
work.

Although, i still have the concerns i mentioned before with using
Polymer JS, unless the Polymer team embraces the idea of Polymer JS,
which from @addyosmani's comments it's highly unlikely at least
short-term.

---

Reply to this email directly or view it on GitHub:
#4 (comment)

[herkulano]

@greenify

Extracting the Polymer HTML to Javascript is already automated and do think that if we ask nicely they (Polymer) will accept a pull request because it is nothing they have to maintain.

So you still have to update it and push it when they make changes or that is also automated?

My opinion is that Angular 2 will have support for Web components. It will definitely come in other frameworks like React too or there will be new frameworks. So I don't think we should build our own polymer. We probably lack the resources to maintain it. However if a developer doesn't want to include the 150kb for Polymer he shouldn't need to.

Support for web components is not being questioned. Polymer is, which is not the same.

That means you vote for 3, right?

If we're going with 3 we should think about:

• Documentation for developers that using Polymer JS != Polymer HTML
• Documentation and example of using Polymer components with BioJS components
• Fully automated process for Polymer JS [you might already have this]
• Build for production. Fully using jspm build, for simplicity or other build tools? [we already have the jspm build working in the systemjs-plugin-html]

[wilzbach]

On 2015-07-13 14:50, Herculano Campos wrote:

@greenify https://github.com/greenify

Extracting the Polymer HTML to Javascript is already automated and do think that if we ask nicely they (Polymer) will accept a pull request because it is nothing they have to maintain.

So you still have to update it and push it when they make changes or that is also automated?

Nope it is not triggered automatically, well we could add that.
However I guess asking Polymer is the better way.

My opinion is that Angular 2 will have support for Web components. It will definitely come in other frameworks like React too or there will be new frameworks. So I don't think we should build our own polymer. We probably lack the resources to maintain it. However if a developer doesn't want to include the 150kb for Polymer he shouldn't need to.

Support for web components is not being questioned. Polymer is, which is not the same.

That means you vote for 3, right?

Oh sorry - I was writing the message on my phone. I would prefer if we focus on the basic, Vanilla (#2) approach and find solutions based on this.

If we're going with 3 we should think about:

• Documentation for developers that using Polymer JS != Polymer HTML
• Documentation and example of using Polymer components with BioJS components
• Fully automated process for Polymer JS [you might already have this]
• Build for production. Fully using jspm build, for simplicity or other build tools? [we already have the jspm build working in the systemjs-plugin-html]

Also for approach II we have to solve some of those issue. E.g. we have to find our a documentation format/parser (see #9).

[herkulano]

Also for approach II we have to solve some of those issue. E.g. we have to find our a documentation format/parser (see #9).

@greenify agree, automated documentation is a must 👍 although it will be an issuewith any solution we go for.

[Hypercubed]

@herkulano Using systemjs-plugin-html you can import polymer as a global. The HTML imports are all within the polymer project so they are resolved by the browser. That is what I am doing in my app (I can share more tomorrow). The trouble occurs if the developer includes a polymer component which HTML imports a version of polymer using a external relative path. Bundling we still need to figure out.

[Hypercubed]

BTW, I am also testing this method of including scripts in HTML imports (ModuleLoader/es-module-loader#95). If it works (so far looks good) then we can import HTML files that contain inline ES6 (and dependencies).

[herkulano]

@Hypercubed Great work on systemjs-plugin-html, it's looking good 👍

[herkulano]

@Hypercubed let me know if you need / want any help with the bundling.

[Hypercubed]

Hello guys, I've added unit tests and updated the html plugin to allow inline script modules. Rather than filling up this issue I created an issue here: Hypercubed/systemjs-plugin-html#2 .

[herkulano]

@Hypercubed 👍

[Hypercubed]

Here is a way to bridge SystemJS and polymer elements: https://github.com/Hypercubed/systemjs-plugin-html/tree/master/test/polymer

[herkulano]

@Hypercubed great work. didn't have time to check it out, will do it asap.

[Hypercubed]

Since it looks like we are leaning towards 2 or 3 with JSPM build. Perhaps we need some discussion of what this actually means. What is the expected output of a JSPM build step run on a biojs component? What about a JSPM build on a app that includes a biojs component?

[Hypercubed]

Here is an example polymer component that uses SystemJS for dependencies (using the SystemJS html plugin). This also demonstrates jspm bundling using the vulcanize branch of the plugin.

[herkulano]

@Hypercubed that's really great! 👍

I only have one concern, which might not be an issue for some people. Correct me if I'm wrong, but it's still difficult to use Polymer Elements with jspm, because of the HTML Import links. For example, paper-button.html looks like this:

<link rel="import" href="../polymer/polymer.html">
<link rel="import" href="../paper-material/paper-material.html">
<link rel="import" href="../paper-ripple/paper-ripple.html">
<link rel="import" href="../paper-behaviors/paper-button-behavior.html">

Because of the way bower's folder structure works they can get away with this link: ../

We could change it at runtime and build time to point to the right folder, similar to what polyserve does.

Any thoughts?

[Hypercubed]

The setup I've shown uses polymer as a helper, but doesn't need to, it can work on vanilla elements. To use polymer elements you can do what I showed here: https://github.com/Hypercubed/systemjs-plugin-html/tree/master/test/polymer . Basically, create a ../polymer/polymer.html file that imports the jspm installed version of polymer. It should bundle using vulcanize/system-plugin-html. Unfortunately you need to manually install the elements.

[Hypercubed]

The biggest problem I am seeing is that if you incorporate two copies of the same HTML import with different urls (including polmer.html its self) HTML imports/vulcanize can't deduplicate, resulting in a runtime error.

[herkulano]

@Hypercubed 👍

[doomedramen]

I am 100% behind option 2

[frankandrobot]

Great work guys. But what is the final result? (I counted about half-dozen outgoing links + sample repos).
In particular, I'm interested in Option 3. Polymer JS + SystemJS + JSPM Build
Mainly what I'm interested in is Polymer JS + SystemJS.
If I have to use vulcanize to bundle, that's fine.

What are my options? Is there a sample repo?

Update:
using hypercube's HTML plugin, I was able to create a polymer component that is JSPM installable in an app. (I apologize for not having any code to share since I'm behind an enterprise proxy.)
However, I seem to be unable to JSPM install existing polymer components. Ex: iron-collapse.
Are we supposed to use the JSPM bower endpoint for these?

[davidmaxwaterman]

I'd also like to know the final result...

[Hypercubed]

I can't speak for BioJS but for me, there was too much flux and uncertainty around WC.