I have a library (ES6) which is composed on many sub-modules, say
- package.json
- lib
- my_package
- file1.js
- file2.js
- sub_module1
- file3.js
- file4.js
I currently do imports like this (all inside my package - using file resolution to find, not node_modules resolution):
import {func1} from 'lib/my_package/file1'
import {func3} 'lib/my_package/sub_module1/file3'
So, in practice I have many files across sub-directories.
I am now trying to package and publish my library, which will be installed under node_modules.
It seems to me that the node resolution algorithm (when behind node_modules) only allows for a single entry point (and there is nothing rollup can do about that)
I would like to be able to include many sub directories and files and for them to be resolved individually.
As far as I understand I have to include a single toplevel file that has all the export from machinery. I can only import that single top level file.
This means having to manually create that file. It also means losing the all the sub-module name structuring that comes from the directory structure.
I was wondering: is there any way one can import any other file from a node_module directly?
Node's resolution algorithm only resolves the first part of a module source, so if someone does this...
var foo = require('your-library/subdir/foo.js');
...then Node (or Browserify/Webpack/rollup-plugin-node-resolve) will correctly resolve that to
/path/to/project/node_modules/your-library/subdir/foo.js
The tricky part is that you want to author JavaScript modules, but if someone is using your library in Node then they need CommonJS modules. Rollup can't help you here — you would need to do a one-to-one ESM->CJS conversion for each module. So that means either using Babel, or authoring CommonJS modules in the first place.
Related
In an Ember app, when using certain dependencies like moment installed via bower, we have to also import the same in the ember-cli-build.js file:
app.import('bower_components/moment/moment.js');
My question is why is that needed, since I would assume everything inside node_modules as well as bower_components should be available for use inside the app.
Also if that is not the case, how do we identify which dependencies would require such explicit import to be able to use them ?
You don't have to, actually.
There is a package now that lets you 'just import' things: https://github.com/ef4/ember-auto-import
Some reading on the topic of importing: https://discuss.emberjs.com/t/readers-questions-how-far-are-we-from-being-able-to-just-use-any-npm-package-via-the-import-statement/14462?u=nullvoxpopuli
In in-depth answer to your question and the reasons behind why things are the way they are is posted here:
https://discuss.emberjs.com/t/readers-questions-why-does-ember-use-broccoli-and-how-is-it-different-from-webpack-rollup-parcel/15384?u=nullvoxpopuli
(A bit too long for stack overflow, also on mobile, and I wouldn't want to lose all the links and references in a copy-paste)
Hope this helps
Edit:
To answer:
I just wanted to understand "in what cases" do we need to use the import statement in our ember-cli-build (meaning we do not do import for all the dependencies we have in our package/bower.json)...But only for specific ones...I wanted to know what is the criteria or use case for doing import.
Generally, for every package, hence the appeal of the auto-import and / or packagers (where webpack may be used instead of rollup in the future).
Though, it's common for ember-addons to define their own app.import so that you don't need to configure it, like any of these shims, specifically, here is how the c3 charting library is shimmed: https://github.com/mike-north/ember-c3-shim/blob/master/index.js#L7
Importing everything 'manually' like this is a bit of a nuisance, but it is, in part, due to the fact that js packages do not have a consistent distribution format. There is umd, amd, cjs, es6, etc.
with the rollup and broccoli combo, we need to manually specify which format a file is. There are some big advantages to the rollup + broccoli approach, which can be demonstrated here
and here
Sometimes, depending on the transform, you'll need a "vendor-shim".
These are handy when a module has decided it wants to be available on the window / global object instead of available as a module export.
Link: https://simplabs.com/blog/2017/02/13/npm-libs-in-ember-cli.html
(self represents window/global)
however, webpack has already done the work of figuring out how to detect what format a js file is in, and abstracts all of that away from you. webpack is what ember-auto-import uses, and is what allows you to simply
import { stuff} from 'package-name';. The downside to webpack is that you can't pipeline your transforms (which most people may not need, but it can be handy if you're doing Typescript -> Babel -> es5).
Actually: (almost) everything!
Ember does, by default, not add anything to your app except ember addons. There are however some addons that dynamically add stuff to your app like ember-browserify or ember-auto-import.
Also just because you do app.import this does not mean you can use the code with import ... from 'my-package'. The one thing app.import does is it adds the specified file to your vendor.js file. Noting else.
How you use this dependency depends completely on the provided JS file! Ember uses loader.js, an AMD module loader. So if the JS file you app.imported uses AMD (or UMD) this will work and you can import Foo from 'my-package'. (Because this is actually transpiled to AMD imports)
If the provided JS file provides a global you can just use the global.
However there is also the concept of vendor-shims.. Thats basically just a tiny AMD module you can write to export the global as AMD module.
However there are a lot of ember addons that add stuff to your app. For example things like ember-cli-moment-shim just exist to automagically add a dependency to your project. However how it's done completely depends on the addon.
So the rule is:
If its an ember addon read the addon docs but usually you shouldn't app.import
In every other case you manually need to use the library either by app.import or manual broccoli transforms.
The only exception is if you use an addon that tries to generically add dependencies to your project like ember-browserify or ember-auto-import
Currently my process for building is:
Write lots of typescript files with ES6 module syntax
Generate an index.ts which re-exports all modules from one point
Compile to CommonJS + System
Output Descriptor/Typing files
This results in an index.js file which re-exports all the internal files without the developer consuming it needing to know about it, as well as a lot of d.ts files which mirror the file structure.
Now this works, but if I were to take this approach to the browser I would need to webpack up all the js or it would be a http request nightmare pulling in all the individual files. Currently this library would be consumed as a dependency for other libraries, so it is not an end point for logic or anything it is a module/library.
Now the main question is with webpack I know I can load TS in and get a commonJS module out, however I cannot find any way to generate d.ts files with webpack. So is there a way for me to use webpack as the compiler/packager in this scenario and have an output my-lib.js and my-lib.d.ts rather than the current approach which yields lots of individual files.
== Extra Clarification on Use Case ==
Just to try and make sure everyone is on the same page here when I say it is a library that would be re-used what I mean is that this is something that would be loaded via npm or jspm or something as a module dependency for other modules.
So for example let us pretend jquery doesn't exist and I am going to create it but write it in typescript for other developers to consume in both JS and TS. Now typescript outputs both js files and d.ts files, the js files are to be used as you would expect, but the d.ts files explain to other typescript files what the types contained within the library are.
So assuming I have developed jquery in TS as listed above, I would then want to publish this output (be it created by webpack or tsc) on npm/jspm/bower etc. So then others can re-use this library in their own projects.
So webpack typically is used to package an "application" if you will, which contains logic and business concerns and is consumed directly as an entry point to a larger set of concerns. In this example it would be used as a compilation and packaging step for a library and would be consumed via var myLib = require("my-lib"); or similar.
Generating the .d.ts files is not related to webpack. With webpack you can use either ts-loader or awesome-typescript-loader. Both of them make use of tsconfig.json. What you need to do is to add declaration: true in your tsconfig.json.
I'd also suggest you to take a look at typescript-library-starter. You'll find how's set up there, including UMD bundle and type definitions :).
With the use of transpilers it is already possible to use ES6 modules. One of the easiest ways is using Browserify and Babelify.
The problem I'm having is how to handle dependency management.
In the old days you'd just have some Bower dependencies. The build would bundle non-CDN to vendor.js and project specific files to foobar.js (or whatever).
So then you'd be able to use the resulting code in a different project by simply bower install foobar --save.
If both foobar and your new project had a common dependency it would be easily resolved with Bowers flat dependency.
Now in come ES6 modules:
Say I have a project foo using lodash. The directory structure is as follows:
src/js/foo.js
src/vendor/lodash/dist/lodash.min.js
And foo.js starts by declaring:
import * as _ from '../../vendor/lodash/dist/lodash.min.js';
or (as Browserify wants since Babelify transpiles to CommonJS):
import * as _ from './../../vendor/lodash/dist/lodash.min.js';
If I now round up and publish my foo project and start a new project bar that uses foo this will be my directory structure.
src/js/bar.js
src/vendor/foo/dist/foo.js
src/vendor/lodash/dist/lodash.min.js
But that would not work since the path from foo to lodash is now broken (if I understand Browserify correctly the dot-slash in './blaat/file.js' is relative to the file it's being called from).
Is some way to import without making any file path assumptions?
Isn't there some way to indicate multiple source roots? (ie in the above case src/js and src/vendor ... well, ideally you'd just want to state import * as _ from 'lodash';)
I've only used Browserify with Babelify on cli. Should I be using some other transpiler?
I think that jspm is the solution your looking for. It will help you out without making file path assumptions when importing modules. It uses the SystemJS dynamic ES6 loader. Watch the video that is posted on their site for a very good explanation on how it all works, Guy Bedford: Package Management for ES6 Modules [JSConf2014].
So I know how to require and export modules in ES6. But for frameworks like Aurelia, the docs say that you require aurelia like so:
import {LogManager} from 'aurelia-framework';
Do I have to place a JS file named aurelia-framework in the folder where the JS file I'm executing it from resides, or does the import function work similiar to the require function in NodeJS/CommonJS?
According to this article ES6 modules spec only deals with loading modules that are present in the file path. Downloading these files (via NPM or by other means) is outside of scope of ECMAScript 6 modules spec. Nothing is said in the spec about supporting npm package includes (traversing the directory structure down to the /, one directory at a time, looking for a package.json file and then searching within the node_modules directory where package.json file is found). So while the import syntax is similar to commonJS style, the whole magic of looking for modules in the node_modules directory is not included.
So for your example to work, aurelia-framework must be a javascript file somewhere in your file system and it should contain an exports statement.
import {LogManager} from 'aurelia-framework'; // ./aurelia-framework.js
import {LogManager} from '../libs/aurelia-framework'; // ../libs/aurelia-framework.js
with Aurelia, you can install dependent libraries using jspm. you can see an example of that here. jspm will get the packages for you and bring them into subfolders in your project. jspm uses an index (stored in config.js) to know where to locate the files (similar to how requirejs, but works for amd, commonjs, and es6 modules).
there is also an example of using the aurelia libraries with requirejs amd loader. this example uses a bundle of aurelia libraries generated by r.js as shown here
If I have a node.js application that has hundreds of files that reference a module (say underscore) and I want to replace that module with another (say lodash) then the obvious way to do this substitution would be a global name replace and switch out the modules in the package.json file.
Is there anyway to just change the module that a name refers to so that when node.js sees require('moduleA') it actually loads 'moduleB' instead? Now I know that this would cause naming hell because anyone working on the project would see require('moduleA') and wouldn't know that the real module being loaded was 'moduleB' so ultimately you'd probably want to go with the first solution. The use case that I'm thinking of is if you want to try a few alternatives for API compatible modules to measure your application's performance (for example) with each module.
If this is an on-going thing and you want to maintain the ability to programmatically switch between the options often, such as in tests:
Instead of using require("underscore"); throughout your codebase, require a local file instead like require("./lib/underscore");, and have that file conditionally re-export underscore or a different library:
if (global.USE_LODASH) {
module.exports = require("lodash");
} else {
module.exports = require("underscore");
}
If this is a one-off thing to try out an alternative library before making the decision to switch, and you want to do this test quickly first without find-and-replacing in all of your files:
Go inside your node_modules folder, delete or rename the underscore folder, and make a symlink named underscore to the replacement module's folder. I don't recommend this as a long-term solution: running npm install again will likely undo this hack, and most projects choose to avoid checking the node_modules folder into their source repository.
Try to use mock-require module.