NPM modules in Grunt based projects - javascript

Node has a simple module loading system which uses require() method call to load modules from different locations in the root folder.
E.g.
var qr = require('qr-image');
I am trying to do something similar in grunt but i am unsuccessful with that.
I had added this module to package.json file in the following fashion and then ran npm install at root directory of the project.
"devDependencies": {
.
.
.
"qr-image": "^2.0.0"
},
Now whenever I use require I get the following error on console and my code breaks.
ReferenceError: require is not defined
Please suggest as how to use the npm module in Grunt based project, Thanks.

The require function isn't available in web browsers. Instead it's part of nodejs, which is a server-side language (e.g., something you might run directly from your computer terminal, not in a browser) and used to load dependencies in that language.
In a web browser, I usually just include my dependencies as additional scripts on the page, e.g.,:
<script src="path/to/my/dependency.js"></script>
<script src="path/to/my/code.js"></script>
Some other options are RequireJS or what's listed in this question or as more of a general purpose dependency manager for front-end code: Bower.
Looking closer at your question, it's likely that the "qr-image" npm dependency won't work in client-side code for you (since it was built to run via node in server-side code).
A quick search for QR code client-side code brought up this SO post, which points to the QRCode.js project for client-side QR code generation—I haven't used it, but it looks like a step in the right direction for what you're working on.

Related

Allow node module relying on uuid() to execute in client and server

I have a node module that I am building, and I want it to be able to execute on the server (in nextjs server side rendering) and the client (call additional lifecycle methods in the UI). This same module also needs to work when used purely as a js library that can be included in a <script> tag on the page. This module depends on the uuid module, which has logic in it to check if it is running in a browser or server context, and use the proper random number generators/crypto libraries that are available in that context.
If I don't specify a target in my webpack config, the bundle works great in a client browser. It includes the webpack browser logic just fine. But it doesn't work in the server case - webpack removed the server capable logic in the uuid module.
If I target: 'node' in my webpack config - it executes just fine as a node module on the server and the client. It seemingly included all of the logic this time. But now it doesn't work if included just as a script tag on the page. I get ReferenceError: require is not defined from the file that depends on the uuid module.
It seems like the uuid module should handle these different environments just fine, but webpack is messing with that. How can I let that module resolve the proper implementation at runtime?
I unfortunately do not have a minimally reproducible example, or additional code to share at this time. I figured someone might have run into this with webpack (or even webpack and the uuid module) and know the solution.
I was trying to do this by building a single version of the package, but I don't think that is possible.
What is possible is building multiple versions, and then hosting the web bundle via unpkg or jsdelivr via an entry in package.json. Those entries can point to the target: 'web' version of the package, while the npm package can point to the target: 'node' version.

NPM Jquery Client vs Server Side

I am fairly new to node and npm so this is a theoretical question.
I want so start using Jquery on my websites, so I did npm install jquery and this created a node_modules directory inside my webpage directory along with my html, js and css files.
In order to make it work I had to add the following in my head tag in html <script src="node_modules/jquery/dist/jquery.js"></script>
Question 1) Why npm installed node_modules directly into my directory? I would assume it would install it in my global directory usr/local/lib/node_modules
Question 2) If I would host the website somewhere else this would not work correct? I assume this method only works on client side.
Question 3) How would I have to setup my html or js in order to import jquery?
Apologies if this is a stupid question, but I'm trying to understand the mechanics of what I program.
Why npm installed node_modules directly into my directory? I would assume it would install it in my global directory usr/local/lib/node_modules
npm is designed to manage dependencies for a project. While you can make it install modules globally, this is really only intended for whole applications that are distributed via NPM.
If I would host the website somewhere else this would not work correct? I assume this method only works on client side.
The src attribute needs to be a URL that resolves to the script file. If you don't upload the script to the right place then it won't resolve.
How would I have to setup my html or js in order to import jquery?
With a <script> element.
Note that npm is not very good at managing dependencies for a client-side JavaScript project by itself. It only becomes useful when combined with a module system and a bundler such as browserify.

How to use external JS library methods in my Node.JS project if this library does not exist in NPM

I have a problem which can be easily solved by importing an external JS library into Node.js. However, this library does not exist in NPM.
I found out an old solution on StackOverflow which seems to fix the problem. However, it looks wierd.
Is there a more convenient solution in 2k20 to use external JS library methods into my Node.js code?
If your library have a package.json: You can install the package directly from the git repository, for example npm install https://github.com/vendor-creator/vendor-package. NOTE that for this method to work, in cases where the module has to be built, the git repository should contain a dist/ folder containing the built code or have in its package.json, a prepare step responsible for building the package upon installation.
If your library does not have a package.json and is simply a vanilla JavaScript file like the Lodash JavaScript file, I would advise just like in the post you linked, to create a vendor.js file (.min if the script is minified), copy and paste the content of the file and require it. Be aware that some libraries using CDN and not NPM, are designed for browser environment and may lack CommonJS support preventing you from using require. In that case you'll have to modify the library source code.
If it's a small library, there is no need to create an advanced build system. If the library is stable, just copy and paste it and you'll be fine. When in doubt always follow the K.I.S.S principle.

Package an npm module so that it can work in the frontend (eg. with create-react-app) `You may need an appropriate loader to handle this file type`

I have written a node module and published it as a node package. It works when I use it in backend applications (pure nodejs, no babel or transpile).
However, when I use this same npm module in the frontend (in my case a 'create-react-app') application, it breaks. Bellow is the exact error:
Module parse failed: Unexpected token (14:2)
You may need an appropriate loader to handle this file type.
The error is referring to my use of the spread operator (...). I would prefer not to have to rewrite the library, and would rather add some kind of transpiler to package my library. I haven't found a clear solution to this, they are all very convoluted.
I have tried using rollupjs, and https://github.com/insin/nwb. Neither sadly seem to be what I'm after.
Run my code:
You can install the library to your create react app using npm i cbs-proxy-client#0.0.3. And then add the line const cbsProxyClient = require('cbs-proxy-client') or import cbsProxyClient from 'cbs-proxy-client' to any of your scripts.
Advice would be appreciated :)
A library used for frontend is expected to package an already transpiled version of the source javascript. To do this, you might want to use rollup as a build process in your library to create a bundle file. You can use babel to transpile your code for desired browser support. Let's say the bundle file is saved in dist/bundle.js. Now you will modify the package.json to load this bundled file as the entry file using main parameter in package.json
If you are building using rollup or webpack, it is easy to miss that the bundled file should be prepared as a library. This means that importing the file using commonjs should be able to export correct variables. A typical webpack build removes such exports because it is supposed to work straight on a browser. This blog is in my bookmarks titled "js library steps" since I was creating my first js library.
Note that you do not need to put your generated file in version control. You can use npm files property in package.json to package your bundled files while ignoring them in git.

How can I manage client-side JavaScript dependencies? [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 6 years ago.
Improve this question
Although there are great solutions to manage dependencies on the server side, I could not find any that satisfies all my needs to have a coherent client side JavaScript dependency management workflow. I want to satisfy these five requirements:
Manage my client-side dependencies in a format similar to npm's package.json or Bower's bower.json
It should have the flexibility to point to a Git repository or actual JavaScript files (either on web or locally) in my dependency.json file for lesser known libraries (npm let you point to Git repositories)
It should minify and namespace all libraries into a single file like Ender - that's the only JavaScript file I would need to put in my <script> tag in the client side
It should have out of box support for CoffeeScript like BoxJS4 (now dead)
In the browser, I should be able to use either require style:
var $ = require('jquery');
var _ = require('underscore');
Or better yet, do headjs style:
head.js(['jquery', 'underscore', 'mylib'], function($, _, mylib) {
// Executed when all libraries are loaded
});
If no one such single tool exists, what is the best combination of tools i.e. a tool-chain that I can combine using something like Volo (or Grunt)?
I have already researched all the tools I have linked to in here and they satisfy only up to three of my requirements at best individually.
So, please don't post again about these tools. I would only accept an answer that provides a single tool that satisfies all five of my requirements or if someone posts a concrete workflow/script/working example of a toolchain of multiple such tools that also satisfies all my requirements.
RequireJS does everything you need.
My answer to this question may help you.
Example:
Client app project hierarchy:
sampleapp
|___ main.js
|___ cs.js
|___ require.js
main.js is where you initialize your client application and configure RequireJS:
require.config({
baseUrl: "/sampleapp",
paths: {
jquery: "libs/jquery", // Local
underscore: "http://underscorejs.org/underscore-min.js", // Remote
backbone: "https://github.com/documentcloud/backbone/blob/master/backbone-min.js" // Remote on GitHub
},
shim: {
backbone: {
deps: ["underscore", "jquery"] // Backbone depends on jQuery and Underscore.js
}
}
});
require(["cs!someCoffeescriptFile", "jquery", "backbone", "underscore"], function (SomeCoffeescriptFile, $, Backbone, _) {
// Dependencies are loaded...
// Execute code
});
Dependencies will use the cs plugin when prepended by "cs!". The cs plugin compiles the CoffeeScript file.
When you go in production, you can precompile your whole project with r.js.
node ./node_modules/requirejs/bin/r.js -o buildclientconfig.js
Here are your requirements:
Manage my client side dependencies in a format similar to npm's
package.json or Bower's component.json file. Different but as good!
I should have the flexibility to point to a Git repository or the actual JavaScript files (either on web or locally) in my dependency.json file for lesser-known libraries (npm lets you point to Git repositories). Yes
It should minify and namespace all libraries into a single file like Ender. That's the only JavaScript file I would need to put in my script-tag on the client side. Yes with r.js.
It should have out of box support for CoffeeScript, like Box. Yes
In the browser I can use either require style or headjs. Yes
RequireJS is the one you are looking for, I believe.
As Guillaume86, I think Hem will get you the closest to where you want to be.
In Hem, dependencies are managed using a combination of npm and Hem. Use npm
to explicitly install all of your projects external dependencies. Use Hem to specify which dependencies (both external and local) should be stitched together for you client side operations.
I created a skeleton project of this, so you can see how this would work. You can see it at Client-side Hem.
Adding dependencies
Use npm to search for a specific dependency and then modify the package.json file to ensure that the dependency is tracked in the future. Then specify the
dependency for your application in slug.json.
For example, suppose you wanted to add the CoffeeScript dependency. Just use npm to install the dependency and save it to your package.json file:
npm --save install coffee-script
Manually edit the slug.json file. Add "coffee-script" to "dependencies".
Suppose you wanted to include your own module 'bloomfilters' and it wasn't in the npm registry. You could add it to your project in the following way:
npm --save install https://github.com/dsummersl/bloomfilters/tarball/master
Manually edit the slug.json file. Add "bloomfilters" to "dependencies".
Local modules
If you want to include your own CoffeeScript or JavaScript code, you can do so by adding those files to the app/ folder. Note that in order to expose your script via the 'require' method you must make it a CommonJS module. It is very simple—see the Hem documentation.
Local files
If you want to include non-CommonJS non 'require' code, you can also stitch that by referencing your custom JavaScript or CoffeeScript code via the 'libs' list in
file slug.json.
CSS
Hem will stitch together your CSS too, if you want. See the Hem documentation.
Building
Once you have your dependencies listed, you can use hem to stitch them all together.
# Make sure all dependencies are present:
npm install .
# Make public/application.js
hem build
# See your minified js in public/application.js
Notes
Hem was meant for the Spine.js project - but you don't have to use it for that. Ignore any documentation mentioning spine as you wish...
There is also Browserify.
supports the package.json format
uses npm underneath which can use a GitHub (or any Git) repository as a package source
minifies and concatenates all dependencies into a single file.
supports CoffeeScript if you include it in your dependencies
require style all the way.
supports source maps
I'm pretty sure Hem meets your requirements (I use a personal fork with additional compilers, Jade and Stylus. It's easy to customize to your needs). It uses npm to manage dependencies.
You might want to take a look at Yeoman, which uses several techniques to help you with your requirements.
Our workflow is comprised of three tools for improving your
productivity and satisfaction when building a web app: Yo (the
scaffolding tool), Grunt (the build tool) and Bower (for package
management).
Built-in support for CoffeeScript, Compass and more. It works with r.js (RequireJS), unit testing, etc.
As for your requirements:
Bower is used for dependency management
Bower can work with local files, git://, http:// and more
Built-in support for minification and concatenation (even for your images)
Built-in support to automatically compile CoffeeScript and Compass (with LiveReload)
As stated in the build process: if you're using AMD, I will pass those modules through r.js so you don't have to.
All features:
Lightning-fast scaffolding — Easily scaffold new projects with
customizable templates (e.g HTML5 Boilerplate, Twitter Bootstrap),
RequireJS and more.
Great build process — Not only do you get
minification and concatenation; I also optimize all your image files,
HTML, compile your CoffeeScript and Compass files, if you're using
AMD, I will pass those modules through r.js so you don't have to.
Automatically compile CoffeeScript & Compass — Our LiveReload watch
process automatically compiles source files and refreshes your browser
whenever a change is made so you don't have to.
Automatically lint your scripts — All your scripts are automatically run against JSHint to ensure they're following language best-practices.
Built-in preview server — No more having to fire up your own HTTP Server. My built-in
one can be fired with just one command.
Awesome Image Optimization — I optimize all your images using OptiPNG and JPEGTran so your users can spend less time downloading assets and more time using your app.
Killer package management — Need a dependency? It's just a keystroke
away. I allow you to easily search for new packages via the
command-line (e.g. bower search jquery), install them and keep them
updated without needing to open your browser.
PhantomJS Unit Testing — Easily run your unit tests in headless WebKit via PhantomJS. When
you create a new application, I also include some test scaffolding for
your app.
Bower may suit your needs (1). And (2) for the rest you have RequireJS.
From the README:
Bower is a package manager for the web. Bower lets you easily install assets such as images, CSS and JavaScript, and manages dependencies for you.
To install a package:
bower install jquery
bower install git://github.com/maccman/package-jquery.git
bower install http://code.jquery.com/jquery-1.7.2.js
bower install ./repos/jquery
I just came across inject.js
Some of the features, from the project site:
Inject (Apache Software License 2.0) is a revolutionary way to manage your dependencies in a Library Agnostic way. Some of its major features include:
CommonJS Compliance in the Browser (exports.*)
View the full CommonJS Support Matrix
Cross domain retrieval of files (via easyXDM)
localStorage (load a module once)
Look at the Jam package manager. Following is the description from its homepage
For front-end developers who crave maintainable assets, Jam is a package manager for JavaScript. Unlike other repositories, we put the browser first.
It seems a lot similar to npm in how it works.
Install the package like below:
jam install backbone
Keep the packages up-to-date by executing:
jam upgrade
jam upgrade {package}
Optimize packages for production
jam compile compiled.min.js
Jam dependencies can be added in package.json file.
For complete documentation, read the Jam documentation.
There are a couple of options:
Browserify which allows you to import modules
RequireJS addresses the same problem
One that seems to be in active development is JoinJS
Component might also be of interest. It does not manage dependencies per se, but it allows you to use chopped up versions of otherwise large libraries.
Here's a solution that takes a very different approach: package up all the modules into a JSON object and require modules by reading and executing the file content without additional requests.
Pure client-side demo implementation: http://strd6.github.io/editor/
https://github.com/STRd6/require/blob/master/main.coffee.md
STRd6/require depends on having a JSON package available at runtime. The require function is generated for that package. The package contains all the files your app could require. No further HTTP requests are made because the package bundles all dependencies. This is as close as one can get to the Node.js style require on the client.
The structure of the package is as follows:
entryPoint: "main"
distribution:
main:
content: "alert(\"It worked!\")"
...
dependencies:
<name>: <a package>
Unlike Node.js a package doesn't know its external name. It is up to the package including the dependency to name it. This provides complete encapsulation.
Given all that setup here's a function that loads a file from within a package:
loadModule = (pkg, path) ->
unless (file = pkg.distribution[path])
throw "Could not find file at #{path} in #{pkg.name}"
program = file.content
dirname = path.split(fileSeparator)[0...-1].join(fileSeparator)
module =
path: dirname
exports: {}
context =
require: generateRequireFn(pkg, module)
global: global
module: module
exports: module.exports
PACKAGE: pkg
__filename: path
__dirname: dirname
args = Object.keys(context)
values = args.map (name) -> context[name]
Function(args..., program).apply(module, values)
return module
This external context provides some variable that modules have access to.
A require function is exposed to modules so they may require other modules.
Additional properties such as a reference to the global object and some metadata are also exposed.
Finally we execute the program within the module and given context.
This answer will be most helpful to those who wish to have a synchronous Node.js style require statement in the browser and are not interested in remote script loading solutions.
I use Hem with npm, and I wanted to add some additional benefits that I think weren't covered so far.
Hem has a self-contained web server (Strata) so you can develop your code without even needing to recompile. I never use hem build unless I am publishing an application.
You don't need to use Spine.js to use Hem. You can use it to compile arbitrary CoffeeScript packages if you set up file slug.json correctly. Here's one of my packages that is auto-compiled with cakefile:
TurkServer
Speaking of the above, Hem allows you to link other dependencies on your local system in with an npm link and combines them seamlessly even when you are using the Strata server. In fact, you needn't even use the cake method above; you can just link directly to CoffeeScript from dependent projects.
Hem supports eco (embedded CoffeeScript) for views and Stylus for CSS, and compiles all that, along with your CoffeeScript code, into one JavaScript file and one CSS file.
Here's a basic list for getting set up with a Spine.js, Hem, and CoffeeScript application. Feel free to ignore the Spine.js parts. In fact, sometimes I use spine app to set up a directory structure for a non-Spine.js application, then edit file slug.json to change to a different compilation structure.
Install NPM: curl http://npmjs.org/install.sh | sh on a Unix-like system. I'll assume it's available from the command line.
Install Hem globally (npm install -g hem). Development has branched as of late, so you might want to get it straight out of GitHub, checkout a branch, and npm install -g . in that folder.
npm install -g spine.app will make spine available as a global command
spine app folder will make a Spine project called app in folder, generating the right directory structure and a bunch of skeleton files to get started.
cd to folder and edit file dependencies.json for the libraries you need. Add them to file slug.json, so that Hem knows where to find them as well.
Optional: npm link any of your local packages in development to folder node_modules, and you can add them to file slug.json for Hem (either an index.js file to include directly or an index.coffee file if you want hem to compile it.)
npm install . to download all the dependencies you just entered in.
If you take a look at the default spine configuration, there is a app/lib/setup.coffee where you require all the libraries you need from your dependencies. Examples:
# Spine.app had these as dependencies by default
require('json2ify')
require('es5-shimify')
require('jqueryify')
require('spine')
require('spine/lib/local')
require('spine/lib/ajax')
require('spine/lib/manager')
require('spine/lib/route')
# D3.js was installed via file 'dependencies.json'
require 'd3/d3.v2'
In file index.coffee, you just do require lib/setup and load the main controller for your application. In addition, you need to do require on any other classes in those other controllers. You can use spine controller something or spine model something to generate templates for controllers and models. A typical Spine controller looks like the following, using Node.js' require:
Spine = require('spine')
# Require other controllers
Payment = require('controllers/payment')
class Header extends Spine.Controller
constructor: ->
# Initialize the class
active: ->
super
#render()
render: ->
# Pull down some eco files
#html require('views/header')
# Makes this visible to other controllers
module.exports = Header
The default generated index.html file will usually be fine for loading your application, but modify as necessary. Per your requirements, it only pulls in one .js and one .css file, which you never need to modify.
Edit your stylus files as necessary in the css folder. It's a lot more flexible than CSS :)
From folder, run hem server to start a Hem server, and navigate to localhost:9294 to see your application (if you installed Hem globally). It has some hidden arguments, for example, --host 0.0.0.0 listens on all ports.
Build the rest of your application using proper MVC techniques, and use stylus for CSS and eco for views. Or don't use Spine at all, and Hem will still work great with CoffeeScript and npm. There are many examples of projects using both models.
One more thing: normally, hem server will update automatically as you update your code and save files, which makes it a cinch to debug. Running hem build will compile your application into two files, application.js, which is minified and application.css. If you run hem server after this, it will use those files and no longer update automatically. So don't hem build until you actually need a minified version of your application for deployment.
Additional references: Spine.js and Hem: Getting started
Check out Cartero if you are using Node.js or Express.js on the backend.
I'd suggest to check out the Dojo Toolkit which seems to meet most of your requirements. The one I'm not sure about is CoffeeScript.
Dojo works with modules written in the Asynchronous Module Definition (AMD) format. It has a build system with packages and you can aggregate them in one or several files (called layers). Apparently it accepts Git type repositories, and more details on the build system are on Creating Builds.
For the record, v1.9 beta is expected next month.
Dependency injection with asynchronous loading and Browserify will be another good choice, compares to RequireJS.
asynchronous-frontend-dependency-management-without-AMD
Another framework that satisfies all my criteria released recently is http://duojs.org/ (and it also supports treating other resources, like CSS, as dependencies).

Categories

Resources