Develop Reference

Separating/combining gulp task sources using yargs and gulp-if

Lets say I want to include all js files in a source folder concatenated in a specific order (using gulp-concat). However, I want to only include certain files if there is a production flag.
For example, in production I need these files:
modals.js
utilities.js
analytics.js
signup.js
cookies.js
However, for local and staging, I don't want these two:
analytics.js
cookies.js
Is it possible using yargs and perhaps gulp-if to specify this? I know that I could have two lists of sources, for instance:
if (argv.production) {
return gulp.src([
'modals.js',
'utilities.js',
'analytics.js',
'signup.js',
'cookies.js'
])
} else if (argv.staging) {
return gulp.src([
'modals.js',
'utilities.js',
'signup.js',
])
}
However, this would mean I'm not adhering to DRY principles, and just feels silly and prone to error. Can I somehow stitch these together as a single source that only includes things like analytics and cookies if the task has a production flag?
In an ideal world, I could inject gulp if statements inside the gulp.src array that would only include the production-only files if the correct flag were passed.
Some out-loud thinking:
- Should I just create two files here? One a base that includes everything necessary across environments, and then a production specific one, then combine those? Seems unnecessarily complex, but could solve the problem.

If your goal is to be a bit more DRY, you can give this a shot, since your set of staging files is a subset of the production files:
var base_list = ['modals.js', 'utilities.js', 'signup.js'];
if (argv.staging) {
return gulp.src(base_list);
} else if (argv.production) {
var prod_list = base_list.concat(['analytics.js', 'cookies.js']);
return gulp.src(prod_list);
}
Or more simplified (as long as only production needs the other files, and all other environments get the subset):
var file_list = ['modals.js', 'utilities.js', 'signup.js'];
if (argv.production) {
file_list = file_list.concat(['analytics.js', 'cookies.js']);
}
return gulp.src(file_list);
There might be some other gulp-specific best practices or tools that could come into play here, but some simple JavaScript might do the trick.

Different settings for debug/local ("grunt serve") vs. dist/build ("grunt")?

I want to define some application settings, but I want to provide different values depending on whether I'm running in 'debug' mode (e.g. grunt serve), or whether the final compiled app is running (e.g. the output of grunt). That is, something like:
angular.module('myApp').factory('AppSettings', function() {
if (DebugMode()) { // ??
return { apiPort: 12345 };
} else {
return { apiPort: 8008 };
}
});
How can I accomplish this?

The way I handle it in my apps:
move all your config data for one environment to a file: config.js, config.json,... whatever your app finds easy to read.
now modify your config file to turn it into a template using grunt config values, and generate the file with grunt-template as part of your build - for example: app.constant('myAppConfig', {bananaHammocks: <%= banana.hammocks %>});
finally, add grunt-stage to switch grunt config values depending on environment: create your different config/secret/(env).json files, update your template (app.constant('myAppConfig', {bananaHammocks: <%= stg.banana.hammocks %>});), and then grunt stage:local:build or grunt stage:prod:build
I find this the good balance between complexity and features (separation between environments, runtime code not concerned with building options,...)

Exclude folders from builds in Brocfile

Is there a way to exclude a folder from a build in a Brocfile (or any other place).
The use case is packaging, where I have an app made of sub-apps within pods. eg.
/app/modules/components
/app/modules/app1
/app/modules/app2
/app/modules/app3
I'd like to build them all when environment is set to 'development' or only eg. 'app1' when environment is 'app1'. Any suggestions?
I have tried different combinations of broccoli-file-remover, broccoli-funnel and broccoli-merge-trees to no avail.
var removeFile = require('broccoli-file-remover');
module.exports = removeFile(app.toTree(), {
paths: ['app/modules/pod1/', 'app/modules/pod2/']
});

Ah, so after actually thinking about this clearly, everything is actually working exactly as expected in my previous example.
I clearly wasn't paying enough attention. app.toTree() is far too late to perform this operation, as everything has already been built and concated.
Luckily, ember-cli does enable addons to modify the appropriate trees at various life cycle milestones.
See: https://github.com/ember-cli/ember-cli/blob/master/ADDON_HOOKS.md for more details on which hooks are currently available.
The hook that should do the trick is Addon.prototype.postprocessTree. Now we have two choices, we can build a standalone addon, via ember addon or we can create a light-weight in-repo addon via ember g in-repo-addon. Typically for these types of situations, I prefer in-repo-addons as they don't require a second project, but otherwise they are the same.
ember g in-repo-addon remove
we need to install broccoli-stew via npm install --save broccoli-stew
include it var stew = require('broccoli-stew');
add hook postprocessTree to the add-on
when the postprocessTree is for the type we care about, use broccoli-stew to remove the directories we no longer care care.
The resulting pull request: https://github.com/WooDzu/ember-exclude-pod/pull/1
Note: I noticed template wasn't one of the types available in postprocess, so I added it: https://github.com/ember-cli/ember-cli/pull/4263 (should be part of the next ember-cli release)
Note: we really do want an additional hook
Addon.prototype.preprocessTree, as to ignore the files before we
even build them. I have opened a related issue:
https://github.com/ember-cli/ember-cli/issues/4262
output of the above steps
var stew = require('broccoli-stew');
module.exports = {
name: 'remove',
isDevelopingAddon: function() {
return true;
},
postprocessTree: function(type, tree){
if (type === 'js' || type === 'template') {
return stew.rm(tree, '*/modules/pod{1,2}/**/*');
} else {
return tree;
}
}
};

I am pretty confident broccoli-stew's rm will handle this correctly.
https://github.com/stefanpenner/broccoli-stew/blob/master/lib/rm.js#L4-L40 there are even tests that test a very similar scenario: https://github.com/stefanpenner/broccoli-stew/blob/master/tests/rm-test.js#L48-L57
var stew = require('broccoli-stew');
module.exports = stew.rm(app.tree(), 'app/modules/{pod1,pod2}');
If this doesn't work, feel free to open an issue on broccoli-stew. Be sure to provide a running example though

This is really late, but I created a Broccoli plugin to do just this. It's available at https://www.npmjs.com/package/broccoli-rm.
(The trick is to detect whether an excluded path is a folder, and then use a glob match to make sure that none of the children of the folder get symlinked during copying.)
var rm = require('broccoli-rm');
var input = app.toTree();
module.exports = output = rm([input], {
paths: ['app/modules/pod1', 'app/modules/pod2']
});

How to obfuscate client js file on http GET in browser in nodejs?

I have express application in nodejs:
var app = express();
// view engine setup
app.set('views', path.join(__dirname, 'views'));
app.set('view engine', 'ejs');
app.use(logger('dev'));
app.use(bodyParser.json());
app.use(bodyParser.urlencoded({ extended: false }));
app.use(cookieParser());
app.use(express.static(path.join(__dirname, 'public')));
app.use('/', routes);
app.use('/users', users);
My js files which go the browser are in /public folder. I want to obfuscate them automatically in runtime, so it'd be hard to read them in the browser. Is it possible to tune this app setup somehow?

Don't obfuscate at runtime. Create a build step which obfuscates your code and serve the obfuscated code in production.
Install UglifyJS:
npm install -g uglify-js
Then you can run the following command to obfuscate your code:
uglifyjs < src/script.js > dist/script.js
Alternatively, install grunt and the uglify plugin:
npm install -g grunt-cli
npm install --save-dev grunt grunt-contrib-uglify
Create a file called "Gruntfile" in the root of your project with the following contents:
'use strict';
module.exports = function (grunt) {
grunt.loadNpmTasks('grunt-contrib-uglify');
grunt.initConfig({
uglify: {
all: {
files: {
'public/scripts.js': 'src/**/*.js'
}
}
}
});
grunt.registerTask('default', ['uglify']);
};
Run grunt from anywhere in your project to obfuscate your code. Update the index.html file in public/ with a <script src="scripts.js"></script> to load the obfuscated, concatenated scripts.
If order matters, list your scripts in order:
files: {
'public/scripts.js': [
'src/1.js',
'src/3.js',
'src/2.js'
]
}

So, you can't actually keep a user from reading your javascript when it's sent to the browser. You can make it very hard, but a determined user will always be able to transform it to something readable through a combination of automated tools and manual labor.
In short, there's no DRM for JavaScript (or any code, for that matter). If you want to protect the intellectual property in your scripts, put a copyright notice on them. If you think someone stole them, talk to a lawyer.
It's certainly worthwhile to minify your JavaScripts, but the only reason, in my opinion, to obfuscate them is because your stupid boss told you to.
If you want to obfuscate it, check out the answer to this question:
How can I obfuscate (protect) JavaScript?
Note that obfuscation is rather expensive, so you really don't want to do it at runtime. You should simply apply it to your javascripts when they change using a build tool like grunt or bower (personally I prefer grunt). These systems have plugins like this to perform the obfuscation.

You can use UglifyJs in your node application and do what you need using its API. Check out their documentation for further details.
Example from Unglify docs
var jsp = require("uglify-js").parser;
var pro = require("uglify-js").uglify;
var orig_code = "... JS code here";
var ast = jsp.parse(orig_code); // parse code and get the initial AST
ast = pro.ast_mangle(ast); // get a new AST with mangled names
ast = pro.ast_squeeze(ast); // get an AST with compression optimizations
var final_code = pro.gen_code(ast); // compressed code here
P.S. People noting that it is slow and blah blah are of course all correct, but hey, if you need it here it is, I just don't understand why judge and post opinions in response to a clear question.

You can obfuscate the code using command line utility, such as this:
app.use('/javascripts',function(req,res,next){
if (process.env.OBFUSCATE != 1){
next();
}
var cmd = "java -jar ./gcc/compiler.jar ./public/javascripts" + req.url;// + "> ./tmp" + req.url + ".tmp";
exec(cmd,function (error, stdout, stderr){
res.write(stdout);
res.end();
if(error !== null){
console.log('exec error: ' + error);
}
});
});

My tips are easily bypassed, but without be careful, we can be trapped. And it is only client part, not nodejs part.
Only live view of the page
You can replace or remove script tag with javascript for hide this in live view of the page. But if you watch directly the network, you can see easily the javascript file/code.
<div id="RemoveMe0">
<script type="text/javascript">
//This code it is hidden to live view.
var my_var = 5 + 5;
$('#RemoveMe0').remove();
//or document.getElementById("RemoveMe0").innerHTML = "";
</script>
</div>
For include javascript :
<div id="RemoveMe1">
<script type="text/javascript" src="Javascript/MyJS.js"></script>
<script>
//Your include it is hidden to live view.
$('#RemoveMe1').remove();
</script>
</div>
Only direct view
Put your files in an HTML file (myfile.js to myfile.html), like this on a direct view you can execute a javascript function.
function Hello() {
alert("Hello");
}
Hello();
//<script>document.body.innerHTML = "";</script>
Conclusion :
For this tip, rename file or use .htaccess. And like first tip, if you watching the network you see full file.
Or minize/parse your JS
You can use tool like this :
minize your js : This tool use eval function, and try to complicate your script.
Javascript Obfuscator : Complicates the code for beginners, it's easy to by-pass.
Google Closure JS Compiler : Optimize, compress, and minify your code. It is a natural tool for the production environment.

I see you are using express and EJS. What I do is create another folder called jsobf that holds the obfucated files. Basically duplicate all your js files with the same name if you want.
In EJS you can do if else check in your ejs files like this :
<% if (process.env.NODE_ENV === "production") { %>
<script src="/jsobf/myjavascriptfile-prod.js"></script>
<% } else { %>
<script src="/js/myjavascriptfile-dev.js"></script>
<% } %>
At the top of the file where you do app.listen.. declare you a global variable:
const isProduction = true;
app.listen(PORT, function () {
if(isProduction){
process.env.NODE_ENV = 'production';
}
});
Then all you need to do before you deploy your code is manually obfuscate your code at a site like obfuscator.io
In your dev branch set the isProduction to false and in production branch to true. This way when you are developing everything is taken care of for you.
This may not be the cleanest solution but it works perfectly fine for me if you dont care to obfuscate manually before production deploys.

JavaScript require() on client side

Is it possible to use require() (or something similar) on client side?
Example
var myClass = require('./js/myclass.js');

You should look into require.js or head.js for this.

I've been using browserify for that. It also lets me integrate Node.js modules into my client-side code.
I blogged about it here: Add node.js/CommonJS style require() to client-side JavaScript with browserify

If you want to have Node.js style require you can use something like this:
var require = (function () {
var cache = {};
function loadScript(url) {
var xhr = new XMLHttpRequest(),
fnBody;
xhr.open('get', url, false);
xhr.send();
if (xhr.status === 200 && xhr.getResponseHeader('Content-Type') === 'application/x-javascript') {
fnBody = 'var exports = {};\n' + xhr.responseText + '\nreturn exports;';
cache[url] = (new Function(fnBody)).call({});
}
}
function resolve(module) {
//TODO resolve urls
return module;
}
function require(module) {
var url = resolve(module);
if (!Object.prototype.hasOwnProperty.call(cache, url)) {
loadScript(url);
}
return cache[url];
}
require.cache = cache;
require.resolve = resolve;
return require;
}());
Beware: this code works but is incomplete (especially url resolving) and does not implement all Node.js features (I just put this together last night).
YOU SHOULD NOT USE THIS CODE in real apps but it gives you a starting point. I tested it with this simple module and it works:
function hello() {
console.log('Hello world!');
}
exports.hello = hello;

I asked myself the very same questions. When I looked into it I found the choices overwhelming.
Fortunately I found this excellent spreadsheet that helps you choice the best loader based on your requirements:
https://spreadsheets.google.com/lv?key=tDdcrv9wNQRCNCRCflWxhYQ

Take a look at requirejs project.

I have found that in general it is recommended to preprocess scripts at compile time and bundle them in one (or very few) packages with the require being rewritten to some "lightweight shim" also at compile time.
I've Googled out following "new" tools that should be able to do it
http://mixu.net/gluejs/
https://github.com/jrburke/almond
https://github.com/component/builder2.js
And the already mentioned browserify should also fit quite well - http://esa-matti.suuronen.org/blog/2013/04/15/asynchronous-module-loading-with-browserify/
What are the module systems all about?
Older Stack Overflow explanation - Relation between CommonJS, AMD and RequireJS?
Detailed discussion of various module frameworks and the require() they need is in Addy Osmani - Writing Modular JavaScript With AMD, CommonJS & ES Harmony

You can create elements to the DOM, which loads items.
Like such:
var myScript = document.createElement('script'); // Create new script element
myScript.type = 'text/javascript'; // Set appropriate type
myScript.src = './js/myclass.js'; // Load javascript file

Simply use Browserify, what is something like a compiler that process your files before it go into production and packs the file in bundles.
Think you have a main.js file that require the files of your project, when you run browserify in it, it simply process all and creates a bundle with all your files, allowing the use of the require calls synchronously in the browser without HTTP requests and with very little overhead for the performance and for the size of the bundle, for example.
See the link for more info: http://browserify.org/

Some answers already - but I would like to point you to YUI3 and its on-demand module loading. It works on both server (node.js) and client, too - I have a demo website using the exact same JS code running on either client or server to build the pages, but that's another topic.
YUI3: http://developer.yahoo.com/yui/3/
Videos: http://developer.yahoo.com/yui/theater/
Example:
(precondition: the basic YUI3 functions in 7k yui.js have been loaded)
YUI({
//configuration for the loader
}).use('node','io','own-app-module1', function (Y) {
//sandboxed application code
//...
//If you already have a "Y" instance you can use that instead
//of creating a new (sandbox) Y:
// Y.use('moduleX','moduleY', function (Y) {
// });
//difference to YUI().use(): uses the existing "Y"-sandbox
}
This code loads the YUI3 modules "node" and "io", and the module "own-app-module1", and then the callback function is run. A new sandbox "Y" with all the YUI3 and own-app-module1 functions is created. Nothing appears in the global namespace. The loading of the modules (.js files) is handled by the YUI3 loader. It also uses (optional, not show here) configuration to select a -debug or -min(ified) version of the modules to load.

Here's a solution that takes a very different approach: package up all the modules into a JSON object and require modules by reading and executing the file content without additional requests.
https://github.com/STRd6/require/blob/master/main.coffee.md
STRd6/require depends on having a JSON package available at runtime. The require function is generated for that package. The package contains all the files your app could require. No further http requests are made because the package bundles all dependencies. This is as close as one can get to the Node.js style require on the client.
The structure of the package is as follows:
entryPoint: "main"
distribution:
main:
content: "alert(\"It worked!\")"
...
dependencies:
<name>: <a package>
Unlike Node a package doesn't know it's external name. It is up to the pacakge including the dependency to name it. This provides complete encapsulation.
Given all that setup here's a function that loads a file from within a package:
loadModule = (pkg, path) ->
unless (file = pkg.distribution[path])
throw "Could not find file at #{path} in #{pkg.name}"
program = file.content
dirname = path.split(fileSeparator)[0...-1].join(fileSeparator)
module =
path: dirname
exports: {}
context =
require: generateRequireFn(pkg, module)
global: global
module: module
exports: module.exports
PACKAGE: pkg
__filename: path
__dirname: dirname
args = Object.keys(context)
values = args.map (name) -> context[name]
Function(args..., program).apply(module, values)
return module
This external context provides some variable that modules have access to.
A require function is exposed to modules so they may require other modules.
Additional properties such as a reference to the global object and some metadata
are also exposed.
Finally we execute the program within the module and given context.
This answer will be most helpful to those who wish to have a synchronous node.js style require statement in the browser and are not interested in remote script loading solutions.

I find the component project giving a much more streamlined workflow than other solutions (including require.js), so I'd advise checking out https://github.com/component/component . I know this is a bit late answer but may be useful to someone.

Here's a light weight way to use require and exports in your web client. It's a simple wrapper that creates a "namespace" global variable, and you wrap your CommonJS compatible code in a "define" function like this:
namespace.lookup('org.mydomain.mymodule').define(function (exports, require) {
var extern = require('org.other.module');
exports.foo = function foo() { ... };
});
More docs here:
https://github.com/mckoss/namespace

The clientside-require library provides an asynchronous load() function that can be used to load any JS file or NPM module (which uses module.exports), any .css file, any .json, any .html, any any other file as text.
e.g.,
npm install clientside-require --save
<script src = '/node_modules/clientside-require/dist/bundle.js'></script>
<script>
load('color-name') // an npm module
.then(color_name=>{
console.log(color_name.blue); // outputs [0, 0, 255]
})
</script>
A really cool part of this project is that inside of any load()ed script, you can use the synchronous require() function the same way you would expect in node.js!
e.g.,
load('/path/to/functionality.js')
and inside /path/to/functionality.js:
var query_string = require("qs") // an npm module
module.exports = function(name){
return qs.stringify({
name:name,
time:new Date()
}
}
That last part, implementing the synchronous require() method, is what enables it to utilize NPM packages built to run on the server.
This module was designed to implement the require functionality as closely as possible in the browser. Disclaimer: I have written this module.

Yes it is very easy to use, but you need to load javascript file in browser by script tag
<script src="module.js"></script>
and then user in js file like
var moduel = require('./module');
I am making a app using electron and it works as expected.