Develop Reference

Can I achieve this with Gulp + Browserify? Do I need Webpack instead?

I've been using Gulp for a while now. Then I was introduced to Browserify. Now Webpack is on my horizon given I'm moving towards React. I digress...
I'd like to achieve these steps:
require front end dependencies, such as jQuery and Backbone, in a node application so as to have one source of truth - npm.
Concatenate those dependencies (in whatever order I choose) into a dependencies.js file suitable for the browser (Browserify, right?).
Concatenate the above file with my own JavaScript files (in whatever order I choose) into a all.min.js file, minified, wrapped in a self executing anonymous function - (function(){ /*all code here*/})() - so as to avoid any global variables / variables on the window object / global pollution. (<-- this one is the key).
I'd love to be able to handle this all in just Gulp as I'm used to it, but the whole global pollution thing is killing me. Take a look at the gulp file:
var gulp = require('gulp');
var concat = require('gulp-concat-util'); // Makes concat.header, concat.footer available.
var uglify = require('gulp-uglify');
var browserify = require('browserify');
var source = require('vinyl-source-stream');
var buffer = require('vinyl-buffer');
gulp.task('browserify', function() {
return browserify('libraries.js') // File containing a list of requires's.
.bundle()
.pipe(source('dependencies.js')) // Concatenated file.
.pipe(buffer())
.pipe(gulp.dest('./dev/dependencies')); // Destination directory.
});
gulp.task('scripts', function() {
return gulp.src([
'dev/dependencies/dependencies.js',
'dev/js/models/**/*.js', // Models before views & collections.
'dev/js/**/!(app|templates)*.js',
'dev/js/templates.js',
'dev/js/app.js'
])
.pipe(concat('all.min.js')) // Final file.
.pipe(concat.header('(function(){')) // Attempts to wrap in a SEAF.
.pipe(concat.footer('\n})();'))
.pipe(uglify())
.pipe(gulp.dest('public'));
});
All of that code will indeed produce a single file, minified, wrapped in a SEAF, but the libraries I required (say Backbone & jQuery) are still accessible in the global scope. My life! Is this just the way it works? Those libraries are attaching themselves to window (I looked in the code) but I thought maybe some Browserify magic could deal with that. Any thoughts are appreciated!

A lot of modules will use UMD (Universal Module Definition) patterns. One of the patterns is to always declare the module as a global resource so that other libraries that do not use a module loader will have the library available globally. Having single libraries globally registered is not terribly bad as long as they do not collide. Since you are wrapping your app code in an IIFE and hopefully not adding to the window object directly the globals should be limited to just the 3rd party libraries.

I'm using webpack for processing js and css and I'm pretty happy with it.
All library dependencies are installed by npm install and saved in package.json. In the result I have bunch of minified files that contains all my app. JS modules are stored in function scopes, so, no global scope is spoiled. Easy to maintain, easy to understand and track all dependencies used in each module.
I'll give you some basic example.
File: webpack.config.js - configuration for webpack
module.exports = {
// entrypoint for webpack to start processing
entry: {
bundle: './js/main.js'
},
output: {
path: __dirname + '/' + 'assets',
filename: '[name].js',
publicPath : '/assets/',
}
};
File: ./js/main.js - entrypoint for webpack
// This is require from node_modules
var $ = require('jquery');
// this is require for my local module
var MyModule1 = require('./modules/module1.js');
File: ./modules/module1.js - local module
var OtherLib = require('other-lib');
...
// exporting some data for others
module.exports = {
// export object
};
Pros:
One tool for everything
Easy dependency resolving and installing
Bunch of plugins/loaders from community https://webpack.github.io/docs/list-of-loaders.html
Easy to use from the box
Cons that I faced:
webpack is JS preprocessor, so, if you need to have processed CSS as separate file (not in minified JS), you need to extract CSS from JS result. So, I'm using ExtractTextPlugin.
It quite slow while processing lots of SASS files in --watch mode.

browserify and browserify-shim, do we really need browserify-shim?

I am trying to figure out what the difference between browserify and browserify-shim. After reading a few tutorial, the only difference I found that is to make required function or libraries to global scope?
browserify makes all js files to one single file without dependency conflict or issues.
var $ = require('jquery');
$('body').css('background','red');
browserify main.js -o bundle.js
so we can just include bundle.js in html file, but we are unable to use $ in html, as jQuery is not in global environment.
browserify-shim is for making jQuery to global scope?
"browserify-shim": {
"jquery": "$"
}
But if only for making jQuery to global scope, I think we can simply done by using this
var $ = window.jQuery = require("jquery"); in browserify
So my question is that, do we really need browserify-shim or if there is something else we need from browserify-shim but we are unable to do it though browserify?

Migrating node code base to TypeScript: global scope?

I'm trying to migrate large node codebase to TypeScript. To make things easier, I just want to start by renaming .js files to .ts files, fix any semantic issues, without much refactoring, and then gradually improve the code base.
Let's consider the following example:
logger.js:
exports = function(string) {
console.log(string);
}
moduleA.js:
var logger = require('./logger')
exports.someAction = function() {
logger.log('i'm in module a')
}
moduleB.js:
//var logger = require('./logger')
exports.someOtherAction = function() {
logger.log('i'm in module B')
}
moduleC.js:
var moduleA = require('./moduleA');
var moduleB = require('./moduleB');
exports.run = function(string) {
moduleA.someAction();
moduleB.someOtherAction(); //exception here - logger not defined
}
So if i execute moduleB.someOtherAction() i'll get an exception, because logger is not defined in scope in moduleB.js.
But typescript compiles just fine, because logger is declared in moduleA, and this is because (if i understand correctly) typescript treats all of those files as a single compilation unit.
So is there anyway to avoid this without much refactoring?
Update
I've created a sample project which can be found here
If i run typescript compiler, i get no errors, though logger is commented out in moduleB.ts:
g#w (master) ~/projects/ts-demo: gulp generate
[10:39:46] Using gulpfile ~/projects/ts-demo/gulpfile.js
[10:39:46] Starting 'generate'...
[10:39:46] Starting 'clean'...
[10:39:46] Finished 'clean' after 11 ms
[10:39:46] Starting '<anonymous>'...
[10:39:47] Finished '<anonymous>' after 1.4 s
[10:39:47] Finished 'generate' after 1.41 s
g#w (master) ~/projects/ts-demo:
Update 2
Ok, so this is expected behaviour as stated in TypeScript deep dive book:
If you now create a new file bar.ts in the same project, you will be allowed by the TypeScript type system to use the variable foo from foo.ts as if it was available globally

Louy is right. Every file (when compiling for node with CommonJS) is created like its own module. Just like the normal case in node.
So to get this to work you could do something like this:
logger.ts
export default (str: string) => console.log(str);
moduleA.ts
import logger from './logger';
export var someAction = () => {
logger("i'm in module a");
}
moduleB.ts
import logger from './logger';
export var someOtherAction = () => {
logger("i'm in module B");
}
moduleC.ts
import { someAction } from './moduleA';
import { someOtherAction } from './moduleB';
someAction();
someOtherAction();
I also used a tsconfig.json:
{
"compilerOptions": {
"module": "commonjs",
"noImplicitAny": true
},
"files": [
"logger.ts",
"moduleA.ts",
"moduleB.ts",
"moduleC.ts"
]
}
Compiling this (just type tsc in the folder containing the tsconfig.json) results in one .js file per .ts file. It should compile and run just fine. The main thing you need to take from this is that each file is its own self contained module when compiling for node/CommonJS.
EDIT Due to edits in question:
Ok after looking at your code again there is a couple of reasons that this compiles but fail to run properly:
If you send a group of files to the compiler all references in all of those files, and all of the variables in the global scope will be available in all .ts files sent to the compiler. This is confusing and sometimes regrettable, but at the moment it is how things work. This in your case results in that the require function defined in the global scope in node.d.ts is available in all .ts files, not only in moduleC.ts; if the reference is removed from moduleC.ts you will get an error in logger.ts because the require function is not defined for example. It also results in var logger = .. defined in logger.ts will be seen as available in the other .ts files. Typescript assumes that it will be available at run time.. not ideal when compiling for node, but not a problem when you are writing actual typescript not trying to compile pure javascript. In fact, because it is pure javascript the compiler does not recognise it as a node module, and treats it as a generic javascript file.
When compiling with commonjs each file is compiled on its own by the typescript compiler. If any references or imports are found those will be followed and used to type and verify the code of course, but the compilation is still done on a per file basis (var something = require('./whatever'); is not an import, it is interpreted by the compiler as a variable being assigned by a function taking a string as an argument.)
So thats why it compiles, lets go on to why its not working then.
Each file outputted by the compiler in commonjs mode is its own "node-module". If you want to have access to one module from within another in node (this has nothing to do with typescript really, it is just how node works) you will need to require that file. in moduleB.ts (and in moduleB.js for that matter) there is no indication to node to say what logger is. So you simply need to require logger.
In the example you are referring to in your second update of the question commonjs is not being used, and further more it is in connection with typescripts internal modules - not node modules which is a completely different thing.
So the simple answer is, that if you want to use one file from within a another in node you have to require it. Simply uncommenting the commented reference in moduleB.ts should get everything up and running.
In the current state of your code the typescript compiler is not helping you much, because it can't really. The code is close to pure javascript, to get decent help from the compiler you will need to start converting it to typescript. The first step might be to replace var asdf = require("") with import asdf = require("");, this will make it possible for the compiler to check the references and help you a lot in return.
I realise this is a wordy answer, but you managed to create yourself a confusing and difficult to explain behaviour.

There's something called internal/external modules in typescript.
Internal modules can declare stuff that can be referenced everywhere in the project. Think about typings/lib files, you should be able to reference things like Error and String everywhere in your project.
External modules on the other hand have to be imported to be referenced. This is what you want to do.
The difference between internal and external modules is that external ones use import and export.
Internal ones use declare module/namespace/var.
Just add "compilerOptions": {"module": "commonjs"} to your tsconfig.json file to enable external modules.

Use Browserify with JavaScript libraries such as Backbone or Underscore?

I know I can install underscore using npm but that's not what I can do in my work environment. I need to be able to download the Underscore.js library and then make it "browserify-compatible".
So let's assume Underscore.js looks something like this:
(function() {
var root = this;
// Rest of the code
}.call(this));
I downloaded that file on my hard drive and saved it as under.js.
My file that requires underscore looks like this:
var underscore = require("./under");
console.log(underscore);
And then I run browserify from the cli.
I have an HTML page called test.html and basically all it does is load the generated bundle.js.
However, the console.log(underscore) line fails - says that underscore is undefined.
What have I tried?
Obviously I added module.exports to the first line - right before the function definition in under.js, and that's how I got the error mentioned above. I also tried the method from this answer , still got the same error.
So, how would I use Browserify to load libraries such as Underscore.js or Backbone without using npm-installed modules?

That's because browserify does not add variables to the global scope. The version you download is identical to the version that you install via NPM.
You need to explicitly attach it to the window to export it to the top level scope.
If you create a file called "expose_underscore.js" and put this in it:
var _ = require('./under');
window._ = _;
Will do it, followed by: browserify expose_underscore.js > bundle.js and then add bundle.js as a <script> tag you will be able to do the following in your console:
HOWEVER, you shouldn't do this if you're using browserify. The point behind it (and Node's version of commonJS) is that you explicitly require it everywhere you need it. So every file you have that needs underscore should import it to a local variable.
Don't worry -- you will still only have one copy loaded.

I typically add my vendor libs like Underscore as script tags. Underscore will attach itself to the global scope, so then you don't need to require it anywhere to use it.
If you do want to use it in a Browserified fashion, verify that you have the correct path in your require statement (browserify requires are relative paths) and move the module.exports statement to the end of the file.

browserify detect custom require()

I want to use a custom require() function in my application.
Namely I have node's standard require() and a custom one I wrote to require files starting from the root called rootRequire() which internally all it does is:
// rootRequire.js
var path = require('path');
var rootPath = __dirname;
global.rootRequire = function (modulePath) {
var filepath = path.join(rootPath, modulePath);
return require(filepath);
};
module.exports = rootRequire;
But even though rootRequire() internally uses node's require(), it does not pick up any files required through that method
Example:
require('rootRequire.js');
rootRequire('/A.js'); // server side it works, in the browser I get an error saying can't find module A.js

Perhaps, this will answer your question on why it is not working on browser.
Quoting the owner's comment: "Browserify can only analyze static requires. It is not in the scope of browserify to handle dynamic requires".
Technically, you are trying to load files dynamically from a variable with your custom require. Obviously, node.js can handle that.
In case of browserify, all requires are statically loaded and packaged when you run the CLI to build the bundle.
HTH.

What i recommend doing here is this:
First create a bundle to be required of anything you might use in rootRequire().
Something like browserify -r CURRENT_PATH/A.js -r CURRENT_PATH/B.js > rootRequirePackages.js
Then, in your webpage, you can include both rootRequirePackages.js and your regular browserified file.

Perhaps just use RequireJS ? It's simple to set up and relatively easy to use.