Gulp Uglify Producing Strange Characters from Paper.js Library - javascript

I'm using the Bower package of Paper.js in a project. I'm using Gulp for preparing the project for the browser. There are some characters, however, that look like this in bower_components/paper/dist/paper-full.min.js:
\xaa\xb5\xba\xc0-\xd6\xd8-\xf6\xf8-\u02c1\u02c6-\u02d1\u02e0-\u02e4\u02ec...
that end up like this after going through Gulp:
ªµºÀ-ÖØ-öø-ˁˆ-ˑˠ-ˤˬˮͰ-ʹͶͷͺ-ͽΆΈ-ΊΌΎ-ΡΣ-ϵϷ-ҁҊ-ԧԱ-Ֆՙա-ևא-תװ-ײؠ-يٮ...
Resulting in a console error of
Uncaught SyntaxError: Invalid regular expression: /[ªµºÀ-ÖØ-öø-ˈ-Ë‘Ë -ˤˬˮͰ-ʹͶͷͺ-ͽΆΈ-ΊΌΎ-Ρ[Lots more strange characters] Range out of order in character class
Here is a relevant chunk of my gulp file:
var gulp = require('gulp');
var stylus = require('gulp-stylus');
var uglify = require('gulp-uglify');
var concat = require('gulp-concat');
var browserify = require('browserify');
var reactify = require('reactify');
var source = require('vinyl-source-stream');
var jade = require('gulp-jade');
var nib = require('nib');
var del = require('del');
var cfg = require('./cfg.json');
var action = {
clean: function(cb){
del([
['./', cfg.dir.root.dev].join('')
], cb);
},
concatLibs: function(){
gulp.src([
'./bower_components/jquery/dist/jquery.min.js',
'./bower_components/react/react.js',
'./bower_components/when/es6-shim/Promise.js',
'./bower_components/lodash/lodash.min.js',
'./bower_components/postal.js/lib/postal.min.js',
'./bower_components/oboe/dist/oboe-browser.min.js',
'./bower_components/paper/dist/paper-full.min.js'])
.pipe(uglify())
.pipe(concat('lib.js'))
.pipe(gulp.dest(['./', cfg.dir.root.dev, cfg.dir.type.js].join('')));
},
...
I've isolated the problem to this part of the process:
.pipe(uglify()) // in the concatLibs action
That is, commenting out this line does not generate the unusual characters, and does not result in a console error.
The uglify() method seems to be the canonical one, required like so: var uglify = require('gulp-uglify'). So what's the problem? Why is uglify() causing this?

The problem here seems to be that you are uglifying already minified libraries.
A better approach would be to not touch third party libraries and only concat them as is. This would mean that you should include minified sources of your libs whenever possible. You now have unminified versions of react.js and Promise.js.
Both react and es6-shim supply a minified version, which are already in place when doing a bower install. To fix this, your task should look like:
...
concatLibs: function(){
gulp.src([
'./bower_components/jquery/dist/jquery.min.js',
'./bower_components/react/react.min.js',
'./bower_components/when/es6-shim/Promise.min.js',
'./bower_components/lodash/lodash.min.js',
'./bower_components/postal.js/lib/postal.min.js',
'./bower_components/oboe/dist/oboe-browser.min.js',
'./bower_components/paper/dist/paper-full.min.
.pipe(concat('lib.js'))
.pipe(gulp.dest(['./', cfg.dir.root.dev, cfg.dir.type.js].join('')));
}
...
This will supply you with a lib.js with only minified third party libraries with a minimal change to your gulpfile.js.

While, using Unicode characters, ascii_only param need be set as true.
In this case .pipe(uglify({output: {ascii_only:true}})) should work.
ascii_only (default false) -- escape Unicode characters in strings and regexps (affects directives with non-ascii characters becoming invalid). Uglify options

Related

Browserified script not working as expected with Typed.js

Working with javascript library from here. It works as expected , when used without browserify, as it creates global Typed function in browser. But when used browserify with gulp. It's not working as expected, as it not creating global function "Typed", when loaded in browser, throwing error Typed is not function.
in my main script, imported library as below .
import 'typed.js/lib/typed.js';
Is there anything I missed, or I have to use it some other way?
gulp.js
var gulp = require('gulp');
var babelify = require('babelify');
var browserify = require('browserify');
var source = require('vinyl-source-stream');
gulp.task('scripts', async function() {
browserify('./main.js')
.transform("babelify", {"presets":["#babel/preset-env"]})
.bundle()
.pipe(source('script.js'))
.pipe(gulp.dest('dist/js'));
});

custom targets / running arbitrary code

In make it's possible to define custom targets that have no relevance to the actual code that they act upon, in the sense that they are language agnostic.
release_sortof:
#echo packaging release...
tar czf release.tar.gz file1 file2 file3
ls /dev/null
ls /dev/stderr
ls /dev/stdout
I know the example above is horrible, but the point I'm trying to illustrate is that the code in the release_sortof target doesn't depend on the fact that my project uses code written in C, for example; nor does it depend on me using Make built-ins such as foreach.
Is there a way to work with javascript/<INSERT-NAME>script files without using the ever insufficient plugins available for gulp? As in, could I lint my coffeescript with coffeelint by directly calling the coffeelint module:
var gulp = require('gulp')
, coffeelint = require('coffeelint')
;
gulp.task('lint', function() {
/* run coffeelint on source files */
});
Or can this only be done using plugins?
Another example would be to run arbitrary code like so:
var spawn = require('child_process').spawn;
gulp.task('blue', function() {
var child = spawn('ls');
/* do stuff with spawned child process */
});
I do this kind of thing for browserify using vinyl-source-stream - basically allowing you to use the library as it is, and not using gulp-* plugins.
var browserify = require('browserify'),
gulp = require('gulp'),
source = require('vinyl-source-stream'),
stringify = require('stringify'),
plumber = require('gulp-plumber'),
config = require('../config').scripts;
gulp.task('browserify', function () {
return browserify(config.app)
.transform(stringify(['.html']))
.bundle()
.pipe(plumber())
.pipe(source('bundle.js'))
.pipe(gulp.dest(config.dest));
});
Heres the npm - https://www.npmjs.com/package/vinyl-source-stream
Use conventional text streams at the start of your gulp or vinyl
pipelines, making for nicer interoperability with the existing npm
stream ecosystem.
Maybe that will help you?

Gulp starter kit with gulp-load-plugins

I have a gulp starter kit for my project, however, I want to use gulp-load-plugins to for devDependencies of package.json file. My file structure is
ProjectName
Gulp
-Tasks
-broswerify.js
-browserSync.js
-jade.js
-lint.js
Gulpfile.js
config.json
package.json
Gulpfile.js
var requireDir = require('require-dir');
var dir = requireDir('./gulp/tasks', {recurse: true});
jade.js (Which is working as expected using gulp-load-plugins)
var gulp = require('gulp');
var config = require('../../config.json');
var plugins = require('gulp-load-plugins')();
gulp.task('jade', function(){
return gulp.src(config.jade.src)
.pipe(plugins.jade())
.pipe(gulp.dest(config.jade.build))
});
browsersync.js (which is not working using gulp-load-plugins)
var gulp = require('gulp');
var config = require('../../config.json').browsersync;
var plugins = require('browsersync'); // works
//var plugins = require('gulp-load-plugins')(); // it doesn't works.
gulp.task('browsersync', function () {
plugins.browserSync.init(config); // (browsersync required)
//plugins.browserSync.init(config) it doesn't work. (gulp-load-plugins required)
});
I would like to know that if there is a better way to do that?
Why would you wan't to use gulp-load-plugins if you have a seperate file for each plugin?
This is how i load gulp-load-plugins :
$ = require('gulp-load-plugins')({
pattern: ['gulp-*', 'gulp.*'],
replaceString: /\bgulp[\-.]/,
lazy: true,
camelize: true
}),
Here is an example of a revision plugin:
// revision styles
gulp.task('rev-styles', function () {
return gulp.src(dev.css)
.pipe($.rev())
.pipe($.cssmin())
.pipe(gulp.dest(dist.css))
.pipe($.filesize())
.pipe($.rev.manifest({merge: true}))
.pipe(gulp.dest('./'))
//rev replace
.on('end', function() {
return gulp.src(['./rev-manifest.json', 'dist/*.html'])
.pipe($.revCollector({
replaceReved: true,
dirReplacements: {
'css': 'css'
}
}))
.pipe(gulp.dest(dist.dist))
});
});
As you can see all my pipes are called .pipe($.pluginName()) meaning $ stands for gulp- . If you have a plugin named gulp-name-secondname you call it like this: .pipe($.nameSecondname()) . Top were i require gulp-load-plugins i have camelize set to true . Lazy loading loads only the plugins you use not all of them .
As a side note i strongly recommend not separating plugins in diffrent files but you can modulate them, meaning separating important tasks in separate files like compilation file , optimization file , build file, etc .
This might help you understand gulp file separation better http://macr.ae/article/splitting-gulpfile-multiple-files.html
Careful with gulp-load-plugins because it slows your tasks , for example i run gulp-webserver , when i use it with gulp-load-plugins the task finishes after 200ms versus 20ms if i use it normally. So don't use with everything, play with it see how much performance you lose on each task and prioritize.
I have used gulp-load-plugins but found that it mainly adds complexity and obscures my code. At also makes it harder to understand for people less familiar with Gulp. It looks cleaner and easier to understand to have all modules explicitly declared at the top.

gulp-filter not filtering out excluded files correctly

I'm experimenting with using gulpjs instead of grunt for a project. I'm attempting to use gulp filter to ignore vendor libraries when running jsHint on my code. I've based my code off of the code from the readme's example, but the files have not been filtered.
I'm running node 0.10.26, gulp 3.8.0,and gulp filter 0.4.1
I'm trying to run jshint on a directory wcui/app/js that contains many other directories of JS files, with about 120 js files total. I want to exclude the vendor directory only.
My code looks like this:
var gulp = require('gulp');
var gulpFilter = require('gulp-filter');
var jshint = require('gulp-jshint');
var srcs = {
scripts: ['wcui/app/js/**/*.js'],
styles: ['wcui/app/css/**/*.less','wcui/app/css/**/*.css']
};
var dests = {
scripts: 'wcui/static/js/',
styles: 'wcui/static/css/'
};
gulp.task('scripts', function() {
var filter = gulpFilter('!wcui/app/js/vendor');
return gulp.src(srcs.scripts)
.pipe(filter)
.pipe(jshint())
.pipe(jshint.reporter('jshint-stylish'))
.pipe(filter.restore)
.pipe(gulp.dest(dests.scripts));
});
gulp.task('styles', function() {
return gulp.src(srcs.styles)
.pipe(gulp.dest(dests.styles));
});
gulp.task('dev',['scripts','styles']);
Right now running gulp dev does the same thing it did before I added the filter, linting every js file. How can I change this to make it filter correctly? The gulp example had the src in the format 'wcui/app/js/*.js' but when I admit the ** glob, I don't get subdirectories at all. Other than that I think I'm following the readme to the letter (with changes for my particular task).
For readers that have a more up-to-date version of gulp-filter (release at the time of writing is 1.0.0)
The release of version 0.5.0 of gulp-filter introduced multimatch 0.3.0 which come with a breaking change.
Breaking change
Using a negate ! as the first pattern no longer matches anything.
Workaround: ['*', '!cake']
Basically, what it means is you need to replace
var filter = gulpFilter('!wcui/app/js/vendor');
with
var filter = gulpFilter(['*', '!wcui/app/js/vendor']);
and you are good to go.
Also, as noted in the comment by MildlySerious, you should have .pipe(filter.restore()) instead of .pipe(filter.restore)
Use filter like this gulpFilter(['*', '!app/vendor'])

Concat scripts in order with Gulp

Say, for example, you are building a project on Backbone or whatever and you need to load scripts in a certain order, e.g. underscore.js needs to be loaded before backbone.js.
How do I get it to concat the scripts so that they’re in order?
// JS concat, strip debugging and minify
gulp.task('scripts', function() {
gulp.src(['./source/js/*.js', './source/js/**/*.js'])
.pipe(concat('script.js'))
.pipe(stripDebug())
.pipe(uglify())
.pipe(gulp.dest('./build/js/'));
});
I have the right order of scripts in my source/index.html, but since files are organized by alphabetic order, gulp will concat underscore.js after backbone.js, and the order of the scripts in my source/index.html does not matter, it looks at the files in the directory.
So does anyone have an idea on this?
Best idea I have is to rename the vendor scripts with 1, 2, 3 to give them the proper order, but I am not sure if I like this.
As I learned more I found Browserify is a great solution, it can be a pain at first but it’s great.
I had a similar problem recently with Grunt when building my AngularJS app. Here's a question I posted.
What I ended up doing is to explicitly list the files in order in the grunt config. The config file will then look like this:
[
'/path/to/app.js',
'/path/to/mymodule/mymodule.js',
'/path/to/mymodule/mymodule/*.js'
]
Grunt is able to figure out which files are duplicates and not include them. The same technique will work with Gulp as well.
Another thing that helps if you need some files to come after a blob of files, is to exclude specific files from your glob, like so:
[
'/src/**/!(foobar)*.js', // all files that end in .js EXCEPT foobar*.js
'/src/js/foobar.js',
]
You can combine this with specifying files that need to come first as explained in Chad Johnson's answer.
I have used the gulp-order plugin but it is not always successful as you can see by my stack overflow post gulp-order node module with merged streams. When browsing through the Gulp docs I came across the streamque module which has worked quite well for specifying order of in my case concatenation. https://github.com/gulpjs/gulp/blob/master/docs/recipes/using-multiple-sources-in-one-task.md
Example of how I used it is below
var gulp = require('gulp');
var concat = require('gulp-concat');
var handleErrors = require('../util/handleErrors');
var streamqueue = require('streamqueue');
gulp.task('scripts', function() {
return streamqueue({ objectMode: true },
gulp.src('./public/angular/config/*.js'),
gulp.src('./public/angular/services/**/*.js'),
gulp.src('./public/angular/modules/**/*.js'),
gulp.src('./public/angular/primitives/**/*.js'),
gulp.src('./public/js/**/*.js')
)
.pipe(concat('app.js'))
.pipe(gulp.dest('./public/build/js'))
.on('error', handleErrors);
});
With gulp-useref you can concatenate every script declared in your index file, in the order in which you declare it.
https://www.npmjs.com/package/gulp-useref
var $ = require('gulp-load-plugins')();
gulp.task('jsbuild', function () {
var assets = $.useref.assets({searchPath: '{.tmp,app}'});
return gulp.src('app/**/*.html')
.pipe(assets)
.pipe($.if('*.js', $.uglify({preserveComments: 'some'})))
.pipe(gulp.dest('dist'))
.pipe($.size({title: 'html'}));
});
And in the HTML you have to declare the name of the build file you want to generate, like this:
<!-- build:js js/main.min.js -->
<script src="js/vendor/vendor.js"></script>
<script src="js/modules/test.js"></script>
<script src="js/main.js"></script>
In your build directory you will have the reference to main.min.js which will contain vendor.js, test.js, and main.js
The sort-stream may also be used to ensure specific order of files with gulp.src. Sample code that puts the backbone.js always as the last file to process:
var gulp = require('gulp');
var sort = require('sort-stream');
gulp.task('scripts', function() {
gulp.src(['./source/js/*.js', './source/js/**/*.js'])
.pipe(sort(function(a, b){
aScore = a.path.match(/backbone.js$/) ? 1 : 0;
bScore = b.path.match(/backbone.js$/) ? 1 : 0;
return aScore - bScore;
}))
.pipe(concat('script.js'))
.pipe(stripDebug())
.pipe(uglify())
.pipe(gulp.dest('./build/js/'));
});
I just add numbers to the beginning of file name:
0_normalize.scss
1_tikitaka.scss
main.scss
It works in gulp without any problems.
I have my scripts organized in different folders for each package I pull in from bower, plus my own script for my app. Since you are going to list the order of these scripts somewhere, why not just list them in your gulp file? For new developers on your project, it's nice that all your script end-points are listed here. You can do this with gulp-add-src:
gulpfile.js
var gulp = require('gulp'),
less = require('gulp-less'),
minifyCSS = require('gulp-minify-css'),
uglify = require('gulp-uglify'),
concat = require('gulp-concat'),
addsrc = require('gulp-add-src'),
sourcemaps = require('gulp-sourcemaps');
// CSS & Less
gulp.task('css', function(){
gulp.src('less/all.less')
.pipe(sourcemaps.init())
.pipe(less())
.pipe(minifyCSS())
.pipe(sourcemaps.write('source-maps'))
.pipe(gulp.dest('public/css'));
});
// JS
gulp.task('js', function() {
gulp.src('resources/assets/bower/jquery/dist/jquery.js')
.pipe(addsrc.append('resources/assets/bower/bootstrap/dist/js/bootstrap.js'))
.pipe(addsrc.append('resources/assets/bower/blahblah/dist/js/blah.js'))
.pipe(addsrc.append('resources/assets/js/my-script.js'))
.pipe(sourcemaps.init())
.pipe(concat('all.js'))
.pipe(uglify())
.pipe(sourcemaps.write('source-maps'))
.pipe(gulp.dest('public/js'));
});
gulp.task('default',['css','js']);
Note: jQuery and Bootstrap added for demonstration purposes of order. Probably better to use CDNs for those since they are so widely used and browsers could have them cached from other sites already.
Try stream-series. It works like merge-stream/event-stream.merge() except that instead of interleaving, it appends to the end. It doesn't require you to specify the object mode like streamqueue, so your code comes out cleaner.
var series = require('stream-series');
gulp.task('minifyInOrder', function() {
return series(gulp.src('vendor/*'),gulp.src('extra'),gulp.src('house/*'))
.pipe(concat('a.js'))
.pipe(uglify())
.pipe(gulp.dest('dest'))
});
merge2 looks like the only working and maintained ordered stream merging tool at the moment.
Update 2020
The APIs are always changing, some libraries become unusable or contain vulnerabilities, or their dependencies contain vulnerabilities, that are not fixed for years. For text files manipulations you'd better use custom NodeJS scripts and popular libraries like globby and fs-extra along with other libraries without Gulp, Grunt, etc wrappers.
import globby from 'globby';
import fs from 'fs-extra';
async function bundleScripts() {
const rootPaths = await globby('./source/js/*.js');
const otherPaths = (await globby('./source/**/*.js'))
.filter(f => !rootFiles.includes(f));
const paths = rootPaths.concat(otherPaths);
const files = Promise.all(
paths.map(
// Returns a Promise
path => fs.readFile(path, {encoding: 'utf8'})
)
);
let bundle = files.join('\n');
bundle = uglify(bundle);
bundle = whatever(bundle);
bundle = bundle.replace(/\/\*.*?\*\//g, '');
await fs.outputFile('./build/js/script.js', bundle, {encoding: 'utf8'});
}
bundleScripts.then(() => console.log('done');
An alternative method is to use a Gulp plugin created specifically for this problem. https://www.npmjs.com/package/gulp-ng-module-sort
It allows you to sort your scripts by adding in a .pipe(ngModuleSort()) as such:
var ngModuleSort = require('gulp-ng-module-sort');
var concat = require('gulp-concat');
gulp.task('angular-scripts', function() {
return gulp.src('./src/app/**/*.js')
.pipe(ngModuleSort())
.pipe(concat('angularAppScripts.js))
.pipe(gulp.dest('./dist/));
});
Assuming a directory convention of:
|——— src/
| |——— app/
| |——— module1/
| |——— sub-module1/
| |——— sub-module1.js
| |——— module1.js
| |——— module2/
| |——— sub-module2/
| |——— sub-module2.js
| |——— sub-module3/
| |——— sub-module3.js
| |——— module2.js
| |——— app.js
Hope this helps!
For me I had natualSort() and angularFileSort() in pipe which was reordering the files. I removed it and now it works fine for me
$.inject( // app/**/*.js files
gulp.src(paths.jsFiles)
.pipe($.plumber()), // use plumber so watch can start despite js errors
//.pipe($.naturalSort())
//.pipe($.angularFilesort()),
{relative: true}))
I just use gulp-angular-filesort
function concatOrder() {
return gulp.src('./build/src/app/**/*.js')
.pipe(sort())
.pipe(plug.concat('concat.js'))
.pipe(gulp.dest('./output/'));
}
I'm in a module environnement where all are core-dependents using gulp.
So, the core module needs to be appended before the others.
What I did:
Move all the scripts to an src folder
Just gulp-rename your core directory to _core
gulp is keeping the order of your gulp.src, my concat src looks like this:
concat: ['./client/src/js/*.js', './client/src/js/**/*.js', './client/src/js/**/**/*.js']
It'll obviously take the _ as the first directory from the list (natural sort?).
Note (angularjs):
I then use gulp-angular-extender to dynamically add the modules to the core module.
Compiled it looks like this:
angular.module('Core', ["ui.router","mm.foundation",(...),"Admin","Products"])
Where Admin and Products are two modules.
if you would like to order third party libraries dependencies, try wiredep. This package basically checks each package dependency in bower.json then wire them up for you.
I tried several solutions from this page, but none worked. I had a series of numbered files which I simply wanted be ordered by alphabetical foldername so when piped to concat() they'd be in the same order. That is, preserve the order of the globbing input. Easy, right?
Here's my specific proof-of-concept code (print is just to see the order printed to the cli):
var order = require('gulp-order');
var gulp = require('gulp');
var print = require('gulp-print').default;
var options = {};
options.rootPath = {
inputDir: process.env.INIT_CWD + '/Draft',
inputGlob: '/**/*.md',
};
gulp.task('default', function(){
gulp.src(options.rootPath.inputDir + options.rootPath.inputGlob, {base: '.'})
.pipe(order([options.rootPath.inputDir + options.rootPath.inputGlob]))
.pipe(print());
});
The reason for the madness of gulp.src? I determined that gulp.src was running async when I was able to use a sleep() function (using a .map with sleeptime incremented by index) to order the stream output properly.
The upshot of the async of src mean dirs with more files in it came after dirs with fewer files, because they took longer to process.
In my gulp setup, I'm specifying the vendor files first and then specifying the (more general) everything, second. And it successfully puts the vendor js before the other custom stuff.
gulp.src([
// vendor folder first
path.join(folder, '/vendor/**/*.js'),
// custom js after vendor
path.join(folder, '/**/*.js')
])
Apparently you can pass in the "nosort" option to gulp.src gulp.src.

Categories

Resources