Intermittent browserify bundle failure (probably brfs) - javascript

This is an intermittent problem. That is, I can repeat the same build command n times without changing anything and sometimes it will work and sometimes (~30%) it will fail with seemingly no cause. I've been living with this for months now, but it is terribly annoying.
I'm using node (v0.10.35) and browserify (v6.3.4) and brfs (v1.2.0) to bundle some html in my javascript. Intermittently, the build fails with:
events.js:72
throw er; // Unhandled 'error' event
^
SyntaxError: Unterminated string constant (17:4) while parsing file: blah.js
at raise (./node_modules/brfs/node_modules/static-module/node_modules/falafel/node_modules/acorn/acorn.js:333:15)
at readString (./node_modules/brfs/node_modules/static-module/node_modules/falafel/node_modules/acorn/acorn.js:1073:11)
at getTokenFromCode (./node_modules/brfs/node_modules/static-module/node_modules/falafel/node_modules/acorn/acorn.js:854:14)
at readToken (./httpd/node_modules/brfs/node_modules/static-module/node_modules/falafel/node_modules/acorn/acorn.js:902:15)
at next (./node_modules/brfs/node_modules/static-module/node_modules/falafel/node_modules/acorn/acorn.js:1232:5)
at eat (./node_modules/brfs/node_modules/static-module/node_modules/falafel/node_modules/acorn/acorn.js:1335:7)
at expect (./node_modules/brfs/node_modules/static-module/node_modules/falafel/node_modules/acorn/acorn.js:1360:5)
at parseExprList (./node_modules/brfs/node_modules/static-module/node_modules/falafel/node_modules/acorn/acorn.js:2443:9)
at parseSubscripts (./node_modules/brfs/node_modules/static-module/node_modules/falafel/node_modules/acorn/acorn.js:2032:24)
at parseSubscripts (./node_modules/brfs/node_modules/static-module/node_modules/falafel/node_modules/acorn/acorn.js:2021:14)
at parseExprSubscripts (./node_modules/brfs/node_modules/static-module/node_modules/falafel/node_modules/acorn/acorn.js:2012:12)
at parseMaybeUnary (./node_modules/brfs/node_modules/static-module/node_modules/falafel/node_modules/acorn/acorn.js:1995:16)
It's different line numbers, but each time the build fails, it points to
fs.readFileSync(
I don't think it matters, but I am using gulp v3.8.8. My gulp task is quite simple.
var gulp = require('gulp');
var browserify = require('browserify');
var source = require('vinyl-source-stream');
gulp.task('build', function(cb) {
return browserify(app.src, {
fullPaths: true,
transform: ['brfs'],
debug: true
}).bundle().pipe(source(app.name)).pipe(gulp.dest(app.dest));
});
Others on this same project with slightly different versions of node and gulp and even different OSes are also experiencing the same problem.
Update: It also fails with just browserify+brfs (most current versions) with the same message. I'm convinced this is a problem with brfs because it doesn't seem to fail if I leave out the transform.
var browserify = require('browserify');
var fs = require('fs');
var b = browserify('blah.js');
b.transform('brfs');
b.bundle().pipe(fs.createWriteStream('out.js'));

I was getting this problem with a similar setup (using reactify to transform, though; it was just giving me "Unterminated string constant" in the error message and didn't specify a js file). Turned out I had a syntax error in a json file (a stray newline inside of a string literal)...

Related

UNKOWN error thrown when compiling typescript using Gulp in VS2015

I have installed Visual Studio 2015 (with no other previous versions) on a new laptop and have pulled down the source for our MVC web app. We have a gulp file with tasks to compile our less and typescript.
When running this task ...
cmd.exe /c gulp -b "C:\Code\Trunk\MyProj\MyProj.Web" --color --gulpfile "C:\Code\Trunk\MyProj\MyProj.Web\Gulpfile.js" typescript
... I get the following error:
[09:43:16] Using gulpfile C:\Code\Trunk\MyProj\MyProj.Web\Gulpfile.js
[09:43:16] Starting 'typescript'...
[09:43:34] Plumber found unhandled error:
Error: UNKNOWN, open 'C:\Code\Trunk\MyProj\MyProj.Web\app\allergy\main.js'
Process terminated with code 0.
Here is the task in the gulp file (with other parts removed for brevity):
var gulp = require("gulp");
var plumber = require("gulp-plumber");
var sourcemaps = require("gulp-sourcemaps");
var typescript = require("gulp-typescript");
var merge = require("merge2");
var paths = {
typescript: {
globpatterns: {
all: "./Scripts/**/*.ts",
excludedefinitions: "!./Scripts/**/*.d.ts"
}
}
};
gulp.task("typescript", function () {
var result = gulp.src([
paths.typescript.globpatterns.all,
paths.typescript.globpatterns.excludedefinitions
])
.pipe(plumber())
.pipe(sourcemaps.init())
.pipe(typescript({
removeComments: true,
declarationFiles: false,
noImplicitAny: false,
noEmitOnError: true,
module: "amd",
target: "ES5"
}));
return merge([
result.js.pipe(gulp.dest("./")),
result.pipe(sourcemaps.write()).pipe(gulp.dest("./"))
]);
});
My colleague has the same set-up as me and gets no error.
Typescript is set to version 1.0 in the project file (<TypeScriptToolsVersion>1.0</TypeScriptToolsVersion>) and I can't change this just now. I wondered if the reason was beacuse I don't have this version installed on my machine but my colleague doesn't either. C:\Program Files (x86)\Microsoft SDKs\TypeScript only has a folder for 1.7
I noticed that the task completes successfully if I remove either of the 2 lines with in the merge block.
It's a different .js file in the error message each time
I searched the web to see what the UNKNOWN error even means but couldn't find anything obvious / helpful. Anyone know how to fix this error? Or how I go about finding out why it's being thrown?
EDIT 20-Jan-2016
So, I was getting this error consistently for about a week ... and now it has stopped happening. I haven't made any changes to my development environment either. I'd like to leave this question open since I'm curious as to why this happened.

Gulp Uglify Producing Strange Characters from Paper.js Library

I'm using the Bower package of Paper.js in a project. I'm using Gulp for preparing the project for the browser. There are some characters, however, that look like this in bower_components/paper/dist/paper-full.min.js:
\xaa\xb5\xba\xc0-\xd6\xd8-\xf6\xf8-\u02c1\u02c6-\u02d1\u02e0-\u02e4\u02ec...
that end up like this after going through Gulp:
ªµºÀ-ÖØ-öø-ˁˆ-ˑˠ-ˤˬˮͰ-ʹͶͷͺ-ͽΆΈ-ΊΌΎ-ΡΣ-ϵϷ-ҁҊ-ԧԱ-Ֆՙա-ևא-תװ-ײؠ-يٮ...
Resulting in a console error of
Uncaught SyntaxError: Invalid regular expression: /[ªµºÀ-ÖØ-öø-ˈ-Ë‘Ë -ˤˬˮͰ-ʹͶͷͺ-ͽΆΈ-ΊΌΎ-Ρ[Lots more strange characters] Range out of order in character class
Here is a relevant chunk of my gulp file:
var gulp = require('gulp');
var stylus = require('gulp-stylus');
var uglify = require('gulp-uglify');
var concat = require('gulp-concat');
var browserify = require('browserify');
var reactify = require('reactify');
var source = require('vinyl-source-stream');
var jade = require('gulp-jade');
var nib = require('nib');
var del = require('del');
var cfg = require('./cfg.json');
var action = {
clean: function(cb){
del([
['./', cfg.dir.root.dev].join('')
], cb);
},
concatLibs: function(){
gulp.src([
'./bower_components/jquery/dist/jquery.min.js',
'./bower_components/react/react.js',
'./bower_components/when/es6-shim/Promise.js',
'./bower_components/lodash/lodash.min.js',
'./bower_components/postal.js/lib/postal.min.js',
'./bower_components/oboe/dist/oboe-browser.min.js',
'./bower_components/paper/dist/paper-full.min.js'])
.pipe(uglify())
.pipe(concat('lib.js'))
.pipe(gulp.dest(['./', cfg.dir.root.dev, cfg.dir.type.js].join('')));
},
...
I've isolated the problem to this part of the process:
.pipe(uglify()) // in the concatLibs action
That is, commenting out this line does not generate the unusual characters, and does not result in a console error.
The uglify() method seems to be the canonical one, required like so: var uglify = require('gulp-uglify'). So what's the problem? Why is uglify() causing this?
The problem here seems to be that you are uglifying already minified libraries.
A better approach would be to not touch third party libraries and only concat them as is. This would mean that you should include minified sources of your libs whenever possible. You now have unminified versions of react.js and Promise.js.
Both react and es6-shim supply a minified version, which are already in place when doing a bower install. To fix this, your task should look like:
...
concatLibs: function(){
gulp.src([
'./bower_components/jquery/dist/jquery.min.js',
'./bower_components/react/react.min.js',
'./bower_components/when/es6-shim/Promise.min.js',
'./bower_components/lodash/lodash.min.js',
'./bower_components/postal.js/lib/postal.min.js',
'./bower_components/oboe/dist/oboe-browser.min.js',
'./bower_components/paper/dist/paper-full.min.
.pipe(concat('lib.js'))
.pipe(gulp.dest(['./', cfg.dir.root.dev, cfg.dir.type.js].join('')));
}
...
This will supply you with a lib.js with only minified third party libraries with a minimal change to your gulpfile.js.
While, using Unicode characters, ascii_only param need be set as true.
In this case .pipe(uglify({output: {ascii_only:true}})) should work.
ascii_only (default false) -- escape Unicode characters in strings and regexps (affects directives with non-ascii characters becoming invalid). Uglify options

combined coffee and js development with Gulp

Using Gulp to streamline my build process on a per-save basis, while eliminating the need for "temp" files is my goal (and why I chose to use Gulp over Grunt)
I've recently discovered that, apparently, the coffeescript compiler cannot handle dealing with basic Javascript as a source
So this, errors out:
gulp.task('scripts', function() {
var stream =
gulp.src([
"bower_components/d3/d3.js",
"bower_components/angular/angular.js",
"bower_components/foundation/js/foundation.js",
"public-dev/scripts/app.js",
"public-dev/scripts/**/*.js",
"public-dev/scripts/**/*.coffee"
])
.pipe(coffee())
.pipe(concat('compiled.js'))
.pipe(gulp.dest('./public/scripts'))
;
return stream;
});
This produces the output/errpr:
[11:58:51] Starting 'scripts'...
events.js:72
throw er; // Unhandled 'error' event
^
/var/www/node.fortiguard.dev/bower_components/d3/d3.js:1:2: error: reserved word "function"
!function() {
^
How should I set this up, without again creating any temp files?
What I liked about the LESS compiler, is that it can still be fed straight CSS files and it'd be fine. I was kind of hoping the Coffee compiler would react the same way
I use gulp-add-src to do that.
var gulp = require('gulp'),
coffee = require('gulp-coffee'),
concat = require('gulp-concat'),
addsrc = require('gulp-add-src');
// Scripts
gulp.task('coffee', function () {
return gulp.src('src/coffee/**/*.coffee')
.pipe(coffee())
.pipe(addsrc('src/coffee/lib/*.js'))
.pipe(concat('compiled.js'))
.pipe(gulp.dest('dist'));
});
Sadly, it doesn't. What you can do, is using the underlying event-stream's merge method. Then you'll have one pipeline for the coffee files that gets compiled and one for the javascript side.
Here is an example Gulpfile.coffee:
coffee = require 'gulp-coffee'
es = require 'event-stream'
gulp.task 'scripts', () ->
es.merge(
gulp.src(["public-dev/app.js", "public-dev/scripts/**/*.js"]) # ...
gulp.src("public-dev/**/*.coffee").pipe coffee()
)
.pipe concat 'all.js'
.pipe gulp.dest "build"
I marked Patrick J. S.'s answer as "Correct" becuase in reality, this is exactly what I needed to do.
That said, "event-stream" isn't what I ended up going with, simply because I needed to preserve my dependency structure of files, and event-stream's merge() method does not preserver order nor does it have options to.
Instead I opted for a package called streamqueue, which does preserve order of glob. Probably slower, but order matters in my app unfortunately. In the future I will try to be as modular as possible.

Traversing directory in Node (Grunt)

Having a strange issue when building a grunt plugin.
Basically, I'd like to use a node_module in my grunt task itself. To do this, I want to traverse a level up, then down into the node modules to call their one file specifically.
Originally, I wanted to do this:
../node_modules/github-changes/bin/index.js
However, I get the following error:
Warning: Command failed: /bin/sh: 1: ../node_modules/github-changes/bin/index.js: not found
Use --force to continue.
So, for now I have a hack on using Node's __dirname variable, but it's not very pretty:
var dirHack = __dirname.replace("/tasks", ""), // Terrible hack, need to fix
ghC = dirHack + '/node_modules/github-changes/bin/index.js';
This works, but I'd love to avoid it.
You can see the line here.
What's missing that's causing that error? Am I missing something here?
If github-changes is a dependency, you can resolve a path to file within the package using ghC = require.resolve('github-changes/bin/index.js').
But you're getting that error because you're trying to run that file as a shell script, which it is not. You need to run it with node. The easiest way to find the path to the node executable is process.execPath. See http://nodejs.org/api/process.html#process_process_execpath
Here is an example:
var exec = require('child_process').exec;
var ghC = require.resolve('github-changes/bin/index.js');
exec(process.execPath + ' ' + ghC, function(error) {});

Logging with grunt and qunit

I am running javascript unittests with grunt/qunit. Sometimes the tests fails because of e.g syntax errors in the source files (works fine with file info if syntax errors are introduced in the test files). When that happens grunt simply prints the line number and not the file where the problem is.
Running "qunit:all" (qunit) task
Warning: Line 99: Unexpected identifier Use --force to continue.
Aborted due to warnings.
This does not help much since I have 100 of js files. I have looked into:
https://github.com/gruntjs/grunt-contrib-qunit
and tried to add the following to my Gruntfile.js (grunt.event.on):
module.exports = function(grunt) {
"use:strict";
var reportDir = "output/reports/"+(new Date()).getTime().toString();
grunt.initConfig({
pkg: grunt.file.readJSON('package.json'),
qunit: {
options: {
'--web-security': 'no',
coverage: {
src: ['../src/**/*.js'],
instrumentedFiles: 'output/instrument/',
htmlReport: 'output/coverage',
coberturaReport: 'output/',
linesTresholdPct: 85
}
},
all: ["testsSuites.html"]
}
});
// Has no effect
grunt.event.on('qunit.error.onError', function (msg, stack) {
grunt.util._.each(stack, function (entry) {
grunt.log.writeln(entry.file + ':' + entry.line);
});
grunt.warn(msg);
});
grunt.loadNpmTasks('grunt-contrib-qunit');
grunt.loadNpmTasks('grunt-qunit-istanbul');
grunt.registerTask('test', ['qunit']);
Where testsSuites.html contains:
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<link rel="stylesheet" href="qunit/qunit.css">
<script src="qunit/qunit.js"></script>
<script src="sinonjs/sinon-1.7.3.js"></script>
<script src="sinonjs/sinon-qunit-1.0.0.js"></script>
<!-- Sources -->
<script src="../src/sample.js"></script>
<!-- Test-->
<script src="test/sample-test.js"></script>
</head>
<body>
<div id="qunit"></div>
<div id="qunit-fixture"></div>
<script>
</script>
</body>
</html>
But the source file where the problem is located is still not printed. Is is out of Grunts hands to verify source code/show line number/file where e.g a syntax error is located?
I have also tried running:
grunt test --debug 9
It prints some debug info but not any information regarding syntax errors in the javascript sources.
I have tried to install JSHint and call it on all my javascript source files:
for i in $(find ../src -iname "*.js"); do jshint $i; done
Now I get tons of errors but Grunt is still happy. If I introduce a simple syntax error e.g:
(function(){
var sampleVar 32;
}
to provoke an error in Grunt:
Running "qunit:all" (qunit) task
Warning: Line 2: Unexpected number Use --force to continue.
Aborted due to warnings.
it simply disappears in the stream of errors generated by JSHint. How do I filter JSHint "warnings" from critical errors that will actually make Grunt fail?
Or is it qunit that should be configured for more verbose output?
grunt-contrib-qunit will display filenames when encountering a syntax error. Take this simplified version of your Gruntfile.js:
module.exports = function(grunt) {
"use:strict";
grunt.initConfig({
qunit: {
options: { '--web-security': 'no' },
all: ["testsSuites.html"]
}
});
grunt.loadNpmTasks('grunt-contrib-qunit');
};
Running it gives the error you're looking for:
$ grunt qunit
Running "qunit:all" (qunit) task
Testing testsSuites.html F.
>> global failure
>> Message: SyntaxError: Parse error
>> file:///tmp/src/sample.js:2
Warning: 1/2 assertions failed (17ms) Use --force to continue.
Aborted due to warnings.
The issue you're having looks to be a bug(?) in grunt-qunit-istanbul. The warning you're getting:
Warning: Line 99: Unexpected identifier Use --force to continue.
is Grunt handling an uncaught exception. The exception is being raised by the grunt-qunit-istanbul task. You can prove it by modifying this line in your original Gruntfile.js from:
src: ['../src/**/*.js'],
to:
src: ['../src/**/*.js.nomatch'],
This will prevent grunt-qunit-istanbul from finding and parsing any Javascript files before Qunit is run. If you let Qunit run, its error handler prints out the filenames with syntax errors like you want.
The only fix is the workaround I've described, or to patch grunt-qunit-istanbul to add an error handler for parse errors like Qunit does.
Patching grunt-qunit-istanbul
The function that is throwing the exception is Instrumenter.instrumentSync, which it is supposed to do:
instrumentSync ( code, filename )
Defined in lib/instrumenter.js:380
synchronous instrumentation method. Throws when illegal code is passed to it
You can fix it by wrapping the function call:
diff -r 14008db115ff node_modules/grunt-qunit-istanbul/tasks/qunit.js
--- a/node_modules/grunt-qunit-istanbul/tasks/qunit.js Tue Feb 25 12:14:48 2014 -0500
+++ b/node_modules/grunt-qunit-istanbul/tasks/qunit.js Tue Feb 25 12:19:58 2014 -0500
## -209,7 +209,11 ##
// instrument the files that should be processed by istanbul
if (options.coverage && options.coverage.instrumentedFiles) {
- instrumentedFiles[fileStorage] = instrumenter.instrumentSync(String(fs.readFileSync(filepath)), filepath);
+ try {
+ instrumentedFiles[fileStorage] = instrumenter.instrumentSync(String(fs.readFileSync(filepath)), filepath);
+ } catch (e) {
+ grunt.log.error(filepath + ': ' + e);
+ }
}
cb();
Then the test will keep running (and inform you of the syntax error):
$ grunt qunit
Running "qunit:all" (qunit) task
>> /tmp/src/sample.js: Error: Line 2: Unexpected number
Testing testsSuites.html F.
>> global failure
>> Message: SyntaxError: Parse error
>> file:///tmp/src/sample.js:2
Warning: 1/2 assertions failed (19ms) Use --force to continue.
Aborted due to warnings.
I have used grunt-contrib-qunit in the past but I have never attempted something like this. The problem you are facing is fairly interesting because the docs mention that the event qunit.error.onError should be emitted by grunt but it is not happening for you.
I created a new project using a jquery template and changed the code so that my test would fail. After that I wrote the following code:
grunt.event.on('qunit.error.onError', function(message, stackTrace) {
grunt.file.write('log/qunit-error.log', message);
});
When I ran the command grunt, I received no output in the file. To check this, I made a change to the event:
grunt.event.on('qunit.log', function(result, actual, expected, message, source) {
grunt.file.write('log/qunit-error.log', message);
});
Now, this piece of code did give me the error message in my file but it was useless because I could not get the stacktrace or the exact error message.
After this, I though of reading up the source and this is what I found:
phantomjs.on('error.onError', function (msg, stackTrace) {
grunt.event.emit('qunit.error.onError', msg, stackTrace);
});
The grunt event is only emitted when phantomjs throws an error.
At the moment I am not sure how I can get a phantomjs error while testing a simple JavaScript file without any browser related testing. This is my analysis so far and I hope this helps you in someway.

Categories

Resources