Develop Reference

I'm using Gulp and failing to produce the final development script for production.

So I'm having a slight problem with producing production ready scripts for my project. I'm using gulp to concatenate and minify my css and js, and while the css is working fine the gulp js function isn't generating my final file. Please refer to my code below:
gulp.task('js', function() {
return gulp.src([source + 'js/app/**/*.js'])
.pipe(concat('development.js'))
.pipe(gulp.dest(source + 'js'))
.pipe(rename({
basename: 'production',
suffix: '-min',
}))
.pipe(uglify())
.pipe(gulp.dest(source + 'js/'))
.pipe(notify({ message: 'Scripts task complete', onLast: true }));
});
If anyone has encountered a similar problem or has any tips it would be much appreciated :)

There is nothing wrong with your gulpfile. I tested it and it works perfectly.
The only thing I can guess is that your source is not set correctly. Did you forget the trailing slash '/' ?
I would suggest 2 things to figure it out. Include node path library to check where source is actually pointing to like this:
var path = require('path');
// in gulp task ...
path.resolve(path.resolve(source + 'js/app'));
Make sure it points where you think it does.
Secondly, you could use gulp-debug to establish that any files are found:
npm install gulp-debug
Then
var debug = require('gulp-debug');
// in gulp task ...
return gulp.src([source + 'js/app/**/*.js'])
.pipe(concat('development.js'))
.pipe(debug())
.pipe(gulp.dest(source + 'js'))
.pipe(debug())
// etc.
Good luck!

Based on additional infomation in the comments I realise you are generating JS files in a separate process ...
gulp is asynchronous by default. What this boils down to is that all functions try to run at the same time - if you want a specific order it must be by design. This is great because it's very fast but can be a headache to work with.
Problem
Here is what's basically happening:
// SOME TASK THAT SHOULD BE RUN FIRST
gulp.task('copy-vendor-files-to-tempfolder', function (done) {
// copy files to vendor folder
done()
})
// SOME TASKS THAT DEPEND ON FIRST TASK
gulp.task('complile-styles', function () { /* independent task */ })
gulp.task('concat-vendor-files', function () { /* concat files in vendor folder. depends on vendor files existing */ })
// GENERAL TASK WHICH STARTS OTHERS
gulp.task('ready', ['copy-vendor-files-to-tempfolder', 'compile-styles', 'concat-vendor-files])
When you try to run:
$ gulp ready
GULP TASK WILL FAIL! Folder is being created at the same time!!
NOWHERE TO COPY FILES!
Solution
There are many solutions but the following module has come in handy for me again and again:
npm install run-sequence
Then in your gulpfile.js:
var runSequence = require('run-sequence')
gulp.task('ready', function (done) {
runSequence(
'create-folders', // do this first,
[
'copy-css-files',
'copy-html-files'
], // do these AFTER but in parallel
done // callback when ready
)
})
This will guarantee the folder exists when you try to run the other functions.
In your specific case, you should make sure the task that concatenates the JS files is run after the task that copies them out of vendor.
Note: I'm leaving other answer because it contains useful help for debugging similar issues.
HTH!

gulp-remember seems to output wrong path

[Using gulp 3.9.0, gulp-cached 1.1.0, gulp-remember 0.3.0, gulp-rename 1.2.2]
We're using gulp as a build tool and gulp-cached together with gulp-remember to allow fast incremental rebuilds. Part of the files under build have to be moved to a different output directory and this has to happen in-stream (i.e. not in gulp.dest()), because we're zipping the results afterwards. We use gulp-rename for this.
However, when the build script is called multiple times (gulp.watch() for incremental rebuilds), then it seems to apply the gulp-rename transformation multiple times. It looks like gulp-remember is not actually outputting the files with the path it saw them last time, but with the path they got after the gulp-rename step.
I've narrowed the problem down to the following script:
var gulp = require('gulp');
var cached = require('gulp-cached');
var remember = require('gulp-remember');
var rename = require('gulp-rename');
function build() {
gulp.src("src/**")
.pipe(cached())
.pipe(remember())
.pipe(rename(function(path) {path.dirname = "resources/" + path.dirname;}))
.pipe(gulp.dest('build/'));
}
gulp.task('default', [], function() {
build();
build();
});
Running this on a source directory src with just one file "myfile.js" produces the output file:
/build/resources/resources/myfile.js
If the second call to build() is removed, it produces correctly
/build/resources/myfile.js
And if we insert a third call to build(), it produces
/build/resources/resources/resources/myfile.js
How can we fix this? Shouldn't gulp-remember output the files with the path they had when they passed through it last time before they have been processed further?

You should call remember after the transformations applied. So in this case:
.pipe(cached())
.pipe(rename(function(path) {path.dirname = "resources/" + path.dirname;}))
.pipe(remember())

How can I achieve this using gulp?

I am enumerating the subdirectories in a directory. For each sub directory I would like to apply a number of gulp activities like less compilation, and then create an output file specific to that subdirectory.
I would like the gulp process to continue, as further transformation steps need to be performed later.
Can someone help me understand how I can create these files half way through the "gulp pipeline"?

This seems quite interesting to achieve and gulp has no limitations at all.
I will give you detailed example how I have managed to accomplish such a task a while ago.
Let assume that you have directoryA. Subdirectories childA, childB and childC are contained into directoryA. So basically your tree structure looks like:
directoryA
--childA
--childB
--childC
I am always looking for a flexible solutions so I would suggest to include a JSON file in each subdirectory naming the tasks you would like to running. Using fs you can access these files. You can also use run-sequence to execute gulp tasks synchronously.
For demo purposes place a file named manifest.json inside childA subdirectory.
Manifest.json contains the following declarations:
{
"filesToProccess" : ["./childA/*.js", "./childB/*.js"],
"tasksToRun" :["taskA", "taskB"]
}
Finally gulpfile would like this:
'use strict';
//dependencies declared into package.json
//install them using npm
var gulp = require('gulp'),
fs = require('fs'),
runSequence = require('run-sequence'),
path = require('path');
//these two array will keep the actions you have included into manifest file.
var filesHolder = [], tasksHolder = [];
gulp.task('taskA', function () {
return gulp.src(filesHolder)
.pipe(whatever)
.pipe(gulp.dest('whatever')); //chailed actions
});
gulp.task('taskB', function () {
return gulp.src(filesHolder)
.pipe(whatever)
.pipe(gulp.dest('whatever'));
});
//a simple utility function to read all subdirectories of directoryA
function getDirectories(srcpath) {
return fs.readdirSync(srcpath).filter(function(file) {
return fs.statSync(path.join(srcpath, file)).isDirectory();
});
}
//finally insert the default gulp task
gulp.task('default', function(){
var manifest;
//map directory's A subdirectories
var availableDirs = getDirectories("./directoryA");
//finally loop the available subdirectories, load each manifest file and
availableDirs.forEach(function(subdir) {
manifest = require("./directoryA/"+subdir+"manifest.json");
filesHolder = manifest.filesToProccess;
tasksHolder = manifest.tasksToRun;
runSequence( tasksHolder , function () {
console.log( " Task ended :" + tasksHolder + " for subdirectory : " + subdir);
});
});
});

Gulp starter kit with gulp-load-plugins

I have a gulp starter kit for my project, however, I want to use gulp-load-plugins to for devDependencies of package.json file. My file structure is
ProjectName
Gulp
-Tasks
-broswerify.js
-browserSync.js
-jade.js
-lint.js
Gulpfile.js
config.json
package.json
Gulpfile.js
var requireDir = require('require-dir');
var dir = requireDir('./gulp/tasks', {recurse: true});
jade.js (Which is working as expected using gulp-load-plugins)
var gulp = require('gulp');
var config = require('../../config.json');
var plugins = require('gulp-load-plugins')();
gulp.task('jade', function(){
return gulp.src(config.jade.src)
.pipe(plugins.jade())
.pipe(gulp.dest(config.jade.build))
});
browsersync.js (which is not working using gulp-load-plugins)
var gulp = require('gulp');
var config = require('../../config.json').browsersync;
var plugins = require('browsersync'); // works
//var plugins = require('gulp-load-plugins')(); // it doesn't works.
gulp.task('browsersync', function () {
plugins.browserSync.init(config); // (browsersync required)
//plugins.browserSync.init(config) it doesn't work. (gulp-load-plugins required)
});
I would like to know that if there is a better way to do that?

Why would you wan't to use gulp-load-plugins if you have a seperate file for each plugin?

This is how i load gulp-load-plugins :
$ = require('gulp-load-plugins')({
pattern: ['gulp-*', 'gulp.*'],
replaceString: /\bgulp[\-.]/,
lazy: true,
camelize: true
}),
Here is an example of a revision plugin:
// revision styles
gulp.task('rev-styles', function () {
return gulp.src(dev.css)
.pipe($.rev())
.pipe($.cssmin())
.pipe(gulp.dest(dist.css))
.pipe($.filesize())
.pipe($.rev.manifest({merge: true}))
.pipe(gulp.dest('./'))
//rev replace
.on('end', function() {
return gulp.src(['./rev-manifest.json', 'dist/*.html'])
.pipe($.revCollector({
replaceReved: true,
dirReplacements: {
'css': 'css'
}
}))
.pipe(gulp.dest(dist.dist))
});
});
As you can see all my pipes are called .pipe($.pluginName()) meaning $ stands for gulp- . If you have a plugin named gulp-name-secondname you call it like this: .pipe($.nameSecondname()) . Top were i require gulp-load-plugins i have camelize set to true . Lazy loading loads only the plugins you use not all of them .
As a side note i strongly recommend not separating plugins in diffrent files but you can modulate them, meaning separating important tasks in separate files like compilation file , optimization file , build file, etc .
This might help you understand gulp file separation better http://macr.ae/article/splitting-gulpfile-multiple-files.html
Careful with gulp-load-plugins because it slows your tasks , for example i run gulp-webserver , when i use it with gulp-load-plugins the task finishes after 200ms versus 20ms if i use it normally. So don't use with everything, play with it see how much performance you lose on each task and prioritize.

I have used gulp-load-plugins but found that it mainly adds complexity and obscures my code. At also makes it harder to understand for people less familiar with Gulp. It looks cleaner and easier to understand to have all modules explicitly declared at the top.

gulp-filter not filtering out excluded files correctly

I'm experimenting with using gulpjs instead of grunt for a project. I'm attempting to use gulp filter to ignore vendor libraries when running jsHint on my code. I've based my code off of the code from the readme's example, but the files have not been filtered.
I'm running node 0.10.26, gulp 3.8.0,and gulp filter 0.4.1
I'm trying to run jshint on a directory wcui/app/js that contains many other directories of JS files, with about 120 js files total. I want to exclude the vendor directory only.
My code looks like this:
var gulp = require('gulp');
var gulpFilter = require('gulp-filter');
var jshint = require('gulp-jshint');
var srcs = {
scripts: ['wcui/app/js/**/*.js'],
styles: ['wcui/app/css/**/*.less','wcui/app/css/**/*.css']
};
var dests = {
scripts: 'wcui/static/js/',
styles: 'wcui/static/css/'
};
gulp.task('scripts', function() {
var filter = gulpFilter('!wcui/app/js/vendor');
return gulp.src(srcs.scripts)
.pipe(filter)
.pipe(jshint())
.pipe(jshint.reporter('jshint-stylish'))
.pipe(filter.restore)
.pipe(gulp.dest(dests.scripts));
});
gulp.task('styles', function() {
return gulp.src(srcs.styles)
.pipe(gulp.dest(dests.styles));
});
gulp.task('dev',['scripts','styles']);
Right now running gulp dev does the same thing it did before I added the filter, linting every js file. How can I change this to make it filter correctly? The gulp example had the src in the format 'wcui/app/js/*.js' but when I admit the ** glob, I don't get subdirectories at all. Other than that I think I'm following the readme to the letter (with changes for my particular task).

For readers that have a more up-to-date version of gulp-filter (release at the time of writing is 1.0.0)
The release of version 0.5.0 of gulp-filter introduced multimatch 0.3.0 which come with a breaking change.
Breaking change
Using a negate ! as the first pattern no longer matches anything.
Workaround: ['*', '!cake']
Basically, what it means is you need to replace
var filter = gulpFilter('!wcui/app/js/vendor');
with
var filter = gulpFilter(['*', '!wcui/app/js/vendor']);
and you are good to go.
Also, as noted in the comment by MildlySerious, you should have .pipe(filter.restore()) instead of .pipe(filter.restore)

Use filter like this gulpFilter(['*', '!app/vendor'])