How can I use gulp to replace a string in a file? - javascript

I am using gulp to uglify and make ready my javascript files for production. What I have is this code:
var concat = require('gulp-concat');
var del = require('del');
var gulp = require('gulp');
var gzip = require('gulp-gzip');
var less = require('gulp-less');
var minifyCSS = require('gulp-minify-css');
var uglify = require('gulp-uglify');
var js = {
src: [
// more files here
'temp/js/app/appConfig.js',
'temp/js/app/appConstant.js',
// more files here
],
gulp.task('scripts', ['clean-js'], function () {
return gulp.src(js.src).pipe(uglify())
.pipe(concat('js.min.js'))
.pipe(gulp.dest('content/bundles/'))
.pipe(gzip(gzip_options))
.pipe(gulp.dest('content/bundles/'));
});
What I need to do is to replace the string:
dataServer: "http://localhost:3048",
with
dataServer: "http://example.com",
In the file 'temp/js/app/appConstant.js',
I'm looking for some suggestions. For example perhaps I should make a copy of the appConstant.js file, change that (not sure how) and include appConstantEdited.js in the js.src?
But I am not sure with gulp how to make a copy of a file and replace a string inside a file.
Any help you give would be much appreciated.


Gulp streams input, does all transformations, and then streams output. Saving temporary files in between is AFAIK non-idiomatic when using Gulp.
Instead, what you're looking for, is a streaming-way of replacing content. It would be moderately easy to write something yourself, or you could use an existing plugin. For me, gulp-replace has worked quite well.
If you want to do the replacement in all files it's easy to change your task like this:
var replace = require('gulp-replace');
gulp.task('scripts', ['clean-js'], function () {
return gulp.src(js.src)
.pipe(replace(/http:\/\/localhost:\d+/g, 'http://example.com'))
.pipe(uglify())
.pipe(concat('js.min.js'))
.pipe(gulp.dest('content/bundles/'))
.pipe(gzip(gzip_options))
.pipe(gulp.dest('content/bundles/'));
});
You could also do gulp.src just on the files you expect the pattern to be in, and stream them seperately through gulp-replace, merging it with a gulp.src stream of all the other files afterwards.


You may also use module gulp-string-replace which manages with regex, strings or even functions.
Example:
Regex:
var replace = require('gulp-string-replace');
gulp.task('replace_1', function() {
gulp.src(["./config.js"]) // Every file allown.
.pipe(replace(new RegExp('#env#', 'g'), 'production'))
.pipe(gulp.dest('./build/config.js'))
});
String:
gulp.task('replace_1', function() {
gulp.src(["./config.js"]) // Every file allown.
.pipe(replace('environment', 'production'))
.pipe(gulp.dest('./build/config.js'))
});
Function:
gulp.task('replace_1', function() {
gulp.src(["./config.js"]) // Every file allown.
.pipe(replace('environment', function () {
return 'production';
}))
.pipe(gulp.dest('./build/config.js'))
});


I think that the most correct solution is to use the gulp-preprocess module. It will perform the actions you need, depending on the variable PRODUCTION, defined or not defined during the build.
Source code:
/* #ifndef PRODUCTION */
dataServer: "http://localhost:3048",
/* #endif */
/* #ifdef PRODUCTION **
dataServer: "http://example.com",
/* #endif */
Gulpfile:
let preprocess = require('gulp-preprocess');
const preprocOpts = {
PRODUCTION: true
};
gulp.task('scripts', ['clean-js'], function () {
return gulp.src(js.src)
.pipe(preprocess({ context: preprocOpts }))
.pipe(uglify())
.pipe(concat('js.min.js'))
.pipe(gulp.dest('content/bundles/'));
}
This is the best solution because it allows you to control the changes that are made during the build phase.


There I have a versioning specific example for your reference.
let say you have version.ts file and it contains the version code inside it. You now can do as the follows:
gulp.task ('version_up', function () {
gulp.src (["./version.ts"])
.pipe (replace (/(\d+)\.(\d+)(?:\.(\d+))?(?:\-(\w+))?/, process.env.VERSION))
.pipe (gulp.dest ('./'))
});
the above regex works for many situation on any conventional version formats.

Related

How to locate the path of parent directory in gulp.dest()?

I'm new comer in Gulp, and i got stock when i try to output my files to their parent's path.
My directories like this:
app/page_a/less/a.less
app/page_a/css
My goal is:
app/page_a/less/a.less
app/page_a/css/a.css
Here's what i'm doing:
I use ** because i want to match all of my page_* module.
gulp.task('test', function () {
var files = ['./app/**/*.less'];
return gulp.src(files, {
base: '.' // Now the base url becomes where the *.less is.
})
.pipe(less())
.pipe(gulp.dest('.'));
});
My result:( which is obviously wrong :( )
app/page_a/less/a.less
app/page_a/less/a.css
app/page_a/css
So how can i output *.css into the corresponding app/page_*/css path?

The path you specify in the base option to gulp.src() isn't relative to each respective .less file, but rather the current working directory. This is usually the directory your Gulpfile.js resides in.
What you actually want to do is alter the path for each resulting .css file after your less() plugin has run, so when the .css files are written to the directory specified by gulp.dest() the /less/ part of the path has been replaced with /css/.
This is exactly what the gulp-rename plugin is for:
var gulp = require('gulp');
var less = require('gulp-less');
var rename = require('gulp-rename');
gulp.task('default', function () {
return gulp.src( './app/**/*.less')
.pipe(less())
.pipe(rename(function(file) {
file.dirname = file.dirname.replace(/less$/, "css");
}))
.pipe(gulp.dest('./app/'));
});

With gulp.dest('.') you say 'here'
Try with:
.pipe(gulp.dest('./app/page_a/css/'));
This should be work

How can I achieve this using gulp?

I am enumerating the subdirectories in a directory. For each sub directory I would like to apply a number of gulp activities like less compilation, and then create an output file specific to that subdirectory.
I would like the gulp process to continue, as further transformation steps need to be performed later.
Can someone help me understand how I can create these files half way through the "gulp pipeline"?

This seems quite interesting to achieve and gulp has no limitations at all.
I will give you detailed example how I have managed to accomplish such a task a while ago.
Let assume that you have directoryA. Subdirectories childA, childB and childC are contained into directoryA. So basically your tree structure looks like:
directoryA
--childA
--childB
--childC
I am always looking for a flexible solutions so I would suggest to include a JSON file in each subdirectory naming the tasks you would like to running. Using fs you can access these files. You can also use run-sequence to execute gulp tasks synchronously.
For demo purposes place a file named manifest.json inside childA subdirectory.
Manifest.json contains the following declarations:
{
"filesToProccess" : ["./childA/*.js", "./childB/*.js"],
"tasksToRun" :["taskA", "taskB"]
}
Finally gulpfile would like this:
'use strict';
//dependencies declared into package.json
//install them using npm
var gulp = require('gulp'),
fs = require('fs'),
runSequence = require('run-sequence'),
path = require('path');
//these two array will keep the actions you have included into manifest file.
var filesHolder = [], tasksHolder = [];
gulp.task('taskA', function () {
return gulp.src(filesHolder)
.pipe(whatever)
.pipe(gulp.dest('whatever')); //chailed actions
});
gulp.task('taskB', function () {
return gulp.src(filesHolder)
.pipe(whatever)
.pipe(gulp.dest('whatever'));
});
//a simple utility function to read all subdirectories of directoryA
function getDirectories(srcpath) {
return fs.readdirSync(srcpath).filter(function(file) {
return fs.statSync(path.join(srcpath, file)).isDirectory();
});
}
//finally insert the default gulp task
gulp.task('default', function(){
var manifest;
//map directory's A subdirectories
var availableDirs = getDirectories("./directoryA");
//finally loop the available subdirectories, load each manifest file and
availableDirs.forEach(function(subdir) {
manifest = require("./directoryA/"+subdir+"manifest.json");
filesHolder = manifest.filesToProccess;
tasksHolder = manifest.tasksToRun;
runSequence( tasksHolder , function () {
console.log( " Task ended :" + tasksHolder + " for subdirectory : " + subdir);
});
});
});

How to setup jade includes with gulp-jade

I am currently using gulp-jade and I am struggling on how to setup Jade includes in my gulpfile.js.(For clarification, I am referring to this here http://jade-lang.com/reference/includes/) The following is the code in my gulpfile.js
var gulp = require('gulp');
var browserSync = require('browser-sync');
var sass = require('gulp-sass');
var uglify = require('gulp-uglify');
var jade = require('gulp-jade');
var jshint = require('gulp-jshint');
var fileinclude = require('gulp-file-include');
var reload = browserSync.reload;
//compile jade to html
gulp.task('templates', function() {
var YOUR_LOCALS = {};
gulp.src('./app/jade/*.jade')
.pipe(jade({
locals: YOUR_LOCALS
}))
.pipe(gulp.dest('./dist/'))
});
//reload files, once jade compilation happens
gulp.task('jade-watch', ['templates'], reload);
//Sass task for live injecting into all browsers
gulp.task('sass', function () {
gulp.src('./app/scss/*.scss')
.pipe(sass())
.pipe(gulp.dest('./dist/css'))
.pipe(reload({stream: true}));
});
//Separate task for the reaction to js files make change even without compilation and what not
gulp.task('compress', function() {
return gulp.src('./app/js/*.js')
.pipe(uglify())
.pipe(gulp.dest('./dist/js'));
});
gulp.task('js-watch', ['compress'], reload);
//Serve and watch the scss/jade files for changes
gulp.task('default', ['sass', 'templates', 'compress'], function () {
browserSync({server: './dist'});
gulp.watch('./app/**/*.jade', ['jade-watch']);
gulp.watch('./app/scss/*.scss', ['sass']);
gulp.watch('./app/js/*.js', ['js-watch']);
});
I know it is quite a bit to parse through. I am hoping it is a standard something, that won't take too long. If you are interested in seeing the entire file structure, it can be seen at my github here https://github.com/CharlieGreenman/Gulp-with-foundation-and-sass
Thank you, and any help would be more than appreciated!

I wrote a Gulp plugin that simplifies your includes by allowing you to add some arbitrary paths to resolve includes and extends to, so you don't have to worry so much about relative pathing. Take a look: https://github.com/tomlagier/gulp-jade-modules

Turns out it was really simple. There were one thing I was doing wrong
I was using includes ../includes/head instead of include ../includes/head (using includes actually worked for me in grunt, upon further research I saw I was using it wrong for gulp.).

Concat scripts in order with Gulp

Say, for example, you are building a project on Backbone or whatever and you need to load scripts in a certain order, e.g. underscore.js needs to be loaded before backbone.js.
How do I get it to concat the scripts so that they’re in order?
// JS concat, strip debugging and minify
gulp.task('scripts', function() {
gulp.src(['./source/js/*.js', './source/js/**/*.js'])
.pipe(concat('script.js'))
.pipe(stripDebug())
.pipe(uglify())
.pipe(gulp.dest('./build/js/'));
});
I have the right order of scripts in my source/index.html, but since files are organized by alphabetic order, gulp will concat underscore.js after backbone.js, and the order of the scripts in my source/index.html does not matter, it looks at the files in the directory.
So does anyone have an idea on this?
Best idea I have is to rename the vendor scripts with 1, 2, 3 to give them the proper order, but I am not sure if I like this.
As I learned more I found Browserify is a great solution, it can be a pain at first but it’s great.

I had a similar problem recently with Grunt when building my AngularJS app. Here's a question I posted.
What I ended up doing is to explicitly list the files in order in the grunt config. The config file will then look like this:
[
'/path/to/app.js',
'/path/to/mymodule/mymodule.js',
'/path/to/mymodule/mymodule/*.js'
]
Grunt is able to figure out which files are duplicates and not include them. The same technique will work with Gulp as well.

Another thing that helps if you need some files to come after a blob of files, is to exclude specific files from your glob, like so:
[
'/src/**/!(foobar)*.js', // all files that end in .js EXCEPT foobar*.js
'/src/js/foobar.js',
]
You can combine this with specifying files that need to come first as explained in Chad Johnson's answer.

I have used the gulp-order plugin but it is not always successful as you can see by my stack overflow post gulp-order node module with merged streams. When browsing through the Gulp docs I came across the streamque module which has worked quite well for specifying order of in my case concatenation. https://github.com/gulpjs/gulp/blob/master/docs/recipes/using-multiple-sources-in-one-task.md
Example of how I used it is below
var gulp = require('gulp');
var concat = require('gulp-concat');
var handleErrors = require('../util/handleErrors');
var streamqueue = require('streamqueue');
gulp.task('scripts', function() {
return streamqueue({ objectMode: true },
gulp.src('./public/angular/config/*.js'),
gulp.src('./public/angular/services/**/*.js'),
gulp.src('./public/angular/modules/**/*.js'),
gulp.src('./public/angular/primitives/**/*.js'),
gulp.src('./public/js/**/*.js')
)
.pipe(concat('app.js'))
.pipe(gulp.dest('./public/build/js'))
.on('error', handleErrors);
});

With gulp-useref you can concatenate every script declared in your index file, in the order in which you declare it.
https://www.npmjs.com/package/gulp-useref
var $ = require('gulp-load-plugins')();
gulp.task('jsbuild', function () {
var assets = $.useref.assets({searchPath: '{.tmp,app}'});
return gulp.src('app/**/*.html')
.pipe(assets)
.pipe($.if('*.js', $.uglify({preserveComments: 'some'})))
.pipe(gulp.dest('dist'))
.pipe($.size({title: 'html'}));
});
And in the HTML you have to declare the name of the build file you want to generate, like this:
<!-- build:js js/main.min.js -->
<script src="js/vendor/vendor.js"></script>
<script src="js/modules/test.js"></script>
<script src="js/main.js"></script>
In your build directory you will have the reference to main.min.js which will contain vendor.js, test.js, and main.js

The sort-stream may also be used to ensure specific order of files with gulp.src. Sample code that puts the backbone.js always as the last file to process:
var gulp = require('gulp');
var sort = require('sort-stream');
gulp.task('scripts', function() {
gulp.src(['./source/js/*.js', './source/js/**/*.js'])
.pipe(sort(function(a, b){
aScore = a.path.match(/backbone.js$/) ? 1 : 0;
bScore = b.path.match(/backbone.js$/) ? 1 : 0;
return aScore - bScore;
}))
.pipe(concat('script.js'))
.pipe(stripDebug())
.pipe(uglify())
.pipe(gulp.dest('./build/js/'));
});

I just add numbers to the beginning of file name:
0_normalize.scss
1_tikitaka.scss
main.scss
It works in gulp without any problems.

I have my scripts organized in different folders for each package I pull in from bower, plus my own script for my app. Since you are going to list the order of these scripts somewhere, why not just list them in your gulp file? For new developers on your project, it's nice that all your script end-points are listed here. You can do this with gulp-add-src:
gulpfile.js
var gulp = require('gulp'),
less = require('gulp-less'),
minifyCSS = require('gulp-minify-css'),
uglify = require('gulp-uglify'),
concat = require('gulp-concat'),
addsrc = require('gulp-add-src'),
sourcemaps = require('gulp-sourcemaps');
// CSS & Less
gulp.task('css', function(){
gulp.src('less/all.less')
.pipe(sourcemaps.init())
.pipe(less())
.pipe(minifyCSS())
.pipe(sourcemaps.write('source-maps'))
.pipe(gulp.dest('public/css'));
});
// JS
gulp.task('js', function() {
gulp.src('resources/assets/bower/jquery/dist/jquery.js')
.pipe(addsrc.append('resources/assets/bower/bootstrap/dist/js/bootstrap.js'))
.pipe(addsrc.append('resources/assets/bower/blahblah/dist/js/blah.js'))
.pipe(addsrc.append('resources/assets/js/my-script.js'))
.pipe(sourcemaps.init())
.pipe(concat('all.js'))
.pipe(uglify())
.pipe(sourcemaps.write('source-maps'))
.pipe(gulp.dest('public/js'));
});
gulp.task('default',['css','js']);
Note: jQuery and Bootstrap added for demonstration purposes of order. Probably better to use CDNs for those since they are so widely used and browsers could have them cached from other sites already.

Try stream-series. It works like merge-stream/event-stream.merge() except that instead of interleaving, it appends to the end. It doesn't require you to specify the object mode like streamqueue, so your code comes out cleaner.
var series = require('stream-series');
gulp.task('minifyInOrder', function() {
return series(gulp.src('vendor/*'),gulp.src('extra'),gulp.src('house/*'))
.pipe(concat('a.js'))
.pipe(uglify())
.pipe(gulp.dest('dest'))
});

merge2 looks like the only working and maintained ordered stream merging tool at the moment.
Update 2020
The APIs are always changing, some libraries become unusable or contain vulnerabilities, or their dependencies contain vulnerabilities, that are not fixed for years. For text files manipulations you'd better use custom NodeJS scripts and popular libraries like globby and fs-extra along with other libraries without Gulp, Grunt, etc wrappers.
import globby from 'globby';
import fs from 'fs-extra';
async function bundleScripts() {
const rootPaths = await globby('./source/js/*.js');
const otherPaths = (await globby('./source/**/*.js'))
.filter(f => !rootFiles.includes(f));
const paths = rootPaths.concat(otherPaths);
const files = Promise.all(
paths.map(
// Returns a Promise
path => fs.readFile(path, {encoding: 'utf8'})
)
);
let bundle = files.join('\n');
bundle = uglify(bundle);
bundle = whatever(bundle);
bundle = bundle.replace(/\/\*.*?\*\//g, '');
await fs.outputFile('./build/js/script.js', bundle, {encoding: 'utf8'});
}
bundleScripts.then(() => console.log('done');

An alternative method is to use a Gulp plugin created specifically for this problem. https://www.npmjs.com/package/gulp-ng-module-sort
It allows you to sort your scripts by adding in a .pipe(ngModuleSort()) as such:
var ngModuleSort = require('gulp-ng-module-sort');
var concat = require('gulp-concat');
gulp.task('angular-scripts', function() {
return gulp.src('./src/app/**/*.js')
.pipe(ngModuleSort())
.pipe(concat('angularAppScripts.js))
.pipe(gulp.dest('./dist/));
});
Assuming a directory convention of:
|——— src/
| |——— app/
| |——— module1/
| |——— sub-module1/
| |——— sub-module1.js
| |——— module1.js
| |——— module2/
| |——— sub-module2/
| |——— sub-module2.js
| |——— sub-module3/
| |——— sub-module3.js
| |——— module2.js
| |——— app.js
Hope this helps!

For me I had natualSort() and angularFileSort() in pipe which was reordering the files. I removed it and now it works fine for me
$.inject( // app/**/*.js files
gulp.src(paths.jsFiles)
.pipe($.plumber()), // use plumber so watch can start despite js errors
//.pipe($.naturalSort())
//.pipe($.angularFilesort()),
{relative: true}))

I just use gulp-angular-filesort
function concatOrder() {
return gulp.src('./build/src/app/**/*.js')
.pipe(sort())
.pipe(plug.concat('concat.js'))
.pipe(gulp.dest('./output/'));
}

I'm in a module environnement where all are core-dependents using gulp.
So, the core module needs to be appended before the others.
What I did:
Move all the scripts to an src folder
Just gulp-rename your core directory to _core
gulp is keeping the order of your gulp.src, my concat src looks like this:
concat: ['./client/src/js/*.js', './client/src/js/**/*.js', './client/src/js/**/**/*.js']
It'll obviously take the _ as the first directory from the list (natural sort?).
Note (angularjs):
I then use gulp-angular-extender to dynamically add the modules to the core module.
Compiled it looks like this:
angular.module('Core', ["ui.router","mm.foundation",(...),"Admin","Products"])
Where Admin and Products are two modules.

if you would like to order third party libraries dependencies, try wiredep. This package basically checks each package dependency in bower.json then wire them up for you.

I tried several solutions from this page, but none worked. I had a series of numbered files which I simply wanted be ordered by alphabetical foldername so when piped to concat() they'd be in the same order. That is, preserve the order of the globbing input. Easy, right?
Here's my specific proof-of-concept code (print is just to see the order printed to the cli):
var order = require('gulp-order');
var gulp = require('gulp');
var print = require('gulp-print').default;
var options = {};
options.rootPath = {
inputDir: process.env.INIT_CWD + '/Draft',
inputGlob: '/**/*.md',
};
gulp.task('default', function(){
gulp.src(options.rootPath.inputDir + options.rootPath.inputGlob, {base: '.'})
.pipe(order([options.rootPath.inputDir + options.rootPath.inputGlob]))
.pipe(print());
});
The reason for the madness of gulp.src? I determined that gulp.src was running async when I was able to use a sleep() function (using a .map with sleeptime incremented by index) to order the stream output properly.
The upshot of the async of src mean dirs with more files in it came after dirs with fewer files, because they took longer to process.

In my gulp setup, I'm specifying the vendor files first and then specifying the (more general) everything, second. And it successfully puts the vendor js before the other custom stuff.
gulp.src([
// vendor folder first
path.join(folder, '/vendor/**/*.js'),
// custom js after vendor
path.join(folder, '/**/*.js')
])

Apparently you can pass in the "nosort" option to gulp.src gulp.src.

Make browserify modules external with Gulp

I have a library lib.js that I want to create from lib/a.js and lib/b.js and to be able to use it from a script client.js using var a = require('lib/a.js'); and that it works when I just include the compiled lib.js library before client.js (therefore, lib.js has to declare a require function that knows about lib/a.js)
I guess I have to use external and alias but I am not sure what is the proper way to do it
Also, is it possible to have a Gulp file that creates all the alias automatically for the folders in my library? eg. creates an alias for all the files in the lib/ dir?

Here are a couple of gulp tasks that would help to build your common lib.js and the client.js bundles separately.
Note that you have to tell browserify to b.require() lib/*.js when bundling lib.js, and you have to tell it to b.external() the libraries that will be loaded separately when bundling client.js
var path = require('path');
var gulp = require('gulp');
var browserify = require('browserify');
var concat = require('gulp-concat');
var transform = require('vinyl-transform');
gulp.task('build-lib', function () {
// use `vinyl-transform` to wrap around the regular ReadableStream returned by b.bundle();
// so that we can use it down a vinyl pipeline as a vinyl file object.
// `vinyl-transform` takes care of creating both streaming and buffered vinyl file objects.
var browserified = transform(function(filename) {
// basename, for eg: 'a.js'
var basename = path.basename(filename);
// define the exposed name that your client.js would use to require();
// for eg: require('lib/a.js'); // -> exposed name should be 'lib/a.js'
var expose = 'lib/' + basename;
return browserify(filename)
.require(filename, { expose: expose})
.bundle();
});
return gulp.src(['./lib/*.js'])
.pipe(browserified)
.pipe(concat('lib.js'))
.pipe(gulp.dest('./dist'));
});
gulp.task('build-client', function () {
var browserified = transform(function(filename) {
// filename = './client.js'
// let browserify know that lib/a.js and and lib/b.js are external files
// and will be loaded externally (in your case, by loading the bundled lib.js
// for eg: <script src='dist/lib.js'>)
return browserify(filename)
.external('lib/a.js')
.external('lib/b.js')
.bundle();
});
return gulp.src(['./client.js'])
.pipe(browserified)
.pipe(gulp.dest('./dist'));
});
gulp.task('default', ['build-lib', 'build-client']);

Are you looking for external requires?
To use with gulp-browserify, check the README
.on('prebundle', function(bundle) {
bundle.external('domready');
bundle.external('react');
})
Should work with bundle.require as well.

Categories

Resources