I have 3 Node Js functions. What I'm trying to do here is, I want to call normalizeFilePath and get the normalized path, after that check whether a file exist or not with that normalizedFilePath and after all these, create a file if the file doesn't already exist. This is the first day of using promises (Bluebird) and I'm new to Node JS and Java Script. Below code structure is getting complex. Of course this is not a good idea at all.
var createProjectFolder = function (projectName) {
};
var checkFileExistance = function (filePath) {
return new promise(function (resolve, reject) {
normalizeFilePath(filePath).then(function (normalizedFilePath) {
return fs.existSync(normalizedFilePath);
});
})
};
var normalizeFilePath = function (filePath) {
return new promise(function (resolve, reject) {
resolve(path.normalize(filePath));
});
};
How can i manage promises to implement that concept?
Let's improve your code in two simple steps.
Promises are meant for async functions
As long as path.normalize is synchronous, it should not be wrapped in promise.
So it can be as simple as that.
var normalizeFilePath = function (filePath) {
return path.normalize(filePath);
};
But for now lets pretend that path.normalize is async, so we can use your version.
var normalizeFilePath = function (filePath) {
return new Promise(function (resolve, reject) {
resolve( path.normalize(filePath) );
});
};
Promisify all the things
Sync is bad. Sync blocks event loop. So, instead of fs.existsSync we will use fs.exists.
var checkFileExistance = function (filePath) {
return new Promise(function (resolve, reject) {
fs.exists(filePath, function (exists) {
resolve(exists);
});
});
};
As You can see, we are wrapping async function that accepts a callback with a promise. It's quite a common concept to "promisify" a function, so we could use a library for that. Or even use fs-promise, that is -- you guess it -- fs with promises.
Chaining promises
Now, what we want is making three actions one after another:
Normalize file path
Check if file already exists
If not, create a directory
Keeping that in mind, our main function can look like this.
var createProjectFolder = function (projectName) {
normalizeFilePath(projectName)
.then(checkFileExistance)
.then(function (exists) {
if (!exists) {
// create folder
}
})
.catch(function (error) {
// if there are any errors in promise chain
// we can catch them in one place, yay!
});
};
Don't forget to add the catch call so you would not miss any errors.
Related
I wrote the following node.js file:
var csv = require('csv-parser');
var fs = require('fs')
var Promise = require('bluebird');
var filename = "devices.csv";
var devices;
Promise.all(read_csv_file("devices.csv"), read_csv_file("bugs.csv")).then(function(result) {
console.log(result);
});
function read_csv_file(filename) {
return new Promise(function (resolve, reject) {
var result = []
fs.createReadStream(filename)
.pipe(csv())
.on('data', function (data) {
result.push(data)
}).on('end', function () {
resolve(result);
});
})
}
As you can see, I use Promise.all in order to wait for both operations of reading the csv files. I don't understand why but when I run the code the line 'console.log(result)' is not committed.
My second question is I want that the callback function of Promise.all.then() accepts two different variables, while each one of them is the result of the relevant promise.
First question
Promise.all takes an array of promises
Change:
Promise.all(read_csv_file("devices.csv"), read_csv_file("bugs.csv"))
to (add [] around arguments)
Promise.all([read_csv_file("devices.csv"), read_csv_file("bugs.csv")])
// ---------^-------------------------------------------------------^
Second question
The Promise.all resolves with an array of results for each of the promises you passed into it.
This means you can extract the results into variables like:
Promise.all([read_csv_file("devices.csv"), read_csv_file("bugs.csv")])
.then(function(results) {
var first = results[0]; // contents of the first csv file
var second = results[1]; // contents of the second csv file
});
You can use ES6+ destructuring to further simplify the code:
Promise.all([read_csv_file("devices.csv"), read_csv_file("bugs.csv")])
.then(function([first, second]) {
});
Answer to your second question:
If you want the then callback to accept two different arguemnts, then you can use Bluebird and its spread method. See:
http://bluebirdjs.com/docs/api/spread.html
Instead of .then(function (array) { ... }) and having to access array[0] and array[1] inside of your then handler you will be able to use spread(function (value1, value2) { ... }) and have both variables named as you want.
This is a feature of Bluebird, it's not possible with plain Promise.
You use Bluebird just like Promise, e.g.:
var P = require('bluebird');
// and in your code:
return new P(function (resolve, reject) { ...
// instead of:
return new Promise(function (resolve, reject) { ...
Of course you don't have to name it P but whatever you want.
For more examples see the Bluebird Cheatsheets.
Sample code:
const googleLoadPromise = new Promise(function (resolve, reject) {
google.charts.setOnLoadCallback(function () {
resolve(1);
});
});
googleLoadPromise.then(function () {
// consumer 1 - do something
});
googleLoadPromise.then(function () {
// comsumer 2 - do something else
});
i.e. for the googleLoadPromise, there are two consumers. Is this sort of pattern ok? It seems to work ok - i.e. both consumers get called, and they don't seem to cause problems for each other.
Also, if this is ok, is the order of the running the consumers deterministic (just out of interest, more than anything)?
Yes of course.
You can also just do this if you want to run both functions in sequence:
const googleLoadPromise = new Promise(function (resolve, reject) {
google.charts.setOnLoadCallback(function () {
resolve(1);
});
});
googleLoadPromise
.then(function () {
// consumer 1 - do something
}).then(function () {
// comsumer 2 - do something else
});
Just also remember to handle rejections and catch your promise.
Promises become much easier with async await as well. The below example is more inline with what you wrote:
const googleLoadPromise = new Promise(function (resolve, reject) {
google.charts.setOnLoadCallback(function () {
resolve(1);
});
});
function1 = () => {
// consumer 1 - do something
}
function2 = () => {
// consumer 1 - do something
}
(async function() {
try {
const result = await googleLoadPromise();
function1();
function2();
}
catch( error ) {
console.error( error );
}
}());
Is this sort of pattern ok?
Sure, thats the main usecase of promises.
is the order of the running the consumers deterministic (just out of interest, more than anything)?
Yes, the callback attached first with then gets executed first, however I wouldnt rely on it. If you want to run one callback after another, chain them accordingly.
// ok:
promise
.then(first);
promise
.then(second);
// better if second depends on first:
promise
.then(first)
.then(second);
I'm trying to get promises work for the first time. However, my Promise.all is never firing.
I'm using node.js & express
const promises = [
new Promise( () => {
var query = `...`;
mssql.query(query, function(obj){
finalRes['key1'] = obj.recordset;
return true;
//this works
});
}),
new Promise( () => {
var query = `...`;
mssql.query(query, function(obj){
finalRes['key2'] = obj.recordset;
return true;
//this works
});
}),
...
]
Promise.all(promises).then(() => {
res.send(finalRes);
// this is never firing
});
I have been googling stuff and I can't find a solution. I would appreciate someone point out what I do wrong here.
Cheers
Your promise creation code is wrong, as they never resolve. What you actually should do is fire resolve functions inside your callback-based code. I'd go a bit further - and make all those promises resolve with their results instead of modifying some external value. Like this:
const promises = [
new Promise( (resolve, reject) => {
var query = `...`;
mssql.query(query, function(obj){
resolve({key1:obj.recordset});
});
}),
new Promise( (resolve, reject) => {
var query = `...`;
mssql.query(query, function(obj){
resolve({key2:obj.recordset});
});
}) // ...
];
Promise.all(promises).then(results => {
res.send(Object.assign({}, ...results));
});
Depending on how queries are built, you might go even further - and write a generic query-generator function, which takes query and key as params, and returns a promise. Again, this function should be easy to test.
As a sidenote, your code is overly optimistic, it should also provide error callbacks with reject() invoke to each query.
I'm new to node and I'm having an issue with resolving an async Promise. My promise isn't resolving and I'm not sure what I did wrong. I'm still having troubles understanding promises and callbacks so any feedback is helpful.
var filterFiles = function(){
return new Promise(function(resolve, reject){
fs.readdir(rootDir, function(err, files){
if(err) return console.log(err);
var task = function(file){
return new Promise(function(resolve, reject){
if(! /^\..*/.test(file)){
fs.stat(rootDir + '/' + file, function(err, stats){
if(stats.isDirectory()){
dirArray.push(file);
console.log(dirArray.length);
resolve(file);
}
if(stats.isFile()){
fileArray.push(file);
console.log(fileArray.length);
resolve(file);
}
})
}
})
};
var actions = files.map(task);
return Promise.all(actions).then(function(resolve, reject){
resolve({dirArray: dirArray, fileArray: fileArray});
});
})
})
}
filterFiles().then(function(data){
console.log(data);
var obj = {
fileArray: fileArray,
dirArray: dirArray
};
res.send(obj);
})
It see at least three errors:
When you hit this if statement if(! /^\..*/.test(file)){ and it does not execute the if block, then the parent promise is never settled.
There is no error handling on fs.stat() so if you get an error on that call, you are ignoring that and will be attempting to use a bad value.
The error handling on your call to fs.readdir() is incomplete and will leave you with a promise that is never settled (when it should be rejected).
For a robust solution, you really don't want to be mixing promises and callbacks in the same code. It leads to the opportunity for lots of mistakes, particularly with error handling (as you can see you had at least three errors - two of which were in error handling).
If you're going to use Promises, then promisify the async operations you are using at the lowest level and use only promises to control your async code flow. The simplest way I know of to promisify the relevant fs operations is to use the Bluebird promise library with its Promise.promisifyAll(). You don't have to use that library. You could instead manually write promise wrappers for the async operations you're using.
Here's a version of your code using the Bluebird promise library:
const Promise = require('bluebird');
const fs = Promise.promisifyAll(require('fs'));
function filterFiles() {
return fs.readdirAsync(rootDir).then(function(files) {
let fileArray = [];
let dirArray = [];
// filter out entries that start with .
files = files.filter(function(f) {
return !f.startsWith(".");
});
return Promise.map(files, function(f) {
return fs.statAsync(f).then(function(stats) {
if (stats.isDirectory()) {
dirArray.push(f);
} else {
fileArray.push(f);
}
});
}).then(function() {
// make the resolved value be an object with two properties containing the arrays
return {dirArray, fileArray};
});
});
}
filterFiles().then(function(data) {
res.json(data);
}).catch(function(err) {
// put whatever is appropriate here
res.status(500).end();
});
This was rewritten/restructured with these changes:
Use promises for all async operations
Fix all error handling to reject the returned promise
Filter out files starting with a . synchronously before processing any files (simplifies async processing).
Use Promise.map() to process an array of values in parallel.
In the filterFiles().then() handler, handle errors
You can't res.send() a Javascript object so I used res.json(data) instead (though I'm not sure what exactly you really want to send).
Replace regex comparison with more efficient and simpler to understand .startsWith().
If you don't want to use the Bluebird promise library, you can make your own promise wrappers for the fs methods you use like this:
fs.readdirAsync = function(dir) {
return new Promise(function(resolve, reject) {
fs.readdir(dir, function(err, data) {
if (err) {
reject(err);
} else {
resolve(data);
}
});
});
}
fs.statAsync = function(f) {
return new Promise(function(resolve, reject) {
fs.stat(f, function(err, data) {
if (err) {
reject(err);
} else {
resolve(data);
}
});
});
}
function filterFiles() {
return fs.readdirAsync(rootDir).then(function(files) {
let fileArray = [];
let dirArray = [];
// filter out entries that start with .
files = files.filter(function(f) {
return !f.startsWith(".");
});
return Promise.all(files.map(function(f) {
return fs.statAsync(f).then(function(stats) {
if (stats.isDirectory()) {
dirArray.push(f);
} else {
fileArray.push(f);
}
});
})).then(function() {
// make the resolved value be an object with two properties containing the arrays
return {dirArray, fileArray};
});
});
}
filterFiles().then(function(data) {
res.json(data);
}).catch(function(err) {
res.status(500).end();
});
The main issue you are having is that outer-most Promise is not resolved or rejected. You can fix this by resolving your Promise.all instead of returning it.
resolve(
Promise.all(actions)
.then(function(resolvedTasks){
// ... next potential issue is here
return {dirArray: dirArray, fileArray: fileArray}
})
);
(I know, kind of awkward-looking right?)
Next, your return value after the Promise.all resolves is a little weird. In the task function, you're pushing items onto dirArray and fileArray, but they are not declared or assigned in your snippet. I will assume that they are in-scope for this code. In this case, you just need to return your desired object.
Additionally, to make your async code more readable, here are some tips:
try not to mix callbacks with Promises
use a Promise library to promisify any code limited to callbacks. Example: bluebird's promisifyAll
avoid nesting callbacks/promises when possible
Let's say I have a file containing certain promises, that when executed in order, prepares an input file input.txt.
// prepareInput.js
var step1 = function() {
var promise = new Promise(function(resolve, reject) {
...
});
return promise;
};
var step2 = function() {
var promise = new Promise(function(resolve, reject) {
...
});
return promise;
};
var step3 = function() {
var promise = new Promise(function(resolve, reject) {
...
});
return promise;
};
step1().then(step2).then(step3);
exports.fileName = "input.txt";
If I run node prepareInput.js, the line step1().then(step2).then(step3) gets executed and creates the file.
How can I alter this so that when other files attempt to retrieve fileName from this module, step1().then(step2).then(step3); is run and completed before fileName is exposed? Something along the line of:
// prepareInput.js
...
exports.fileName =
step1().then(step2).then(step3).then(function() {
return "input.txt";
});
// main.js
var prepareInput = require("./prepareInput");
var inputFileName = require(prepareInput.fileName);
Node.js beginner here; apologize beforehand if my approach makes completely no sense... :)
You can't directly export the results of something retrieved asynchronously because export is synchronous, so it happens before any async results have been retrieved. The usual solution here is to export a method that returns a promise. The caller can then call that method and use that promise to get the desired async result.
module.exports = function() {
return step1().then(step2).then(step3).then(function() {
// based on results of above three operations,
// return the filename here
return ...;
});
}
The caller then does this:
require('yourmodule')().then(function(filename) {
// use filename here
});
One thing to keep is mind is that if any operation in a sequence of things is asynchronous, then the whole operation becomes asynchronous and no caller can then fetch the result synchronously. Some people refer to asynchronous as "contagious" in this way. So, if any part of your operation is asynchronous, then you must create an asynchronous interface for the eventual result.
You can also cache the promise so it is only ever run once per app:
module.exports = step1().then(step2).then(step3).then(function() {
// based on results of above three operations,
// return the filename here
return ...;
});
The caller then does this:
require('yourmodule').then(function(filename) {
// use filename here
});