In my application, I am trying to read a file and then write the contents of the file to another file.
I have used bluebird promises to do this. I need help to confirm that my understanding on using promises is right in my implementation.
The questions that I have is,
In my example, I am first reading the file, then once the file read the file, I am writing the contents into another file in my next '.then' block. Once the contents are written into a file I need log a message. I have included that in the second '.then' block. I need to know whether my understanding is correct on promises. will the second '.then' block works as the callback function for writefile statement?
I need to write more meaningful log messages. Different error messages if an error occurs while reading the file and writing the file. How can I do this with catch blocks?
Thanks in advance!
The code example is below.
var Promise = require("bluebird");
var fs = Promise.promisifyAll(require("fs"));
var logger = require("./logger.js")
var projectJSON = require("../project.json");
var harPath = projectJSON.project.harfile.location;
var harFileNames = projectJSON.project.transactions.transactionsName;
var harFilePath;
var harFiles = {};
var writeFilePath = "../temp/"
harFileNames.forEach(function(harFileName){
harFilePath = harPath + "/" + harFileName + ".har";
fs.readFileAsync(harFilePath, "utf-8")
.then(function(data){
fs.writeFile(writeFilePath + harFileName + ".json", data);
test = data;
})
.then(function(){
console.log("data written successfully: ");
})
.catch(function(err){
logger.error("error", "Error reading har files from location!");
});
});
If you want to capture each error separately, then you can put an error handler immediately after each operation so you can directly capture that particular error.
Then, to propagate the error, you can rethrow the error value.
Then, you want to switch to fs.writeFileAsync() so everything is async and is using promises.
Then, you need to return the promise from fs.writeFileAsync().
var Promise = require("bluebird");
var fs = Promise.promisifyAll(require("fs"));
var logger = require("./logger.js")
var projectJSON = require("../project.json");
var harPath = projectJSON.project.harfile.location;
var harFileNames = projectJSON.project.transactions.transactionsName;
var harFilePath;
var harFiles = {};
var writeFilePath = "../temp/"
harFileNames.forEach(function(harFileName){
harFilePath = harPath + "/" + harFileName + ".har";
fs.readFileAsync(harFilePath, "utf-8")
.then(function(data){
harFiles[JSON.parse(data).log.pages[0].id] = JSON.parse(data);
return data;
}).catch(err) {
logger.error("error", "Error reading har files from location!");
throw err; // propagate error
}).then(function(data){
return fs.writeFile(writeFilePath + harFileName + ".json", data).catch(function(err) {
logger.error("error", "Error writing to harFile!");
throw err; // propagate error
});
}).then(function(){
console.log("data written successfully: ");
}).catch(function(err){
// either one of the errors
});
});
Keep in mind that when you have a .catch() handler, the error is considered "handled" and the resulting promise becomes fulfilled, not rejected. So, if you want to capture the error in a particular spot (so you know exactly where it came from), but you want the resulting promise to remain rejected, then you can either return a rejected promise or rethrow the same error.
Per your additional question about how to return harFiles, you will need a surrounding promise that gets resolved with harFiles when everything is done. Bluebird's Promise.map() is useful for that as it will both do the iteration for you and return a master promise. Here's how that part of the code could look:
function someFunc() {
var harPath = projectJSON.project.harfile.location;
var harFileNames = projectJSON.project.transactions.transactionsName;
var harFilePath;
var harFiles = {};
var writeFilePath = "../temp/"
return Promise.map(harFileNames, function(harFileName) {
harFilePath = harPath + "/" + harFileName + ".har";
return fs.readFileAsync(harFilePath, "utf-8")
.then(function(data){
harFiles[JSON.parse(data).log.pages[0].id] = JSON.parse(data);
return data;
}, function(err) {
logger.error("error", "Error reading har files from location!");
throw err; // propagate error
}).then(function(data){
return fs.writeFile(writeFilePath + harFileName + ".json", data).catch(function(err) {
logger.error("error", "Error writing to harFile!");
throw err; // propagate error
});
}).then(function(){
console.log("data written successfully: ");
}).catch(function(err){
// either one of the errors
});
}).then(function() {
// all results are in harFiles array here
return harFiles;
});
}
someFunc().then(function(hFiles) {
// hFiles should be your files object
});
Related
I have a service below. I will call this service every time when I open a model and when I close the model and then open another one the previous values are getting reflected and in this case I want to cancel the promise every time I close the model.
I have tried the following code,
Model closing.js
$scope.closeButton = function() {
DetailDataSvc.storeDefer().resolve()
}
My Service, (DetailDataSvc)
self.storeDefer = function() {
return self.deferReturn;
};
self.getDetailReportData = function(postData, functionName) {
var promises = {};
var d = $q.defer(),
metricDataType;
self.deferReturn = $q.defer();
promises = {
detailReport: metricDataType,
recommendedMetrics: DataSvc.getData(_logPrefix + functionName, recommendedMetricUrl),
metricInfo: DataSvc.getData(_logPrefix + functionName, metricInfoUrl)
};
$q.all(promises).then(function(res) {
$log.debug(_logPrefix + 'getDetailReportData(). Called from %s. $q.all Response (raw): ', functionName, res);
else {
if (response && !_.isEmpty(_.get(response, 'largeCard.chartData.dataValues.rows')) && response.overlayEnabled) {
self.getMetricOverLay(pdata, functionName).then(function(overlayData) {
response.largeCard.chartData.overlay = overlayData;
d.resolve(response);
}, function(msg, code) {
d.reject(msg);
$log.error(_logPrefix + 'getDetailReportData(). Error code: %s. Error: ', code, msg);
});
} else {
d.resolve(response);
}
}
}, function(msg, code) {
d.reject(msg);
$log.error(_logPrefix + 'getDetailReportData(). Error code: %s. Error: ', code, msg);
});
return d.promise;
};
Can anyone please help me whether the process I followed is the right one.
What you have attempted could be made to work but it's best fixed by racing the promise returned by $q.all() against a rejectable Deferred (ie. a Deferred, of which a reference is kept to its reject method), thus avoiding the deferred anti-pattern.
self.getDetailReportData = function(postData, functionName) {
var metricDataType = ......; // ???
var d = $q.defer();
// cancel previous
if(self.cancelDetailReport) {
self.cancelDetailReport(new Error('previous getDetailReportData() cancelled'));
}
// keep a reference to the deferred's reject method for next time round.
self.cancelDetailReport = d.reject;
var promises = {
'detailReport': metricDataType,
'recommendedMetrics': DataSvc.getData(_logPrefix + functionName, recommendedMetricUrl),
'metricInfo': DataSvc.getData(_logPrefix + functionName, metricInfoUrl)
};
// Race aggregated `promises` against `d.promise`, thus providing the required cancellation effect.
return $q.race([$q.all(promises), d.promise])
.then(function(response) {
// arrive here only if all promises resolve and d.reject() has not been called.
$log.debug(_logPrefix + 'getDetailReportData(). Called from %s. $q.all Response (raw): ', functionName, response);
if (response && !_.isEmpty(_.get(response, 'largeCard.chartData.dataValues.rows')) && response.overlayEnabled) {
return self.getMetricOverLay(pdata, functionName)
.then(function(overlayData) {
response.largeCard.chartData.overlay = overlayData;
return response;
});
} else {
return response;
}
})
.catch(function(msg, code) { // signature?
// all error cases including cancellation end up here.
var message = _logPrefix + `getDetailReportData(). Error: (${code}): ${msg}`; // or similar
$log.error(message);
throw new Error(message); // see https://stackoverflow.com/a/42250798/3478010
});
};
Notes:
$q.race() is transparent to whichever promise wins the race, and opaque to the other. So, if the d is rejected before the promise returned by $q.all() settles, then d will win out; response handling will not happen and d's rejection will fall through to the .catch() clause. Alternatively, if the promise returned by $q.all(promises) wins out then flow will follow that promise's success path (ie response handling) or possibly its error path (which will drop through to the .catch() clause).
Not too sure about the signature of the .catch() callback. You would normally expect it to accept a single error argument.
Assign already created deferred.
Try and change this line:
self.deferReturn = $q.defer();
self.deferReturn = d;
I'm currently trying to code a discord bot using node.js. Currently, I have a function that returns a value which a variable is set to. Afterwards, the console prints out the variable. However, the variable is printed out on the console before it gets assigned the value from the function.
The code:
if(commands[0] === "getlevel"){
let region = commands[1] + "1";
let username = commands[2];
var summonerLevel;
console.log(region, username);
API.setRegion(region);
summonerLevel = await API.getSummonerLevel(username);
console.log("HI");
console.log(summonerLevel);
message.channel.send(summonerLevel);
}
});
So what happens is that the console prints about summonerLevel before the API.getSummonerLevel(username) function executes. How would I make it so that the getSummonerLevel() function runs before console.log()?
Code for function:
RiotAPI.prototype.getSummonerLevel = function(_username){
var summonerJSON; var summoner;
var url = baseURL1 + region + baseURL2 + "/lol/summoner/v3/summoners/by-name/" + _username + "?api_key=" + API_KEY;
console.log("Before");
var options = {
uri: url,
simple: false
};
request.get(options)
.then(function(body){
console.log("After");
summonerJSON = body;
summoner = JSON.parse(summonerJSON);
console.log(summoner);
console.log(summoner.summonerLevel);
return summoner.summonerLevel;
})
.catch(function(err){
//should.throw.error.to.console();
})
.catch(function(err){
console.log(err);
});
}
Also, another problem I have is I get an unhandled promise rejection warning and the .catch() that comes after request.get(...) doesn't seem to solve it. How would I fix that was well?
Console log (not sure if this is useful but...):
Before
HI
undefined
(node:13624) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 2): DiscordAPIError: Cannot send an empty message
(node:13624) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
After
{ id: 45932456,
accountId: 206692908,
name: 'fire263',
profileIconId: 911,
revisionDate: 1511729881000,
summonerLevel: 36 }
36
You have some issues on your code.
Code for function:
RiotAPI.prototype.getSummonerLevel = function(_username){
var summonerJSON; var summoner;
var url = baseURL1 + region + baseURL2 + "/lol/summoner/v3/summoners/by-name/" + _username + "?api_key=" + API_KEY;
console.log("Before");
var options = {
uri: url,
simple: false
};
return request.get(options) // you need to return the promise
.then(function(body){
console.log("After");
summonerJSON = body;
summoner = JSON.parse(summonerJSON);
console.log(summoner);
console.log(summoner.summonerLevel);
return summoner.summonerLevel;
})
.catch(function(err){ // you should only use `.catch` once
console.log(err);
});
}
As jaromanda x mentioned, you need to return the request.get call which in itself is a promise.
But also you shouldn't use multiple .catch calls because if you do it and you do not handle it correctly (read: continue the rejection chain) it won't go to the second .catch call that you had.
You can read more here: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise/catch
More specifically this part: "The Promise returned by catch() is rejected if onRejected throws an error or returns a Promise which is itself rejected; otherwise, it is resolved."
Hopefully this will give you enough information to progress on your issue.
Your getSummonerLevel does not return a promise, which is necessary to be awaitable. The easiest way you can achieve that is to make it async as well:
RiotAPI.prototype.getSummonerLevel = async function(_username) {
var url = baseURL1 + region + baseURL2 + "/lol/summoner/v3/summoners/by-name/" + _username + "?api_key=" + API_KEY;
console.log("Before");
var options = {
uri: url,
simple: false
};
try {
var body = await request.get(options);
console.log("After");
var summonerJSON = body;
var summoner = JSON.parse(summonerJSON);
console.log(summoner);
console.log(summoner.summonerLevel);
return summoner.summonerLevel;
} catch (err) {
//should.throw.error.to.console();
console.log(err);
}
};
Follow-up to Swap order of arguments to "then" with Bluebird / NodeJS Promises (the posted answer worked, but immediately revealed a new issue)
This is the first time I've ever used promises in NodeJS so I apologize if some conventions are poorly adhered to or the code is sloppy. I'm trying to aggregate data from multiple APIs, put it in a database, then compute some statistics based on similarities and differences in the data. As a starting point I'm trying to get an API token for a single one of the APIs.
Here is my full code:
var Promise = require('bluebird');
var fs = require('fs');
var request = require('request');
Promise.promisifyAll(fs);
Promise.promisifyAll(request);
// tilde-expansion doesn't follow the callback(err, data) convention
var tilde = function(str) {
var _tilde = require('tilde-expansion');
return new Promise(function(resolve, reject) {
try {
_tilde(str, resolve);
} catch(e) {
reject(e);
}
});
}
var getToken = function() {
return request.getAsync(process.env.token_url, {
headers: {
"Content-Type": "applications/x-www-form-urlencoded"
},
form: {
client_id: process.env.client_id,
client_secret: process.env.client_secret,
grant_type: "client_credentials"
}
})
.then(function(resp) { return resp.body; });
}
var tokenFile = tilde(process.env.token_file)
.catch(function(err) {
console.log("Error parsing path to file... can not recover");
});
var token = tokenFile
.then(fs.readFileAsync) //, "utf8")
.then(function(data) {
console.log("Token (from file): " + data);
return data;
})
.then(JSON.parse)
.catch(function(err) {
console.log("Error reading token from file... getting a new one");
return getToken()
.then(function(data) {
console.log("Token (from API): " + data);
return data;
})
.then(JSON.stringify)
.then(fs.writeFileAsync.bind(null, tokenFile.value()));
});
token.then(function(data) {
console.log("Token (from anywhere): " + token.value);
});
This code is currently logging:
Token: undefined
if I fall back to the API. Assuming I did my promise stuff correctly (.catch() can return a promise, right?) then I would assume the issue is occurring because fs.writeFileAsync returns void.
I would like to append a .return() on the end of this promise, but how would I gain access to the return value of getToken()? I tried the following:
.catch(function(err) {
console.log("Error reading token from file... getting a new one");
var token = "nope";
return getToken()
.then(function(data) {
console.log("Token (from API): " + data);
token = data;
return data;
})
.then(JSON.stringify)
.then(fs.writeFileAsync.bind(null, tokenFile.value()))
.return(token);
});
However this logs "nope".
Over the weekend I continued my research on promises and upon making a pivotal realization I was able to develop the solution to this. Posting here both the realization and the solution:
The Realization
Promises were invented so that asynchronous code could be used in a synchronous manner. Consider the following:
var data = processData(JSON.parse(readFile(getFileName())));
This is the equivalent of:
var filename = getFileName();
var fileData = readFile(filename);
var parsedData = JSON.parse(fileData);
var data = processData(parsedData);
If any one of these functions is asynchronous then it breaks, because the value isn't ready on time. So for those asynchronous bits we used to use callbacks:
var filename = getFileName();
var data = null;
readFile(filename, function(fileData){
data = processData(JSON.parse(fileData));
});
This is not only ugly, but breaks a lot of things like stack traces, try/catch blocks, etc.
The Promise pattern fixed this, letting you say:
var filename = getFileName();
var fileData = filename.then(readFile);
var parsedData = fileData.then(JSON.parse);
var data = parsedData.then(processData);
This code works regardless of whether these functions are synchronous or asynchronous, and there are zero callbacks. It's actually all synchronous code, but instead of passing values around, we pass promises around.
The led me to the realization that: for every bit of code that can be written with promises, there is a synchronous corollary
The solution
Realizing this, I tried to consider my code if all of the functions were synchronous:
try {
var tokenFile = tilde(process.env.token_file)
} catch(err) {
throw new Error("Error parsing path to file... can not recover");
}
var token = null;
try {
token = JSON.parse(readFile(tokenFile));
} catch(err) {
token = getToken();
writeFile(tokenFile, JSON.stringify(token));
}
console.log("Token: " + token.value);
After framing it like this, the promise version follows logically:
var tokenFile = tilde(process.env.token_file)
.catch(function(err) {
throw new Error("Error parsing path to file... can not recover");
});
var token = tokenFile
.then(readFile)
.then(JSON.parse)
.catch(function(err) {
var _token = getToken();
_token
.then(JSON.stringify)
.then(writeFile.bind(null, tokenFile.value));
return _token;
});
experienced programmer, inexperienced javascript programmer.
I am working with Node-RED and in one of my nodes, I am calling an external program with 'child_process' which sends data back to my node via stdout.
I am using console.info(stdout) to log the child process data, so I know that it is working appropriately. This is the code within the function node:
/* required modules */
var cp = context.global.child_process;
/* variables */
var packet = {};
var cmd = '~/packet/packet.py ';
/* construct the packet object */
packet.networkTime = (msg.payload[3] << 24) + (msg.payload[2] << 16) + (msg.payload[1] << 8) + msg.payload[0];
packet.sampleCount = msg.payload[4];
packet.uncompressedWidth = msg.payload[5];
packet.compressedWidth = msg.payload[6];
packet.sample0 = (msg.payload[8] << 8) + msg.payload[7];
var compressedData = msg.payload.slice(9);
/* change buffer type to array */
packet.compressedData = [];
compressedData.forEach(
function(element){
packet.compressedData.push(element);
}
);
/* stringify the object */
var packet_str = "'" + JSON.stringify(packet) + "'";
/* this section will execute the command */
cp.exec(
cmd + packet_str,
function(error, stdout, stderr){
console.info(stdout);
return JSON.parse(stdout);
}
);
//return data_msg;
I tried setting data_msg to {} and using JSON.parse(stdout) to place into that object, but the 'return data_msg' always executes before the value is returned. I simply want to wait for my external process to return data before returning this node.
Thank you for your assistance,
Node.js has asynchronous execution. Put some things in functions and handle sequential execution in callbacks or use promises. Below is my use of promises that I use to ensure a scipt is finished executing before using it's stdout data within it's calling function.
Upon stderr the promise is rejected and an error message is propagated. Upon stdout data is stored in result. When the process finishes the data is returned as the promise is resolved.
var spawn = require('child_process').spawn;
anExecutable = function(cmd, packet_str){
return new Promise(function(resolve, reject){
var aProcess = spawn(cmd, packet_str);
aProcess.stdout.on('data', (data) =>{
var result = JSON.parse(data);
});
aProcess.on('error', (error) => {
reject(error);
});
aProcess.on('exit', (code) => {
resolve(result);
});
});
}
Call like this:
anExecutable(cmd, packet_str).then(function(data_msg){
return data_msg;
}).catch(function(error){
return error;
});
Let me know if you have any Q's :D
Maybe you're looking for execSync?
I'm trying to use promises with nodejs (I'm trying with node-promise package); however, without any success. See the code below:
var express = require('express'),
request = require('request'),
promise = require('node-promise');
app.get('/promise', function(req, res) {
var length = -1;
new promise.Promise(request(
{uri: "http://www.bing.com"},
function (error, response, body) {
if (error && response.statusCode !== 200) {
console.log("An error occurred when connected to the web site");
return;
}
console.log("I'll return: " + body.length);
length = body.length;
}
)).then(function(result) {
console.log("This is what I got: " + length);
console.log("Done!");
});
res.end();
});
The output of the above code is I'll return: 35857 only and it doesn't go to the then part.
I change the code then to be:
app.get('/promise', function(req, res) {
var length = -1;
promise.when(
request(
{uri: "http://www.bing.com"},
function (error, response, body) {
if (error && response.statusCode !== 200) {
console.log("An error occurred when connected to the web site");
return;
}
console.log("I'll return: " + body.length);
length = body.length;
}
),
function(result) {
console.log("This is what I got: " + length);
console.log("Done!");
},
function(error) {
console.log(error);
}
);
res.end();
});
This time the output is This is what I got: -1 then Done!... looks like the "promise" was not called this time.
So:
What's needed to be done to fix the code above? Obviously I'm not doing it right :)
Is node-promise "the way to go" when I'm doing promises, or is there a better way/package? i.e. simpler and more production-ready.
Thanks.
Try jquery-deferred-for-node.
I'm not an expert but understand that this lib tends to be favoured by programmers who work both server-side and client-side.
Even if you don't already know jQuery's Deferreds, the advantages of going this route are that :
the documentation is excellent (it comprises links to the jQuery docs), though you may struggle to find examples specific to Node.
methods are chainable.
jQuery Callbacks are also included.
when one day you need to do asynchronous stuff client-side, then there's virtually nothing to relearn - the concepts are identical and the syntax very nearly so. See the "Correspondances" section in the github page hyperlinked above.
EDIT
I'm not a node.js person so I'm guessing here but based on your code above, you might want to consider something along the following lines with jquery-deferred-for-node :
var express = require('express'),
request = require('request'),
Deferred = require('JQDeferred');
function fetch(uri, goodCodes) {
goodCodes = (!goodCodes) ? [200] : goodCodes;
var dfrd = Deferred(); // A Deferred to be resolved/rejected in response to the `request()`.
request(uri, function(error, response, body) {
if (!error) {
var isGood = false;
// Loop to test response.statusCode against `goodCodes`.
for (var i = 0; i < goodCodes.length; i++) {
if (response.statusCode == goodCodes[i]) {
isGood = true;
break;
}
}
if (isGood) {
dfrd.resolve(response.statusCode, body);
} else {
dfrd.reject(response.statusCode, "An invalid response was received from " + uri);
}
} else {
dfrd.reject(response.statusCode, "An error occurred attempting to connect to " + uri);
}
});
// Make promise derived from dfrd available to "consumer".
return dfrd.promise();
};
//...
app.get('/promise', function(req, resp) {
fetch("http://www.bing.com").done(function(statusCode, result) {
console.log("Done! This is what I got: " + result.length);
}).fail(function(statusCode, message) {
console.log("Error (" + statusCode + "): " + message);
});
resp.end();
};
Here, I have tried to write a generalized utility for fetching a resource in such a way that the asynchronous response (or error) can be handled externally. I think this is broadly along the lines of what you were trying to achieve.
Out of interest, where do console.log() messages end up with node.js?
EDIT 2
Above, I have given Deferred an initial capital, as is conventional for Constructors
With jQuery Deferreds, there must be any number of ways to fetch() consecutively. The approach below leaves fetch() as it was, and introduces fetch_() to act as its front-end. There may be simpler ways but this allows fetch() to remain a general utility, functionally equivalent to the client-side jQuery.ajax().
function fetch_(uri){
return function(){
return fetch(uri, [200]).then(function(statusCode, result){
console.log("Done! This is what I got: " + result.length);
},function(statusCode, message){
console.log("Error (" + statusCode + "): " + message);
});
};
}
Note that function fetch() returns a function. It has to be like this because where fetch() is called, we want an unexecuted function, not (yet) the result of that function.
Now let's assume an array of uris is available. This can be hard-coded or built dynamically - whatever the application demands.
var uris = [
'http://xxx.example.com',
'http://yyy.example.com',
'http://zzz.example.com'
];
And now, a variety of ways in which fetch_() might be called :
//v1. To call `resp.end()` when the fetching process starts.
app.get('/promise', function(req, resp) {
fetch_(uris[0])().then(fetch_(uris[1])).then(fetch_(uris[2]));
resp.end();
});
//v2. To call `resp.end()` when the fetching process has finished.
app.get('/promise', function(req, resp){
fetch_(uris[0])().then(fetch_(uris[1])).then(fetch_(uris[2])).always(resp.end);
});
//v3. As v2 but building a `.then()` chain of any (unknown) length.
app.get('/promise', function(req, resp){
var dfrd = Deferred().resolve();//
$.each(uris, function(i, uri){
dfrd = dfrd.then(fetch_(uri));
});
dfrd = dfrd.always(resp.end);
});
untested
I have more confidence in v1 and v2. v3 may work.
v2 and v3 should both give exactly the same behaviour but v3 is generalized for any number of uris.
Everything may need debugging.
I would recommend using Q: https://github.com/kriskowal/q. I believe that it's used internally by other frameworks (like jQuery deferred implementation).
I believe that the documentation is "fine"; the syntax is consistent with other promise implementations... and it has a node adapter.
So your deferred style approach:
var deferred = Q.defer();
FS.readFile("foo.txt", "utf-8", function (err, res) {
if (!err) {
deferred.resolve(res);
} else {
deferred.reject(err);
}
});
return deferred.promise;
Can be written more concisely as:
var deferred = Q.defer();
FS.readFile("foo.txt", "utf-8", deferred.makeNodeResolver());
return deferred.promise;