Develop Reference

Loop through AWS Lambda Nodejs SDK function

I'm new to Nodejs and having trouble understand this issue: I tried to run a describe function against an array, and the AWS function seems to run after the main function has finished.
Here's the main function: (loop thru a list of ACM ARNs and check the status)
var checkCertStatus = function(resolveObj){
var promise = new Promise(function(resolve, reject){
console.log('1');
var retObj='';
resolveObj.Items.forEach(function(element) {
var certDescribeParams = {
CertificateArn: element.sslCertId
};
console.log('2');
acm.describeCertificate(certDescribeParams, function(err, data) {
if(err) reject(new Error(err));
else {
console.log(data.Certificate.DomainName + ': ' + data.Certificate.Status);
retObj+=data;
}
});
});
console.log('3');
resolve(retObj);
return promise;
})
}
Based on the debug log, assuming there are 2 items need to be processed, what I got:
1
2
2
3
example.com: ISSUED
example2.com: ISSUED
Basically, I need to pass this result to the next function in the chain (with promise and stuff).

Welcome to Node.js! Speaking generally, it might be helpful to study up on the asynchronous programming style. In particular, you seem to be mixing Promises and callbacks, which may make this example more confusing than it needs to be. I suggest using the AWS SDK's built-in feature to convert responses to Promises.
The first thing I notice is that you are manually constructing a Promise with a resolve/reject function. This is often a red flag unless you are creating a library. Most other libraries support Promises which you can simply use and chain. (This includes AWS SDK, as mentioned above.)
The second thing I notice is that your checkCertStatus function does not return anything. It creates a Promise but does not return it at the end. Your return promise; line is actually inside the callback function used to create the Promise.
Personally, when working with Promises, I prefer to use the Bluebird library. It provides more fully-featured Promises than native, including methods such as map. Conveniently, the AWS SDK can be configured to work with an alternative Promise constructor via AWS.config.setPromisesDependency() as documented here.
To simplify your logic, you might try something along these lines (untested code):
const Promise = require('bluebird');
AWS.config.setPromisesDependency(Promise);
const checkCertStatus = (resolveObj) => {
const items = resolveObj.Items;
console.log(`Mapping ${items.length} item(s)`);
return Promise.resolve(items)
.map((item) => {
const certDescribeParams = {
CertificateArn: item.sslCertId,
};
console.log(`Calling describeCertificate for ${item.sslCertId}`);
return acm.describeCertificate(certDescribeParams)
.promise()
.then((data) => {
console.log(`${data.Certificate.DomainName}: ${data.Certificate.Status}`);
return data;
});
});
};
We're defining checkCertStatus as a function which takes in resolveObj and returns a Promise chain starting from resolveObj.Items. (I apologize if you are not yet familiar with Arrow Functions.) The first and only step in this chain is to map the items array to a new array of Promises returned from the acm.describeCertificate method. If any one of these individual Promises fails, the top-level Promise chain will reject as well. Otherwise, the top-level Promise chain will resolve to an array of the results. (Note that I included an inessential .then step just to log the individual results, but you could remove that clause entirely.)
Hope this helps, and I apologize if I left any mistakes in the code.

Order of promises in AngularJS

Question:
Is there an "easy" way to cancel ($q-/$http-)promises in AngularJS or determine the order in which promises were resolved?
Example
I have a long running calculation and i request the result via $http. Some actions or events require me to restart the calculation (and thus sending a new $http request) before the initial promise is resolved. Thus i imagine i can't use a simple implementation like
$http.post().then(function(){
//apply data to view
})
because I can't ensure that the responses come back in the order in which i did send the requests - after all i want to show the result of the latest calculation when all promises were resolved properly.
However I would like to avoid waiting for the first response until i send a new request like this:
const timeExpensiveCalculation = function(){
return $http.post().then(function(response){
if (isNewCalculationChained) {return timeExpensiveCalculation();}
else {return response.data;}
})
}
Thoughts:
When using $http i can access the config-object on the response to use some timestamps or other identifiers to manually order the incoming responses. However i was hoping I could just tell angular somehow to cancel an outdated promise and thus not run the .then() function when it gets resolved.
This does not work without manual implementation for $q-promises instead of $http though.
Maybe just rejecting the promise right away is the way to go? But in both cases it might take forever until finally a promise is resolved before the next request is generated (which leads to an empty view in the meantime).
Is there some angular API-Function that i am missing or are there robust design patterns or "tricks" with promise chaining or $q.all to handle multiple promises that return the "same" data?

I do it by generating a requestId, and in the promise's then() function I check if the response is coming from the most recent requestId.
While this approach does not actually cancel the previous promises, it does provide a quick and easy way to ensure that you are handling the most recent request's response.
Something like:
var activeRequest;
function doRequest(params){
// requestId is the id for the request being made in this function call
var requestId = angular.toJson(params); // I usually md5 hash this
// activeRequest will always be the last requestId sent out
activeRequest = requestId;
$http.get('/api/something', {data: params})
.then(function(res){
if(activeRequest == requestId){
// this is the response for last request
// activeRequest is now handled, so clear it out
activeRequest = undefined;
}
else {
// response from previous request (typically gets ignored)
}
});
}
Edit:
On a side-note, I wanted to add that this concept of tracking requestId's can also be applied to preventing duplicate requests. For example, in my Data service's load(module, id) method, I do a little process like this:
generate the requestId based on the URL + parameters.
check in requests hash-table for the requestId
if requestId is not found: generate new request and store promise in hash-table
if requestId is found: simply return the promise from the hash-table
When the request finishes, remove the requestId's entry from the hash-table.

Cancelling a promise is just making it not invoke the onFulfilled and onRejected functions at the then stage. So as #user2263572 mentioned it's always best to let go the promise not cancelled (ES6 native promises can not be cancelled anyways) and handle this condition within it's then stage (like disregarding the task if a global variable is set to 2 as shown in the following snippet) and i am sure you can find tons of other ways to do it. One example could be;
Sorry that i use v (looks like check character) for resolve and x (obvious) for reject functions.
var prom1 = new Promise((v,x) => setTimeout(v.bind(null,"You shall not read this"),2000)),
prom2,
validPromise = 1;
prom1.then(val => validPromise === 1 && console.log(val));
// oh what have i done..!?! Now i have to fire a new promise
prom2 = new Promise((v,x) => setTimeout(v.bind(null,"This is what you will see"),3000));
validPromise = 2;
prom2.then(val => validPromise === 2 && console.log(val));

I'm still trying to figure out a good way to unit test this, but you could try out this kind of strategy:
var canceller = $q.defer();
service.sendCalculationRequest = function () {
canceller.resolve();
return $http({
method: 'GET',
url: '/do-calculation',
timeout: canceller.promise
});
};

In ECMA6 promises, there is a Promise.race(promiseArray) method. This takes an array of promises as its argument, and returns a single promise. The first promise to resolve in the array will hand off its resolved value to the .then of the returned promise, while the other array promises that came in second, etc., will not be waited upon.
Example:
var httpCall1 = $http.get('/api/something', {data: params})
.then(function(val) {
return {
id: "httpCall1"
val: val
}
})
var httpCall2 = $http.get('/api/something-else', {data: params})
.then(function(val) {
return {
id: "httpCall2"
val: val
}
})
// Might want to make a reusable function out of the above two, if you use this in Production
Promise.race([httpCall1, httpCall2])
.then(function(winningPromise) {
console.log('And the winner is ' + winningPromise.id);
doSomethingWith(winningPromise.val);
});
You could either use this with a Promise polyfil, or look into the q.race that someone's developed for Angular (though I haven't tested it).

Promises and Q - Completing a long series of functions with a forEach

A little confused on how best to write this - hope this description is clear.
I have a forEach function in which I go through some JS object data, and for each item i execute a function:
found.forEach(function(item) {
processData(item['userID']);
});
Within this processData function I am using a MongoDB find() call.
var processData = function(userIDSelected) {
User.find( {_id: userIDSelected},
{gender: 1, country:1}, function(req, foundUser) {
processUserInfo(foundUser[0]['gender']);
});
}
The issue I have is how do i wait for everything in forEach to complete considering that each call will be running processUserInfo in turn.
I've looked at using the Q library and Q.all however that doesnt work.
Is there a Q function that waits for everything in the long chain to complete?
Thanks

Q.all:
Returns a promise that is fulfilled with an array containing the fulfillment value of each promise, or is rejected with the same rejection reason as the first promise to be rejected.
or Q.allSettled:
Returns a promise that is fulfilled with an array of promise state snapshots, but only after all the original promises have settled, i.e. become either fulfilled or rejected.
So you'd do three things:
Modify processData and possibly your call to MongoDB so that you end up with processData returning a promise for the asynchronous operation. (Apologies, I'm not familiar with MongoDB. Alnitak says that with modern versions, if you don't supply a callback, MongoDB returns a promise (nice!). Otherwise, this question and its answers may help with returning a promise for a callback-based API.)
Use map instead of forEach to get an array of the resulting promises.
Use Q.all or Q.allSettled on that array of promises.
If User.find returns a promise when no callback is specified, #1 looks something like this:
var processData = function(userIDSelected) {
return User.find(
{_id: userIDSelected},
{gender: 1, country:1}
).then(function(req, foundUser) { // <== Check these args with the MongoDB docs!
return processUserInfo(foundUser[0]['gender']);
});
};
If not, you can do it yourself with Q.defer:
var processData = function(userIDSelected) {
var d = Q.defer();
User.find(
{_id: userIDSelected},
{gender: 1, country:1},
function(req, foundUser) {
processUserInfo(foundUser[0]['gender']);
d.resolve(/*...data could go here...*/); // You'd use d.reject() if there were an error
});
return d.promise;
};
Then here's what 2 and 3 look like:
Q.all(found.map(function(item) { // Or Q.allSettled
return processData(item);
}))
.then(...)
.catch(...);
If processData only ever uses its first argument (ignoring any extra ones), you can ditch the intermediary function:
Q.all(found.map(processData)) { // Or Q.allSettled
.then(...)
.catch(...);
...but only if processData ignores extra args, as map will pass it three (the value, its index, and the array).

The way you coded I can assume that userID is an ObjectId from MongoDB.
If that is the case, this is going to work as long as found is not empty, otherwise your users would wait for the server response forever.
processData(
// Retrieve an object
// {
// $in: [ObjectId, ObjectId, ...]
// }
//
found.reduce(
function(query, user) {
return query.$in.push(user.userID), query;
},
{$in: []}
)
);
You can have more info about the $in operator here.

Outer variables not being altered inside Promise.then function

On node while using thinky.js, I am trying to iterate through a loop and add each item to an array. This however, for some reason is not working.
In another place, it is indentical and working, just without a Promise.then function. Why is this not working?
var fixedItems = [];
for (i in tradeItems) {
var item = tradeItems[i];
Item.get(item["id"]).run().then(function(result) {
var f = { "assetid": result["asset_id"] };
console.log(f); // WOrks
fixedItems.push(f); // Doesn't work
});
}
console.log(fixedItems); // Nothing

A Promise represents the future result of a task. In this case you're logging fixedItems before your tasks (the calls to Item.get) have finished working. In other words, the then functions haven't run yet so nothing has been put into fixedItems.
If you want use fixedItems once it contains all of the items, you'll need to wait for all of the promises to resolve.
How you do that depends on the Promise library you're using. This example, with Promise.all, works with many libraries including native ES6 Promises:
// Get an array of Promises, one per item to fetch.
// The Item.get calls start running in parallel immediately.
var promises = Object.keys(tradeItems).map(function(key) {
var tradeItem = tradeItems[key];
return Item.get(tradeItem.id);
});
// Wait for all of the promises to resolve. When they do,
// work on all of the resolved values together.
Promise.all(promises)
.then(function(results) {
// At this point all of your promises have resolved.
// results is an array of all of the resolved values.
// Create your fixed items and return to make them available
// to future Promises in this chain
return results.map(function(result) {
return { assetid: result.asset_id }
});
})
.then(function(fixedItems) {
// In this example, all we want to do is log them
console.log(fixedItems);
});
Recommended reading: the HTML5 rocks intro to Promises.

Your problem is that you are calling console.log(fixedItems) before any of the promises in the loop have finished executing. A better way of doing this that would also solve the asynchronous problem is to put all the item IDs in an array first and retrieve all the items in a single query, which is also more efficient on the database side.
var itemIds = tradeItems.map(function(item) {
return item.id;
});
var fixedItems = [];
//you would need to write your own getItemsById() function or put the code
//to get the items here
getItemsById(itemIds).then(function(items) {
items.forEach(function(item) {
var f = { "assetid": result["asset_id"] };
fixedItems.push(f);
});
whenDone();
});
function whenDone() {
//you should be able to access fixedItems here
}
I couldn't easily find how to look up multiple records by ID in a single query with thinky, but I did find this page which might help:
http://c2journal.com/2013/01/17/rethinkdb-filtering-for-multiple-ids/
While this would be my preferred way of solving this problem, it would also be possible to still use multiple queries and use a promise chain to wait for them all to be resolved before continuing to your subsequent code. If you wanted to go that route, check out this link: http://promise-nuggets.github.io/articles/11-doing-things-in-parallel.html. (Note: I haven't personally used Bluebird, but I think the Bluebird example in that link may be out of date. The map method appears to be the current recommended way to do this with promises: https://stackoverflow.com/a/28167340/560114.)
Update: Or for this latter option, you can just use the code in joews's answer above.

Using promises/callbacks to wait for function to finish in javascript

I have a function which creates a database object out of three arrays. The arrays are filled in an each loop, one of the arrays relies on the value in the same iteration of the loop.
The dependent array uses the requests library and the cheerio library to grab a string to populate the array with.
Currently the dependent array fills with nulls which I think is because the loop is not waiting for the request to be returned.
I am still learning and would like to get this to work without direct blocking to keep things asynchronous so I'm looking into promises/callbacks.
This is being done server-side but from what I've seen in cheerios docs there is no promises capability.
Here's what I have so far. (getFile() is the function that isn't filling the 'c' array, it also depends on the current value being put into 'b'). I do know that the getFile function gets the correct value with a console log test, so the issue must be in the implementation of filling 'c'.
addToDB() is a function which saves a value into mongoDB, from testing I know that the objects are correctly being put into the db, just the c array is not correct.
function getInfo(path) {
$(path).each(function(i,e) {
a.push(...)
b.push(value)
c.push(getFile(value))
})
var entry = new DB...//(a,b,c)
addToDB(entry);
}
function getFile(...) {
request(fullUrl, function (err, resp, page) {
if (!err && resp.statusCode == 200) {
var $ = cheerio.load(page); // load the page
srcEp = $(this).attr("src");
return srcEp;
} // end error and status code
}); // end request
}
I've been reading about promises/callbacks and then() but I've yet to find anything which works.

First, you have to get your mind around the fact that any process that relies, at least in part, on an asynchronous sub-process, is itself inherently asynchronous.
At the lowest level of this question's code, request() is asynchronous, therefore its caller, getFile() is asynchronous, and its caller, getInfo() is also asynchronous.
Promises are an abstraction of the outcome of asynchronous processes and help enormously in coding the actions to be taken when such processes complete - successfully or under failure.
Typically, low-level asynchronous functions should return a promise to be acted on by their callers, which will, in turn, return a promise to their callers, and so on up the call stack. Inside each function, returned promise(s) may be acted on using promise methods, chiefly .then(), and may be aggregated using Promise.all() for example (syntax varies).
In this question, there is no evidence that request() currently returns a promise. You have three options :
discover that request() does, in fact, return a promise.
rewrite request() to return a promise.
write an adapter function (a "promisifier") that calls request(), and generates/returns the promise, which is later fulfilled or rejected depending on the outcome of request().
The first or second options would be ideal but the safe assumption for me (Roamer) is to assume that an adapter is required. Fortunately, I know enough from the question to be able to write one. Cheerio appears not to include jQuery's promise implementation, so a dedicated promise lib will be required.
Here is an adapter function, using syntax that will work with the Bluebird lib or native js promises:
//Promisifier for the low level function request()
function requestAsync(url) {
return new Promise(function(resolve, reject) {
request(url, function(err, resp, page) {
if (err) {
reject(err);
} else {
if (resp.statusCode !== 200) {
reject(new Error('request error: ' + resp.statusCode));
}
} else {
resolve(page);
}
});
});
}
Now getFile(...) and getInfo() can be written to make use of the promises returned from the lowest level's adapter.
//intermediate level function
function getFile(DOMelement) {
var fullUrl = ...;//something derived from DOMelement. Presumably either .val() or .text()
return requestAsync(fullUrl).then(function (page) {
var $ = cheerio.load(page);
srcEp = $(???).attr('src');//Not too sure what the selector should be. `$(this)` definitely won't work.
return srcEp;
});
}
//high level function
function getInfo(path) {
var a = [], b = [], c = [];//presumably
// Now map the $(path) to an array of promises by calling getFile() inside a .map() callback.
// By chaining .then() a/b/c are populated when the async data arrives.
var promises = $(path).map(function(i, e) {
return getFile(e).then(function(srcEp) {
a[i] = ...;
b[i] = e;
c[i] = srcEp;
});
});
//return an aggregated promise to getInfo's caller,
//in case it needs to take any action on settlement.
return Promise.all(promises).then(function() {
//What to do when all promises are fulfilled
var entry = new DB...//(a,b,c)
addToDB(entry);
}, function(error) {
//What to do if any of the promises fails
console.log(error);
//... you may want to do more.
});
}