Develop Reference

Promises structure misunderstood

I have a problem with understanding Promises syntax.
So, what I am trying to do:
getPosts() gets some data from a DB then, I want to get some metadata for each row with another promise call, addMetadata(). Then, once all the metadata is fetched, I want to console it out.
See my attempt below:
var getPosts = function(){
return new Promise(function(resolve, reject){
postModel.find()
.exec(function(err, posts) {
resolve(posts);
);
});
};
var addMetadata = function(posts){
var options = {
host: 'localhost',
port: 3000,
path: '',
method: 'GET'
};
var postPromises = posts.map(function(post) {
return new Promise(function(resolve) {
options.path = '/api/user?id=' + post.profileID;
var req = http.get(options, function(res) {
var bodyChunks = [];
res.on('data', function(chunk) {
bodyChunks.push(chunk);
}).on('end', function() {
var body = Buffer.concat(bodyChunks);
var parsedBody = JSON.parse(body);
post.fullname = parsedBody.user.fullname;
post.profilePic = parsedBody.user.profilePic;
// resolve the promise with the updated post
resolve(post);
});
});
});
});
// Is this the right place to put Promise.all???
Promise.all(postPromises)
.then(function(posts) {
//What should I put here
});
};
getPosts()
.then(function(posts){
return addMetadata(posts);
})
.then(function(posts){//I get a little lost here
console.log();//posts is undefined
});
Of course, my understanding is wrong but I thought I was going the right way. Can someone please guide me to the right direction?
Thanks

Change
// Is this the right place to put Promise.all???
Promise.all(postPromises)
.then(function (posts) {
//What should I put here
});
into
// Is this the right place to put Promise.all???
return Promise.all(postPromises);
This way your addMetadata function will return Promise that resolve when all promises from postPromises resolves or reject if any of postPromises rejects.

The key point to understand the async concept of it and what time the content is available.
Reading this will help to put you in the right direction.
For instance:
var promise = new Promise(function(resolve, reject) {
resolve(1);
});
promise
.then(function(val) {
console.log(val); // 1
return val + 2;
})
.then(function(val) {
console.log(val); // 3
})
After as per your scenario, in order to have all the metadata Promise.all is the way to go.
Promise.all(arrayOfPromises).then(function(arrayOfResults) {
// One result per each promise of AddMetadata
})

What you wanna do here, if I am correct, is called streams, as you wanna call multiple paralel promises as your concept of looping through list of posts using map is not going to work this way
Take a look at this short video introducing streams Streams - FunFunFunction, he is using library for workin with streams called Baconjs
Here is a short example on streams
const stupidNumberStream = {
each: (callback) => {
setTimeout( () => callback(1), 3000 )
setTimeout( () => callback(2), 2000 )
setTimeout( () => callback(3), 1000 )
}
}
stupidNumberStream.each(console.log)

Your getPosts function is good in the sense that its only job is to promisfy the database find. (Though, I think if it's mongo, the exec produces a promise for you).
Your addMetadataToAPost is less good, because it mixes up processing an array of posts and "promisifying" the http.get. Use the same pattern you applied correctly in the first function and return a promise to do a single get and add metadata. (It would be even better to just wrap the get, which you can reuse, and build a simple add-metadata function that returns - rather than creates - a promise)
// renamed pedantically
var addMetadataToAPost = function(post) {
return new Promise(function(resolve) {
options.path = '/api/user?id=' + post.profileID;
var req = http.get(options, function(res) {
var bodyChunks = [];
res.on('data', function(chunk) {
bodyChunks.push(chunk);
}).on('end', function() {
var body = Buffer.concat(bodyChunks);
var parsedBody = JSON.parse(body);
post.fullname = parsedBody.user.fullname;
post.profilePic = parsedBody.user.profilePic;
// resolve the promise with the updated post
resolve(post);
});
});
});
}
Now your batching function is simple:
// also renamed pedantically
var addMetadataToAllPosts = function(posts){
var postPromises = posts.map(function(post) {
return addMetadataToAPost(post)
})
return Promise.all(postPromises)
};
Your original code should work...
getPosts().then(function(posts){
return addMetadataToAllPosts(posts);
})
.then(function(posts){
console.log(posts);//posts should be an array of posts with metadata added
});

Wait for promises inside of a angular.forEach loop

I know this has been asked quite a few times already but after a day of search I still don't get it to work, although it's just like what is shown as a solution everywhere...
I have a async request to a database which returns an array of data. For each object in this array I need to start another async request to the database and as soon as ALL of these async requests resolve, I want to return them. I read you could do it with $q.all(...)
So here's the code:
Factory.firstAsyncRequest(id).then(function (arrayWithObjects) {
var promises = [];
var dataArr = [];
angular.forEach(arrayWithObjects, function (object, key) {
var deferred = $q.defer();
promises.push(deferred);
Factory.otherAsyncRequest(key).then(function (objectData) {
dataArr.push({
name: objectData.name,
key: key,
status: objectData.status
});
deferred.resolve();
console.info('Object ' + key + ' resolved');
});
});
$q.all(promises).then(function () {
$rootScope.data = dataArr;
console.info('All resolved');
});});
From the console I see that the $q.all is resolved BEFORE each object. Did I get something wrong? This seems to work for everyone...
Your help is highly appreciated, been looking the whole night, it's 5:30am now lol..
Cheers
EDIT:
So for anyone who's coming here later: It was just the promises.push(deferred.PROMISE) bit. Tho, I read that anguar.forEach is actually not a recommended method to loop through array because it was originally not constructed to be used by the end-user. Don't know if that's correct but I figured out another way if you don't want to use angular.forEach:
Users.getAll(uid).then(function (users) {
var uids = ObjHandler.getKeys(users); //own function just iterating through Object.keys and pushing them to the array
var cntr = 0;
function next() {
if (cntr < uids.length) {
Users.getProfile(uids[cntr]).then(function (profile) {
var Profile = {
name: profile.name,
key: uids[cntr],
status: profile.status
});
dataArr[uids[cntr]] = Profile;
if(cntr===uids.length-1) {
defer.resolve();
console.info('Service: query finished');
} else {cntr++;next}
});
}
}
next();
});
And the getKey function:
.factory('ObjHandler', [
function () {
return {
getKeys: function(obj) {
var r = [];
for (var k in obj) {
if (!obj.hasOwnProperty(k))
continue;
r.push(k)
}
return r
}
};
}])

Instead of
promises.push(deferred);
Try this:
promises.push(deferred.promise);

How to combine ES6 Generators with Promises

I'm trying to conceptually understand how ES6 Generators can make async code more streamlined. Here's a contrived example:
I have a function called getGitHubUser which takes a username and returns a Promise which ultimately resolves to the github user's info.
I have an array of usernames.
I'd like to call getGitHubUser with the first username and when that Promise resolves, I want to call getGitHubUser with the next username, and continue this until I've iterated through all the usernames.
I have a working implementation but I'm more curious on how I can leverage generators to make this better.
var getGitHubUser = (user) => {
// using jQuery's $.get
return Promise.resolve($.get("https://api.github.com/users/" + user));
};
var usernames = ["fay-jai", "jyek", "Maestro501", "jaclyntsui"];
getGitHubUser(usernames[0])
.then((result) => {
console.log(result); // fay-jai
return getGitHubUser(usernames[1]);
})
.then((result) => {
console.log(result); // jyek
return getGitHubUser(usernames[2]);
})
.then((result) => {
console.log(result); // Maestro501
return getGitHubUser(usernames[3]);
})
.then((result) => {
console.log(result); // jaclyntsui
});

If you would like to get the ide of how it works, consider this good article.
If you're lookign for some woring solution, there are lots of libraries to handle callback hell (becaise basically this is the main reason why people are looking for more elegant solutions).
Q.spawn was already given a brief description by #user890255 in his answer, but threre are others. For example, co which I like most:
var request = require('superagent');
co(function* () {
var data = [];
for(var i = 0; i < usernames.length; i++){
data.push(yield request.get("https://api.github.com/users/" + usernames[i]));
}
return data;
}).then((value) => {
console.log(value);
}, (err) => {
console.error(err.stack);
});
As you can see, co always returns a promise, which is very handy.
And minimalistic (due to a small file size, I suppose) vo
var request = require('superagent');
vo(function* () {
var data = [];
for(var i = 0; i < usernames.length; i++){
data.push(yield request.get("https://api.github.com/users/" + usernames[i]));
}
return data;
})((err, res) => {
console.log(res);
});
As you can see, the code in the generator function s pretty much the same.
Cheers!

This is how you do it using Q. Read also Harmony generators and promises for Node.js async fun and profit and JavaScript Promises.
var usernames = ["fay-jai", "jyek", "Maestro501", "jaclyntsui"];
Q.spawn(function *(){
var index = 0;
while (index < usernames.length){
console.log(yield Promise.resolve($.get("https://api.github.com/users/" + usernames[index])));
index++;
}
});

Promise in JavaScript (angular)

I'm quite new to angular and http calls.
Then i read something about Promise to aync calls. And i cannot understand it at all.
So i need some help to know if im even going the right direction.
I'm programming an API for getting Videoes, and details about them (Views etc) with Youtube API v3.
But i seem to get an error with the getting the details since my array is empty all the time.
/*var promises = [];*/ // PROMISE
var videometrics;
var videodetails = [];
var deferred = $q.defer();
$http.get('https://www.googleapis.com/youtube/v3/playlistItems?part=snippet&maxResults=50&playlistId=' + playlistId + '&key=AIzaSyDQv-WpATIWLinCB3H_sH4W1sKx7plyvRA')
.success(function (response) {
for (var i = 0; i < response.items.length; i++) {
var video = {
id: response.items[i].snippet.resourceId.videoId,
title: response.items[i].snippet.title
};
$scope.video.push(video)
}
/*console.log($scope.video)
var promises = [];*/
for (i = 0; i < $scope.video.length; i++) {
/*console.log("looping")
console.log($scope.video)
console.log("Vi henter fra id:")
console.log($scope.video[i].id)*/
$http.get('https://www.googleapis.com/youtube/v3/videos?part=statistics&id=' + $scope.video[i].id + '&key=AIzaSyDQv-WpATIWLinCB3H_sH4W1sKx7plyvRA')
.success(function (responsevideo) {
/*console.log($scope.video[i].id);*/
// console.log("we are in the metric loop")
// console.log($scope.video[i].id)
// console.log($scope.video[i].title)
// console.log(responsevideo)
videometrics = {
id: responsevideo.items[0].id,
views: responsevideo.items[0].statistics.viewCount,
likes: responsevideo.items[0].statistics.likeCount,
dislikes: responsevideo.items[0].statistics.dislikeCount,
favorites: responsevideo.items[0].statistics.favoriteCount,
comments: responsevideo.items[0].statistics.commentCount
};
videodetails.push(videometrics);
deferred.resolve(responsevideo);
/*detailsOnVideos = $scope.videometrics;*/
})
/* videodetails.push(videometrics);*/
}
/*promises.push(videodetails);*/ // PROMISE
console.log(videodetails);
/* console.log(promises);*/ //PROMISE LOGGER
/* console.log(videodetails);*/
console.log($scope.video);
console.log("")
pagetokenarr = response.nextPageToken;
console.log(pagetokenarr)
});
return deferred;
/*return $q.all(promises);*/ // PROMISE
as you see my first http get, is functionel, but the next one is not. And i cant understand why. But if i push my videodetails array into a promise array it works. And again. I donno why.

You don't need $q with $http. There's basically two ways (that I know of) to work with $http:
1) make a service, isolate $http there. Return the promise as such:
return $http.get({params...}).then(function(data) {
return data.result;
});
then in the controller you'd have:
SuperService.get(params...).then(function(data) { $scope.something = data; });
2) databinding! make a service, put the $http in there. But this time you bind the result to an object inside the service. In the service you also have a method to return that object.
$http.get({params...}).then(function(data) {
myObject = data.result;
});
Then in the controller:
$scope.spiderman = SuperService.getObj();
SuperService.get(params...);
When you call .get() AngularJS does its magic and updates $scope.spiderman (as soon as you get a response from the server, of course).
Relevant literature:
ngDocs
"We have a problem with promises" by Lawson

Here is a solution working in this plunker
Your code was pretty much working. The problem was that your console.log() was fired too early because it wasn't in a .success() or .then() of your promises. It was fired before the $http calls resolved.
I reworked a bit your code (specially theses for loops) to make it more readable.
Also you don't need to build your own promises till $http calls return promises
Here is the final code :
$http.get('https://www.googleapis.com/youtube/v3/playlistItems?part=snippet&maxResults=50&playlistId=' + playlistId + '&key=AIzaSyDQv-WpATIWLinCB3H_sH4W1sKx7plyvRA')
.success(function (response) {
angular.forEach(response.items, function(item){
var video = {
id: item.snippet.resourceId.videoId,
title: item.snippet.title
};
$scope.video.push(video);
})
console.log($scope.video);
angular.forEach($scope.video, function(video){
httpcalls.push($http.get('https://www.googleapis.com/youtube/v3/videos?part=statistics&id=' + video.id + '&key=AIzaSyDQv-WpATIWLinCB3H_sH4W1sKx7plyvRA')
.success(function (responsevideo) {
videometrics = {
id: responsevideo.items[0].id,
views: responsevideo.items[0].statistics.viewCount,
likes: responsevideo.items[0].statistics.likeCount,
dislikes: responsevideo.items[0].statistics.dislikeCount,
favorites: responsevideo.items[0].statistics.favoriteCount,
comments: responsevideo.items[0].statistics.commentCount
};
$scope.videodetails.push(videometrics);
}));
});
pagetokenarr = response.nextPageToken;
$q.all(httpcalls).then(function(){
console.log($scope.videodetails);
})
});
Note that i push all the $http call into a collection (httpcalls) and wrap your console.log into a $q.all(httpcalls).then() function. This will wait until all the $http calls into the collection are resolved.
Hope it helped.

Your code seems to working fine. I have removed all but the essentials, added the variables that I needed as placeholders as I'm not sure of the playlistId etc. Here is the code that I have tested.
var app = angular.module('app', []);
app.controller('ctrl', function ($scope, $http) {
var videometrics;
$scope.videodetails = [];
var playlistId = "PL63F0C78739B09958";
$scope.video = [];
$http.get('https://www.googleapis.com/youtube/v3/playlistItems?part=snippet&maxResults=50&playlistId=' + playlistId + '&key=AIzaSyDQv-WpATIWLinCB3H_sH4W1sKx7plyvRA')
.success(function (response) {
for (var i = 0; i < response.items.length; i++) {
var video = {
id: response.items[i].snippet.resourceId.videoId,
title: response.items[i].snippet.title
};
$scope.video.push(video);
}
for (i = 0; i < $scope.video.length; i++) {
$http.get('https://www.googleapis.com/youtube/v3/videos?part=statistics&id=' + $scope.video[i].id + '&key=AIzaSyDQv-WpATIWLinCB3H_sH4W1sKx7plyvRA')
.success(function (responsevideo) {
var videometrics = {
id: responsevideo.items[0].id,
views: responsevideo.items[0].statistics.viewCount,
likes: responsevideo.items[0].statistics.likeCount,
dislikes: responsevideo.items[0].statistics.dislikeCount,
favorites: responsevideo.items[0].statistics.favoriteCount,
comments: responsevideo.items[0].statistics.commentCount
};
$scope.videodetails.push(videometrics);
});
}
});
});
This will display in the browser using {{videodetails}} when it available on the scope.
If you need to work with the data as opposed to merely displaying it you will likely need to use a service as the other answer suggests.

Nodejs - SolrClient, how to wait for response

I got a question regarding the solr-client module of nodejs. I'm using this module for querying against a solr-index.
The module itself works fine as long as I don't have to wait for finishing of the query and as long I need the result only as a async result.
But currently I cannot find out, how I will be able to await the finishing of a search request and use the result in a sequential way.
I have the follwing method in my manager
SolrManager.prototype.promisedQuery = function(query, callback) {
var solrClient = solr.createClient(this.configuration.cores.page);
var docs = null;
var finished = false;
var deferred = Q.defer();
var request = solrClient.search(query, function(err,obj){
if (!err) {
if (obj.response.numFound > 0) {
deferred.resolve(obj.response.docs);
} else {
deferred.resolve(null);
}
} else {
deferred.reject(err);
}
});
var records = null;
var promise = deferred.promise;
promise.then(function(result) {
records = result;
}).fail(function(error){
records = error;
});
return records;
};
The problem here is, that I try to wait for the result of the query and use it as return value of "promisedQuery".
I try since days to use this method in a sequential call, also with different additional modules like "wait.for", "q", etc. but nothing seems to work.
The callback function of the solr-client will always be executed after the manager-method has already returned. Also the promise-methods will be even called after the return from the manager-method.
Can someone help me out on that topic or have some tips, how I can await the response of the solr-client-search operation and then give it back in a sequential way?
Thanks for any help.
Udo Gerhards

over one week, it seems now that I have found a solution:
SolrManager.prototype.promisedQuery = function(query, callback) {
var solrClient = solr.createClient(this.configuration.cores.page);
var docs = null;
var deferred = Q.defer();
var request = solrClient.search(query, function(err,obj){
if (!err) {
if (obj.response.numFound > 0) {
deferred.resolve(obj.response.docs);
} else {
deferred.resolve(null);
}
} else {
deferred.reject(err);
}
});
return deferred.promise;
};
in all other managers, which are calling the above function:
...
var dbPromise = this.solrManager.promisedQuery(query);
var _self = this;
return Q.async(function*(){
var result = yield dbPromise;
return result;
});
...
After first tests, it seems that synchronized methods will wait until the promise is settled.
The only thing is, that it runs only with NodeJs version 0.11.10, which supports generator functions, with activated --harmony-flag and "q"-module.
Best regards
Udo

You are just using the promises a bit incorrectly. Instead of returning records, you need to return 'deferred.promise'. It should look something like this (note that you don't need the callback you passed into promisedQuery).
SolrManager.prototype.promisedQuery = function(query) {
var solrClient = solr.createClient(this.configuration.cores.page),
deferred = Q.defer();
solrClient.search(query, function(err,obj){
if (!err) {
if (obj.response.numFound > 0) {
deferred.resolve(obj.response.docs);
} else {
deferred.resolve(null);
}
} else {
deferred.reject(err);
}
});
return deferred.promise;
};
To use it you would do something like:
SolrManager.promisedQuery(myquery)
.then(function (data) {
// data is whatever your 'resolved' in promisedQuery
}, function (err) {
// err is whatever you rejected in promisedQuery
});

based on rquinns answer I've changed the code like follows:
SolrManager.prototype.promisedQuery = function(query, callback) {
var solrClient = solr.createClient(this.configuration.cores.page);
var docs = null;
var finished = false;
var deferred = Q.defer();
var request = solrClient.search(query, function(err,obj){
if (!err) {
if (obj.response.numFound > 0) {
deferred.resolve(obj.response.docs);
} else {
deferred.resolve(null);
}
} else {
deferred.reject(err);
}
});
return deferred.promise;
};
...
DemoObject.prototype.toString = function() {
return SolrManager.promisedQuery(this.query).then(function(result){
return result['title'];
}).fail(function(error){
return error;
});
};
DemoObject.prototype.typeOf = function() {
return SolrManager.promisedQuery(this.query).then(function(result){
return result['title'];
}).fail(function(error){
return error;
});
};
I think, this is the right way to use the "promise"-object. But what happens when i do the follwing:
...
var demoObject = new DemoObject();
demoObject.query = "id:1";
console.log(''+demoObject);
...
or if I use "demoObject" by concatenating it to a string
...
var string = "Some string "+demoObject;
...
In case of the string concatenation, I'm currently not sure that the string will contain also the title field from the database. Same for console output.
Will nodejs be so intelligent that it resolves for e.g. the string concatenation "after" the results from the database will be available?
BR
Udo