I'm just learning Javascript and still pretty new to Parse's cloud code. I've been reading some articles and questions on promises and closures and still don't quite understand how to accomplish what I want do do yet. All the other questions/answer seem to be slightly different or difficult to understand.
I have a function that starts with a query that gets all the "Gyms". For each of those gyms, I need to run several other queries. All those inner queries (inside the loop) need to all complete before I can generate a final report for the gym. I want to understand the following things:
a.) How to allow the correct gym object from that each iteration of the loop to be accessible through the entire chain of queries in that iteration.
b.) Will all the results from the previously executed queries in my chain be available in the following queries? e.g. Can I access newWorkouts in the last function?
function createReports() {
var gymQuery = new Parse.Query(Parse.Object.extend("Gym"));
gymQuery.find({
success: function(results) {
for (var i = 0; i < results.length; ++i) {
/* jshint loopfunc: true */
var gym = results[i];
var newWorkoutsQuery = new Parse.Query(Parse.Object.extend("Workout"));
newWorkoutsQuery.equals("gym", gym);
newWorkoutsQuery.find().then(function(newWorkouts) {
var newLogsQuery = new Parse.Query(Parse.Object.extend("WorkoutLog"));
newLogsQuery.equals("gym", gym);
return newLogsQuery.find();
}).then(function(logsLastWeek) {
//Generate final report for gym using data from all above queries.
//Need access to gym, newWorkouts, and logsLastWeek
});
}
},
error:function() {
console.log("error");
}
});
}
Promise.all() should be able to help you out with this.
First, let's break out a function that retrieves the data for a single gym:
function getGymData(gym) {
var newWorkoutsQuery = new Parse.Query(Parse.Object.extend("Workout"));
newWorkoutsQuery.equals("gym", gym);
var newLogsQuery = new Parse.Query(Parse.Object.extend("WorkoutLog"));
newLogsQuery.equals("gym", gym);
return Promise.all([newWorkoutsQuery.find(), newLogsQuery.find()])
.then(function (results) {
return {
gym: gym,
workouts: results[0],
logs: results[1]
};
});
}
Then use Promise.all() across all the gyms:
function createReports() {
var gymQuery = new Parse.Query(Parse.Object.extend("Gym"));
return gymQuery.find()
.then(function (gyms) {
return Promise.all(gyms.map(getGymData));
})
.then(function (results) {
// results should be an array of objects, each with
// the properties gym, workouts, and logs
})
.catch(function (error) {
console.error(error);
});
}
Related
In a controller function, I make some operations:
Get a list of organizations with a promise
In the then of this promise, I loop through each of them to extract some data and populate some of my controller attributes.
One of this operation is to call another promise to gather all users attached to this organization, with a loop inside of it to extract name and other stuff.
When I get ALL of it, so every organization has been parsed, and within them all users too, I must call a function to update my view.
I got it working by setting some flags (orgParsed and usersParsed) but I find it to be... a code shame.
I heard about a way of maybe doing this by using $q to wait for the two promises and maybe loops inside their "then" to be resolve before calling my view function. But I struggle applying this code change since the second promise use the result of the first to gather the organization ID.
Here is my current code:
this.getOrgData = function () {
return Service.getList().then(function (result) {
var orgCount = result.Objects.length;
var orgParsed = 0;
_.forEach(result.Objects, function (org) {
org.Users = [];
// Some logic here using 'org' data
Service.getUsers(org.Id, 0, 0).then(function (userResult) {
usersParsed = 0;
_.forEach(userResult.Objects, function (user) {
// Some Logic here using 'user.Name'
usersParsed++;
});
orgParsed++;
if (orgParsed === orgCount && usersParsed === userResult.Objects.length) {
self.sortMenuList(); // My view Function
}
});
});
$scope.$broadcast("getOrgData");
});
};
Do you see any way to trigger my self.sortMenuList() function only when I can be sure I got all users of every companies parsed in more elegant/efficient/safe way?
Yes, that counting should definitely be replaced by $q.all, especially as you did not bother to handle any errors.
this.getOrgData = function () {
return Service.getList().then(function (result) {
$scope.$broadcast("getOrgData"); // not sure whether you want that here before the results from the loop
return $q.all(_.map(result.Objects, function (org) {
org.Users = [];
// Some logic here using 'org' data
return Service.getUsers(org.Id, 0, 0).then(function (userResult) {
_.forEach(userResult.Objects, function (user) {
// Some Logic here using 'user.Name'
});
});
}));
}).then(function() {
self.sortMenuList(); // My view Function;
})
};
The problem you describe sounds like you want to wait until a certain amount of promises are all resolved, and then do something with the result. That's really easy when you use Promise.all():
this.getOrgData = function () {
return Service.getList().then(function (result) {
var promises = [];
_.forEach(result.Objects, function (org) {
org.Users = [];
// Some logic here using 'org' data
// Store the promise for this user in the promises array
promises.push(Service.getUsers(org.Id, 0, 0));
});
// userResults is an array of all the results of the promises, in the same order as the getUsers was called
Promise.all(promises).then(function (userResults) {
_.forEach(userResults, function(userResult) {
_.forEach(userResult.Objects, function (user) {
// Some Logic here using 'user.Name'
});
});
self.sortMenuList();
});
$scope.$broadcast("getOrgData");
});
};
currently I am struggeling a little bit with node.js (I am new to it) doing different API requests (Usabilla API), work on the results and then combine them in order to work on the whole set (e.g. export).
Requesting the API is not the problem but I can't get the results out to do some other stuff on it (asynchronous code drives me crazy).
Attached please find a overview how I thought to do this. Maybe I am totally wrong about this or maybe you have other more elegant suggestions.
My code works until I have to request the two different API "adresses" (they are provided) and then extract the results to do some other stuff.
My problem here is that there are nested functions with a promise and I cant figure out how to pass this through the parent function inside waterfall to get handled by the next function.
In the code, of course there is nothing parallel as shown in the diagram.
Thats another point, how to do that ? Simply nest parallel and series/ another waterfall inside waterfall ?
I am a little bit confused because that gets more and more complex for a simple problem when this would be done with synchronous code.
Here I build up all my request querys (at the moment 4):
function buildQuery(IDs,callback){
var i = 0;
var max = Object.keys(IDs).length;
async.whilst(
function(){return i < max},
function(callback){
FeedbackQuery[i] =
{
identifier: IDs[i].identifier,
query:
{id: IDs[i].id,
params: {since:sinceDate,}
}
};
i++;
callback(null,i);
})
console.log(FeedbackQuery);
callback (null,FeedbackQuery);
};
I then have to decide which type of query it is and add it to an object which should contain all the items of this identifier type:
function FeedbackRequest(FeedbackQuery,callback)
{
var i = 0;
var max = Object.keys(FeedbackQuery).length;
async.whilst(
function(){return i < max},
function (callback){
identifier = FeedbackQuery[i].identifier;
APIquery = FeedbackQuery[i].query;
switch(identifier)
{
case 'mobilePortal':
console.log(FeedbackQuery[i].identifier, 'aktiviert!');
var result = api.websites.buttons.feedback.get(APIquery);
result.then(function(feedback)
{
var item = Object.keys(feedbackResults).length;
feedbackResultsA[item] = feedback;
callback(null, feedbackResultsA);
})
break;
case 'apps':
console.log(FeedbackQuery[i].identifier, 'aktiviert!');
var result = api.apps.forms.feedback.get(APIquery);
result.then(function(feedback)
{
var item = Object.keys(feedbackResults).length;
feedbackResultsB[item] = feedback;
callback(null, feedbackResultsB);
})
break;
}
i++;
callback(null,i);
})
};
Currently the functions are bundled in an async waterfall:
async.waterfall([
async.apply(buildQuery,IDs2request),
FeedbackRequest,
// a function to do something on the whole feedbackResults array
],function (err, result) {
// result now equals 'done'
if (err) { console.log('Something is wrong!'); }
return console.log('Done!');
})
How it actually should be:
Structure
Thank you very much for any tips or hints!
I'm not proficient with async, and I believe if you'r new to this, it's harder than a simple Promise library like bluebird combine with lodash for helpers.
What I would do based on your schemas :
var firstStepRequests = [];
firstStepRequests.push(buildQuery());// construct your first steps queries, can be a loop, goal is to have firstStepRequests to be an array of promise.
Promise.all(firstStepRequests)
.then((allResults) => {
var type1 = _.filter(allResults, 'request_type_1');
var type2 = _.filter(allResults, 'request_type_2');
return {
type1: type1,
type2: type2
};
})
.then((result) => {
result.type1 = //do some work
result.type2 = //do some work
return result;
})
.then((result) => {
//export or merge or whatever.
});
Goal is to have a simple state machine.
UPDATE
If you want to keep identifier for a request, you can use props to have :
var props = {
id_1:Promise,
id_2:Promise,
id_3:Promise
};
Promise.props(props).then((results) => {
// results is {
id_1:result of promise,
id_2:result of promise,
etc...
}
})
You could do something like :
var type1Promises = getType1Requests(); //array of type 1
var type2Promises = getType2Requests(); // array of type 2
var props = {
type_1: Promise.all(type1Promises),
type_2: Promise.all(type2Promises)
}
Promise.props(props).then((result) => {
//result is : {
type_1: array of result of promises of type 1
type_2: array of result of promises of type 2
}
})
I have a problem with performance on a relatively simple parse.com routine:
We have two classes and we want to make a cross product of them. One class contains a single object of boilerplate for the new objects (description etc.) the other class contains "large" sets (only 1000s of objects) of variable data (name, geopoint etc). Each new object also has some of its own columns not just data from the parents.
To do this, we do a query on the second class and perform an each operation. In each callback, we create and populate our new object and give it pointers to its parents. We call save() on each object (with some then() clauses for retries and error handling) and push the returned promise into an array. Finally we return status inside a when() promise on that array of save promises.
We originally created all the objects and then performed a saveall on them but couldn't get good enough error handling out of it - so we moved to when() with chains of promises with retries.
Trouble is, it's slow. It doesn't feel like the type of thing a nosql database should be slow at so we're blaming our design.
What's the best practice for cloning a bunch of objects from one class to another? or is it possible to get better results from saveAll failures?
My current code looks like this:
var doMakeALoadOfObjects = function(aJob,aCaller) {
Parse.Cloud.useMasterKey();
return aJob.save().then(function(aJob) {
theNumTasksPerLoc = aJob.get("numberOfTasksPerLocation");
if (theNumTasksPerLoc < 1) {
theNumTasksPerLoc = 1;
}
var publicJob = aJob.get("publicJob");
return publicJob.fetch();
}).then(function(publicJob) {
var locationList = aJob.get("locationList");
return locationList.fetch();
}).then(function(locationList) {
publicReadACL = new Parse.ACL();
publicReadACL.setPublicReadAccess(true);
publicReadACL.setRoleReadAccess("Admin",true);
publicReadACL.setRoleWriteAccess("Admin",true);
// Can't create a promise chain inside the loop so use this function.
var taskSaver = function(task) {
return task.save.then(function success(){
numTasksMade++;
},
function errorHandler(theError) {
numTimeOuts++;
numTaskCreateFails++;
logger.log("FAIL: failed to make a task for job " + aJob.get("referenceString") + " Error: " + JSON.stringify(theError));
});
};
var taskSaverWithRetry = function(task) {
return task.save().then(function() {
numTasksMade++;
return Parse.Promise.as();
}, function(error) {
logger.log("makeJobLive: FAIL saving task. Will try again. " + JSON.stringify(error));
numTimeOuts++;
return task.save().then(function() {
numTasksMade++;
return Parse.Promise.as();
}, function(error) {
numTimeOuts++;
numTaskCreateFails++;
logger.log("makeJobLive: FAIL saving task. Give up. " + JSON.stringify(error));
return Parse.Promise.as();
});
})
}
for (var j = 0; j < theNumTasksPerLoc; j++) {
var Task = Parse.Object.extend("Task");
var task = new Task();
task.set("column",stuff);
// Can't create a promise chain in the loop so use the function above.
taskSaverArray.push(taskSaverWithRetry(task));
}
return Parse.Promise.when(taskSaverArray);
}).then(function() {
}).then(function() {
// happy happy
},function(error){
// we never land here.
});
}
..."looks" like because I've deleted a lot of the object creation code and some housekeeping we do at the same time. I may have deleted some variable definitions too so I doubt this would run as is.
I'm trying to to pass a variable number of functions into Q.all()
It works fine if I code the array manually - however I want to build it up in a loop as the system wont know how many times to call the function until runtime - and needs to pass a different ID into it for each AJAX call.
I've tried various methods with no success (e.g. array[i] = function() {func}) - I guess eval() could be a last resort.
Any help would be massively helpful.
// Obviously this array loop wont work as it just executes the functions in the loop
// but the idea is to build up an array of functions to pass into Q
var arrayOfFunctions = [];
for(var i in NumberOfPets) {
arrayOfFunctions[i] = UpdatePets(i);
}
// Execute sequence of Ajax calls
Q.try(CreatePolicy)
.then(updateCustomer)
.then(function() {
// This doesn't work - Q just ignores it
return Q.all(arrayOfFunctions)
// This code below works fine (waits for all pets to be updated) - I am passing in the ID of the pet to be updated
// - But how can I create and pass in a dynamic array of functions to achieve this?
// return Q.all([UpdatePets(1), UpdatePets(2), UpdatePets(3), UpdatePets(4), UpdatePets(5), UpdatePets(5)]);
})
.then(function() {
// do something
})
.catch(function (error) {
// error handling
})
.done();
Thanks in advance.
Q.all doesn't expect an array of functions, but an array of promises. Use
Q.try(CreatePolicy)
.then(updateCustomer)
.then(function() {
var arrayOfPromises = [];
var numberOfPets = pets.length;
for (var i=0; i<numberOfPets; i++)
arrayOfPromises[i] = updatePet(pets[i], i); // or something
return Q.all(arrayOfPromises)
})
.then(function() {
// do something
})
.catch(function (error) {
// error handling
});
i have a recursive query like this (note: this is just an example):
var user = function(data)
{
this.minions = [];
this.loadMinions = function()
{
_user = this;
database.query('select * from users where owner='+data.id,function(err,result,fields)
{
for(var m in result)
{
_user.minions[result[m].id] = new user(result[m]);
_user.minions[result[m].id].loadMinions();
}
}
console.log("loaded all minions");
}
}
currentUser = new user(ID);
for (var m in currentUser.minions)
{
console.log("minion found!");
}
this don't work because the timmings are all wrong, the code don't wait for the query.
i've tried to do this:
var MyQuery = function(QueryString){
var Data;
var Done = false;
database.query(QueryString, function(err, result, fields) {
Data = result;
Done = true;
});
while(Done != true){};
return Data;
}
var user = function(data)
{
this.minions = [];
this.loadMinions = function()
{
_user = this;
result= MyQuery('select * from users where owner='+data.id);
for(var m in result)
{
_user.minions[result[m].id] = new user(result[m]);
_user.minions[result[m].id].loadMinions();
}
console.log("loaded all minions");
}
}
currentUser = new user(ID);
for (var m in currentUser.minions)
{
console.log("minion found!");
}
but he just freezes on the while, am i missing something?
The first hurdle to solving your problem is understanding that I/O in Node.js is asynchronous. Once you know how this applies to your problem the recursive part will be much easier (especially if you use a flow control library like Async or Step).
Here is an example that does some of what you're trying to do (minus the recursion). Personally, I would avoid recursively loading a possibly unknown number/depth of records like that; Instead load them on demand, like in this example:
var User = function(data) {
this.data = data
this.minions;
};
User.prototype.getMinions = function(primaryCallback) {
var that = this; // scope handle
if(this.minions) { // bypass the db query if results cached
return primaryCallback(null, this.minions);
}
// Callback invoked by database.query when it has the records
var aCallback = function(error, results, fields) {
if(error) {
return primaryCallback(error);
}
// This is where you would put your recursive minion initialization
// The problem you are going to have is callback counting, using a library
// like async or step would make this party much much easier
that.minions = results; // bypass the db query after this
primaryCallback(null, results);
}
database.query('SELECT * FROM users WHERE owner = ' + data.id, aCallback);
};
var user = new User(someData);
user.getMinions(function(error, minions) {
if(error) {
throw error;
}
// Inside the function invoked by primaryCallback(...)
minions.forEach(function(minion) {
console.log('found this minion:', minion);
});
});
The biggest thing to note in this example are the callbacks. The database.query(...) is asynchronous and you don't want to tie up the event loop waiting for it to finish. This is solved by providing a callback, aCallback, to the query, which is executed when the results are ready. Once that callback fires and after you perform whatever processing you want to do on the records you can fire the primaryCallback with the final results.
Each Node.js process is single-threaded, so the line
while(Done != true){};
takes over the thread, and the callback that would have set Done to true never gets run because the thead is blocked on an infinite loop.
You need to refactor your program so that code that depends on the results of the query is included within the callback itself. For example, make MyQuery take a callback argument:
MyQuery = function(QueryString, callback){
Then call the callback at the end of your database.query callback -- or even supply it as the database.query callback.
The freezing is unfortunately correct behaviour, as Node is single-threaded.
You need a scheduler package to fix this. Personally, I have been using Fibers-promise for this kind of issue. You might want to look at this or another promise library or at async