I am trying to generate a Data object that I can use to create new documents in my MongoDB database using mongoose in NodeJS.
This is the snippet of code:
console.log(RecipeData.ingredients);
RecipeData.spices.forEach(function(spice){
module.exports.findSpiceByName(spice, function(err, res){
if (err){
return callback(err);
}
RecipeData.ingredients.push(res._id);
return callback(null, RecipeData);
});
});
console.log(RecipeData.ingredients);
basically, i have RecipeData that is a object with a few attributes. the main ones in this case is spice and ingredients. both of which are list of Strings.
I want to check RecipeData.spice:
if there is a element in the list: (the element would be the name of the spice)
I want to find the spice associated to that name in my Spices Collection and add its _id to my RecipeData.ingredients permanently
both the console.log's have the same output when they shouldn't in the example I am using to test this code.
Any idea why the variable RecipeData.ingredients is not changing outside the ForEach loop?
Thanks for any help in advance.
Sounds like an asynchronous programming issue. Though you didn't post the source, I'm assuming that module.exports.findSpiceByName is asynchronous, where forEach is not, so the forEach finishes and your second console.log runs before your findSpiceByName calls have time to complete.
One way to resolve this would be to use Promises and then wait for all of them to complete before trying to inspect the ingredients:
var spicePromises = RecipeData.spices.map(function(spice) {
return new Promise(function(resolve, reject) {
module.exports.findSpiceByName(spice, function(err, res) {
if (err) {
return reject(err);
}
resolve(res._id);
});
});
});
Promise.all(spicePromises).then(function(ingredients) {
Recipe.ingredients = ingredients;
console.log(RecipeData.ingredients);
});
Related
For my project, I am using electron and react with ipcRenderer and ipcMain to communicate with a database. I can see it go through the database, but it returns an empty array. It is like the array is returned before anything is read from the db.
Here is the code I am using in my ipc main:
I am expecting it to return the names of the categories, but all it returns cateNames blank.
ipcMain.on(channels.GET_CATS, async(event,type) => {
console.log("made it")
let cateNames=[];
db.each(`SELECT name FROM categories WHERE type=?`, [type],(err, row) => {
if (err) {
return console.error(err.message);
}
console.log(row.name);
cateNames.push(row.name);
});
console.log(cateNames);
event.sender.send(channels.GET_LOGIN, cateNames);
});
I send the request with
ipcRenderer.send(channels.GET_CATS,"Donation");
with an ipcRenderer.on listening that will output the array to the console.
Javascript / Node.js execute commands in sequential order and does not "wait" when moving from one command to the next unless explicitely told to via the use of promises and async / await commands.
The reason your current code returns an empty cateNames array is becasue execution of the code does not "wait" for the db.each command to return it's callback. The callback is only returned once the DB has someting to return, which will either be a row or an error. This takes time. In the meantime, execution has moved on the next command.
To make this block of code "wait" until the DB has returned all available rows (if any) we could to use promises.
Instead, I propose a simpler method. Instead of pushing row.name with every db.each iteration, just use db.all and craft the response afterwards.
ipcMain.on(channels.GET_CATS, async(event, type) => {
console.log("made it")
let cateNames = [];
db.all(`SELECT name FROM categories WHERE type=?`, [type], (err, rows) => {
if (err) {
return console.error(err.message);
}
for (let row of rows) {
cateNames.push(row.name);
}
console.log(cateNames);
// Use event.reply(channel, data);
event.reply(channels.GET_LOGIN, cateNames);
});
});
I have two asynchronous actions with callback. I would like to be sure that both of them succeed or fail, but not that one succeed and one fail. It should probably be like one process for both actions which could be revert back ?
Let's illustrate :
// In this simplified code, i assume i uploaded a file in a temporary folder.
// Every validations passed and now the goal is to move the file from temporary folder to final destination folder and to save it into database.
// This 'move' and this 'save' are my two async actions with a callback when each action is completed.
// Maybe i am not using the right way / pattern to do it, thank you for enlightening me if necessary.
myController.create = function (req, res, next) {
// I move the file from oldPath (temp directory) to newPath (final destination)
mv(oldPath, newPath, function (err) {
// If err, file is not moved i stop here, went fine. The temp directory is cleared later by the system every X period of time.
if (err) { return next(err); }
var file = new FileModel({
// some properties (name, path...)
});
// The file is now moved, need to save it into database
file.save(function (err) {
if (!err) { return next(); } // everything went fine
// If err, nothing stored in database but the file is still in final folder :o
// I could unlink the file but it still can fail and keep my file inside destination folder with no database entry.
fs.unlink(new_path, function (other_err) {
if (other_err) { return next(other_err); }
return next(err);
}
});
});
};
In the code above, if first action succeed, nothing guarantee that my second action will succeed too and that i could revert back (my first action) if it fail. The two actions are separate and independant instead of being linked / paired and working together.
If moving the file succeed, the save in database should succeed too. If the save in database doesn't succeed, then i should revert back to temp directory or delete the file from the destination folder to be in adequacy with database. In other words, if the second action fail, the first one should fail two.
What is a good way to achieve that ?
EDIT : One solution i can see would be to check every X period of time if each file in the final destination folder has an entry in db and if it doesn't, to delete it.
You need to use promises to implement such kind of thing, For example, you need to create a user and then send a notification. So both actions are async and need to be done one after one.
const user = {};
// This function create user and send back a id
user.createUser = (data) => {
return new Promise((resolve, reject) => {
// ...you callbacks
if (some conditions are true) {
return resolve(id);
} else {
return reject();
}
});
};
// This function finds user detail by id and send notifiaction
user.sendNotification = (id) => {
return new Promise((resolve, reject) => {
// ...you callbacks
return resolve();
});
};
user.action = async () => {
try {
const userId = await user.createUser(); // Wait for promise to resolve
await user.sendNotification(userId);
return true;
} catch (err) {
throw err;
}
};
module.exports = user;
In the above code, you can see the user.action() function call 2 separate function one by one, async/await only works on promises, So we made that functions promisify and use it once using a keyword await. So, in short, you need to use promises to handle such kind of things.
I hope it helps. Happy Coding :)
I'm writing a crawler using node.js. At first, I need to fetch the main page to get the URL of each item on that page, then I crawl URL of each item to get details of them one-by-one
fetchPage(url) is to get HTML text of a link
function fetchPage(url){
return new Promise(
(resolve,reject)=>{
agent
.get(url)
.end(function(err,res){
if (err){
reject(err);
} else{
resolve(res.text);
}
});
});
}
This is the global call of this crawler
fetchPage(link).then(
(result)=>{
const urls=getUrls(result);
for (var i=0;i<5;i++){
fetchItem(urls[i].link).then(
(result)=>{
console.log('Done');
},
(error)=>console.log(error)
);
}
},
(error)=>console.log(error)
);
I processed to get the URLs of all items after fetching the main page (via getUrls)
fetchItem(url) is another Promise that ensures every HTML text of an item should be processed via getItem after being fetched by fetchPage
function fetchItem(url){
return new Promise(
(resolve,reject)=>{
fetchPage(url).then(
(result)=>{
getItem(result);
},
(error)=>reject(error)
);
});
}
It does crawl. It does get all items I need without any lack of information.
But there's something wrong with my code. Why doesn't the console log Done message for me?
Results are not in the right order. The order of crawled results is not as I expected, it is different from the order on the website.
Please point out what have I misunderstood and done wrong with these asynchronous control? How to ensure the order of them? How to fix this code to meet?
How should I do if I want to log a message All done after all items are completely crawled, making sure they're completely fetched in correct order?
Done is not getting called because you are not resolving the Promise created in fetchItem function.
I guess to maintain the order of results, you might want to use Promise.all. It will also help in getting All done message when all items are completely crawled.
I will start with changing fetchPage function by converting urls to a array of fetchItem promises using map that I can pass to Promise.all. Something like this
fetchPage(link).then(
(result)=>{
const urls=getUrls(result);
var promises = urls.map((url) => fetchItem(url.link));
Promise.all(promises).then((values) => {
console.log('All done');
console.log(values);
}, (error) => {
console.log(error);
});
},
(error)=>console.log(error)
);
then adding resolve to your fetchItem method.
function fetchItem(url){
return new Promise(
(resolve,reject)=>{
fetchPage(url).then(
(result)=>{
resolve(getItem(result));
},
(error)=>reject(error)
);
});
}
Im currently implementing a Purchase Order type View. Where I have a PurchaseOrder table and a PurchaseOrderLine table for the items. The first thing I do when the use presses the save button I first save the Purchase Order and then I retrieve the PurchaseOrderID and save to each individual PurchaseOrder item. The problems is the following:
Promise.resolve( app.PurchaseOrder.create(formData) ).then(function(response){
purchaseOrderID = response.collection.models[0].attributes.id;
for(var key in formLineData){
if(formLineData.hasOwnProperty(key)){
formLineData[key]['requestID'] = purchaseOrderID;
app.PurchaseOrderLines.create(formLineData[key]);
}
}
}).catch(function(error){
console.log(error);
})
formData is the PurchaseOrder data, formLineData is the PurchaseOrderLine Data(I do the for loop to insert requestIDs to all items).
I am using a Promise because collection.fetch does not return a promise on Backbone(I think my implementation isn't quite correct because Promise.resolve() is use to make thenables a promise and in this case it isn't). The problem is that when the save button is clicked the then part passes even PurchaseOrder hasn't been created. So when it gets to the PurchaseOrderLine.create, all the items are saved without a PurchaseOrderID. I have two options:
Add a server validation for this. The problem with this is that everytime is going to return an error and this can be bothersome to the user.
Add a setTimeout to at least wait a couple seconds for the write to over on the server.
Could please shed a light on this topic.
you can try something like this
app.PurchaseOrder.create(formData).then(function (response) {
var purchaseOrderID = response.collection.models[0].attributes.id;
return new Promise(async function (resolve) {
for (var key in formLineData) {
if (formLineData.hasOwnProperty(key)) {
formLineData[key]["requestID"] = purchaseOrderID;
await app.PurchaseOrderLines.create(formLineData[key]);
}
}
resolve()
});
});
or maybe doing something like this using Promise.all
Promise.all(formLineData.map(()=>app.PurchaseOrderLines.create(formLineData[key])))
I have this code
App.Model('users').find(function (err, users) {
users.forEach(function(user) {
console.log(user.username);
});
});
//make usernames availible here.
This console logs the usernames.
Instead of just logging to the console I want make use of the data.
But How can do this?
Thanks
They will never be available where you want them. This isn't how async/node.js programming works.
Possible:
App.Model('users').find(function (err, users) {
users.forEach(function(user) {
myCallBack(user);
});
});
var myCallBack = function(user) {
//make usernames availible here.
//this will be called with every user object
console.log(user);
}
Other possibilites: EventEmitter, flow controll libraries (e.g. async).
If you have written code in other languages before you need to be open to take a complete new approach how data will be handled.
With node js you don't do:
//make usernames availible here.
You do:
App.Model('users').find(function (err, users) {
//usernames are available here. Pass them somewhere. Notify some subscribers that you have data.
});
//The code here has executed a long time ago