Trying to find unique ID in Node.js and MongoDB, by creating a while loop that queries MongoDB for existing IDs, until a unique value is found. If the ID is already in use, a number is incremented on the end until Mongo returns nothing.
Everything is working, except for the break; statement when a unique ID is found. Node.js returns: SyntaxError: Illegal break statement
The code:
db.collection('landmarks').findOne({'id':uniqueIDer}, function(err, data){
//if ID exists already
if (data.id){
var uniqueNumber = 1;
while (1) {
var uniqueNum_string = uniqueNumber.toString();
var newUnique = data.id + uniqueNum_string;
db.collection('landmarks').findOne({'id':newUnique}, function(err, data){
if (data.id){
uniqueNumber++;
}
else {
saveLandmark(newUnique);
break;
}
});
}
}
else {
saveLandmark(uniqueIDer);
}
});
What am I doing wrong?
EDIT:
Here's the fixed up code using async if anyone needs it :)
db.collection('landmarks').findOne({'id':uniqueIDer}, function(err, data){
if (data){
var uniqueNumber = 1;
var newUnique;
async.forever(function (next) {
var uniqueNum_string = uniqueNumber.toString();
newUnique = data.id + uniqueNum_string;
db.collection('landmarks').findOne({'id':newUnique,'world':worldVal}, function(err, data){
if (data){
console.log('entry found!');
uniqueNumber++;
next();
}
else {
console.log('entry not found!');
next('unique!'); // This is where the looping is stopped
}
});
},
function () {
saveLandmark(newUnique);
});
}
else {
saveLandmark(uniqueIDer);
}
});
Your break statement is not inside the body of a loop. It is, instead, inside the body of a function, namely the findOne callback. To see this more clearly, it can be helpful to temporarily use a named function as your callback handler:
var cb = function(err, data){
if (data.id){
uniqueNumber++;
}
else {
saveLandmark(newUnique);
break; // not inside a loop!
}
};
db.collection('landmarks').findOne({'id':uniqueIDer}, function(err, data){
//if ID exists already
if (data.id){
var uniqueNumber = 1;
while (1) {
var uniqueNum_string = uniqueNumber.toString();
var newUnique = data.id + uniqueNum_string;
db.collection('landmarks').findOne({'id':newUnique}, cb);
}
}
else {
saveLandmark(uniqueIDer);
}
});
It's pretty clear now that the break in the callback function body is not inside a loop! I've also made things break in other ways because the uniqueNumber and newUnique values are no longer in scope, but that's a different issue. :) The important thing to see here is that a function introduces a "hard" boundary in your code that can be difficult to see based purely on the syntax of the language. This is one of the reasons why this callback style of programming can be so tricky to get right.
In fact, it's much more difficult to do this than your original attempt at the code would imply. You'll need to have a way of passing a success signal up through possibly arbitrary layers of callbacks as you repeatedly call findOne and analyze the result (asynchronously).
You might get some help with this by using the excellent async library, for example https://github.com/caolan/async#whilst.
Related
Maybe this is a general issue, and i need a solution to my case : due to the non blocking aspect of javascript , I don't find how can I execute my function with all iteration in for loop , and here is my example ,
var text_list=[]
for (var i = 0; i < 10; i++) {
var element = array[index];
tesseract.process("img"+i+".jpg", options, function (err, text) {
if (err) {
return console.log("An error occured: ", err);
}
text_list.push(text)
});
}
console.log(text_list) //
And the result as if I do :
tesseract.process("img"+9+".jpg"...
tesseract.process("img"+9+".jpg"...
tesseract.process("img"+9+".jpg"...
.
.
.
and what i need is :
tesseract.process("img"+0+".jpg"...
tesseract.process("img"+1+".jpg"...
tesseract.process("img"+2+".jpg"...
.
.
.
Your question does not really explain what result you are getting and your code looks like it's missing parts of the code. So, all I can really do here to help is to explain generically (using your code where possible) how to solve this class of problem.
If you are ending up with a lot of results that all reference the last value of i in your loop, then you are probably trying to reference i in an async callback but because the callback is called sometime later, the for loop has already finished long before the callback executes. Thus, your value of i is sitting on the last value it would have in the for loop. But, your question doesn't actually show code that does that, so this is just a guess based on the limited result you describe. To solve that type of issue, you have make sure you're separately keeping track of i for each iteration of the loop. There are many ways to do that. In ES6, using let in the for loop definition will solve that entire issue for you. One can also construct a closure, use .forEach(), etc...
Async operations with a loop require extra work and coding to deal with. The modern solution is to convert your async operations to use promises and then use features such as Promise.all() to both tell you when all the async operations are done and to keep the results in order for you.
You can also code it manually without promises. Here's a manual version:
const len = 10;
let text_list = new Array(10);
let doneCnt = 0;
let errFlag = false;
// using let here so each invocation of the loop gets its own value of i
for (let i = 0; i < len; i++) {
tesseract.process("img"+i+".jpg", options, function (err, text) {
if (err) {
console.log("An error occured: ", err);
// make sure err is wrapped in error object
// so you can tell errors in text_list array from values
if (!(err instanceof Error)) {
err = new Error(err);
}
text_list[i] = err;
errFlag = true;
} else {
text_list[i] = text;
}
// see if we're done with all the requests
if (++doneCnt === len) {
if (errFlag) {
// deal with situation where there were some errors
} else {
// put code here to process finished text_list array
}
}
});
}
// you can't process results here because async operations are not
// done yet when code here runs
Or, using promises, you can make a "promisified" version of tesseract.process() and then use promise functionality to track multiple async operations:
// make promisified version of tesseract.process()
tesseract.processP = function(img, options) {
return new Promise(function(resolve, reject) {
tesseract.process(img, options, function(err, text) {
if (err) {
reject(err)
} else {
resolve(text);
}
});
});
}
const len = 10;
let promises = [];
for (let i = 0; i < len; i++) {
promises.push(tesseract.processP("img"+i+".jpg", options));
}
Promise.all(promises).then(function(results) {
// process results array here (in order)
}).catch(function(err) {
// handle error here
});
I am trying to implement a for loop that iterates through a list and subsequently calls two functions, only if the first function results are found.
The issue is that the second function (search.similar) might be taking longer to fetch results.
With the code below, when I run, all of the appropriate output from (search.locate) is correct, but only the last element's results from myList are stored from the (search.similar) function.
ie. all_results = [[cat_res1,mouse_res2],[dog_res1,mouse_res2],[mouse_res1,mouse_res2]]
How do I fix this to append the right results in the right order?
ie. all_results = [[cat_res1,cat_res2],[dog_res1,dog_res2],[mouse_res1,mouse_res2]]
var search = require('./search');
var myList = ['cat','dog','mouse'];
var all_results = [];
for (i=0; i<myList.length; i++){
/* locate function*/
search.locate(myList[i], function (err, searchResult){
if (err){
console.log("Error");
return;
}
if (!searchResult){
console.log("Cannot find it");
return;
}
/*similarity function*/
/* seems to take longer*/
search.similar(myList[i], function (err, similarResult){
if (err){
return;
}
if (!similarResult){
return;
}
var res1 = searchResult.data;
var res2 = similarResult.data;
/* append results to array*/
all_results.push([res1,res2]);
}
});
}
Javascript can be thought of as asynchronous, in that the execution of particular functions do not necessarily happen synchronously, however "describing JavaScript as asynchronous is perhaps misleading. It's more accurate to say that JavaScript is synchronous and single-threaded with various callback mechanisms"
In order to accomplish your goal, though you may still get some ordering issues with the top array, you will need to wrap your .similar() call in another function that takes both arguments. Your reference to the "item" on the top search is changing:
function searchNestedSimilar(item, topRes) {
search.similar(item, function (err, similarResult) {
if (err){
return;
}
if (!topRes){
return;
}
var res1 = topRes.data
var res2 = similarResult.data
// append results to array
all_results.push([res1,res2])
}
}
function searchLocate(item) {
search.locate(item, function (err, searchResult) {
if (err){
console.log("Error");
return;
}
if (!searchResult){
console.log("Cannot find it");
return;
}
searchNestedSimilar(item, searchResults);
}
I encapsulated both calls to keep it modular, but since "item" is in the closure, you really only need the searchLocate() function to wrap your capture your item reference during iteration.
This is a good case for Promises (see Bluebird JS for example http://bluebirdjs.com/docs/getting-started.html) or you could do it with
async.map().
This page talks about it well, too. http://promise-nuggets.github.io/articles/14-map-in-parallel.html
There are many Stack Overflows discussing Promises as well. Understanding JS Promises for example.
A rough example of how to write this with a Promise:
var search = require('./search');
var myList = ['cat','dog','mouse']
var all_results = []
var Promise = require('bluebird');
var locate = Promise.promisify(search.locate);
var similar = Promise.promisify(search.similar);
for (i = 0; i < myList.length; i++){
// locate function
locate(myList[i], function (err, searchResult) {
if (err) {
console.log("Error");
return;
}
if (!searchResult){
console.log("Cannot find it");
return;
}
}).then(function(result) {
//similarity function
similar(myList[i], function (err, similarResult) {
if (err){
return;
}
if (!similarResult){
return;
}
var res1 = searchResult.data
var res2 = similarResult.data
// append results to array
all_results.push([res1,res2])
}).finally(function() {
// NOP
});
});
}
I am doing a for loop to find the result from mongodb, and concat the array. But I am not getting the final results array when the loop is finished. I am new to node.js, and I think it's not working like objective-c callback.
app.get('/users/self/feed', function(req, res){
var query = Bill.find({user: req.param('userId')});
query.sort('-createdAt');
query.exec(function(err, bill){
if (bill) {
var arr = bill;
Following.findOne({user: req.param('userId')}, function(err, follow){
if (follow) {
var follows = follow.following; //this is a array of user ids
for (var i = 0; i < follows.length; i++) {
var followId = follows[i];
Bill.find({user: followId}, function(err, result){
arr = arr.concat(result);
// res.send(200, arr);// this is working.
});
}
} else {
res.send(400, err);
}
});
res.send(200, arr); //if put here, i am not getting the final results
} else {
res.send(400, err);
}
})
});
While I'm not entirely familiar with MongoDB, a quick reading of their documentation shows that they provide an asynchronous Node.js interface.
That said, both the findOne and find operations start, but don't necessarily complete by the time you reach res.send(200, arr) meaning arr will still be empty.
Instead, you should send your response back once all asynchronous calls complete meaning you could do something like:
var billsToFind = follows.length;
for (var i = 0; i < follows.length; i++) {
var followId = follows[i];
Bill.find({user: followId}, function(err, result){
arr = arr.concat(result);
billsToFind -= 1;
if(billsToFind === 0){
res.send(200, arr);
}
});
}
The approach uses a counter for all of the inner async calls (I'm ignoring the findOne because we're currently inside its callback anyway). As each Bill.find call completes it decrements the counter and once it reaches 0 it means that all callbacks have fired (this works since Bill.find is called for every item in the array follows) and it sends back the response with the full array.
That's true. Your codes inside for will be executed in parallel at the same time (and with the same value of i I think). If you added console.log inside and after your for loop you will found the outside one will be printed before inside one.
You can wrap the code that inside your for into array of functions and execute them by using async module (https://www.npmjs.org/package/async) in parallel or series, and retrieve the final result from async.parallel or async.series's last parameter.
I'm new to js.
I am using express for node js, and mongoose as a mongo orm.
function direct_tags_search_in_db(tags){
var final_results = [];
for (var i=0; i<tags.length; ++i) {
var tag = tags[i];
Question.find({tags: tag}).exec(function(err, questions) {
final_results.push(questions);
if (i == tags.length -1 ){
return final_results;
}
});
}
};
I get empty results, because of the asynchronously of the find. But I don't know what the best approach for this.
Appriciate a little help, thanks.
You will often find that methods such as Question.find().exec that accept a function as an argument are async. It is especially common for methods that perform network requests or file system operations. These are most commonly referred to as a callback. That being the case, if you would like something to occur when the async task(s) complete, you need to also implement a callback.
Also, it is possible that your reference to tag is being changed in a way that is likely undesired. There are a number of solutions, here is a simple one.
function direct_tags_search_in_db(tags, callback){
var final_results = [];
// Array.forEach is able to retain the appropriate `tag` reference
tags.forEach(function(tag){
Question.find({tags: tag}).exec(function(err, questions) {
// We should be making sure to handle errors
if (err) {
// Return errors to the requester
callback(err);
} else {
final_results.push(questions);
if (i == tags.length -1 ){
// All done, return the results
callback(null, final_results);
}
}
});
});
};
You will notice that when we implement our own callback, that we follow the same common pattern as the callback for Question.find().exec(function(err, result){}); -- first argument a potential error, second argument the result. That is why when we return the results, we provide null as the first argument callback(null, final_results);
Quick example of calling this function:
direct_tags_search_in_db([1, 2, 3], function(err, results){
if (err) {
console.error('Error!');
console.error(err);
} else {
console.log('Final results');
console.log(results);
}
});
Another option for solving various async goals is the async module, promises, or otherwise.
i have a recursive query like this (note: this is just an example):
var user = function(data)
{
this.minions = [];
this.loadMinions = function()
{
_user = this;
database.query('select * from users where owner='+data.id,function(err,result,fields)
{
for(var m in result)
{
_user.minions[result[m].id] = new user(result[m]);
_user.minions[result[m].id].loadMinions();
}
}
console.log("loaded all minions");
}
}
currentUser = new user(ID);
for (var m in currentUser.minions)
{
console.log("minion found!");
}
this don't work because the timmings are all wrong, the code don't wait for the query.
i've tried to do this:
var MyQuery = function(QueryString){
var Data;
var Done = false;
database.query(QueryString, function(err, result, fields) {
Data = result;
Done = true;
});
while(Done != true){};
return Data;
}
var user = function(data)
{
this.minions = [];
this.loadMinions = function()
{
_user = this;
result= MyQuery('select * from users where owner='+data.id);
for(var m in result)
{
_user.minions[result[m].id] = new user(result[m]);
_user.minions[result[m].id].loadMinions();
}
console.log("loaded all minions");
}
}
currentUser = new user(ID);
for (var m in currentUser.minions)
{
console.log("minion found!");
}
but he just freezes on the while, am i missing something?
The first hurdle to solving your problem is understanding that I/O in Node.js is asynchronous. Once you know how this applies to your problem the recursive part will be much easier (especially if you use a flow control library like Async or Step).
Here is an example that does some of what you're trying to do (minus the recursion). Personally, I would avoid recursively loading a possibly unknown number/depth of records like that; Instead load them on demand, like in this example:
var User = function(data) {
this.data = data
this.minions;
};
User.prototype.getMinions = function(primaryCallback) {
var that = this; // scope handle
if(this.minions) { // bypass the db query if results cached
return primaryCallback(null, this.minions);
}
// Callback invoked by database.query when it has the records
var aCallback = function(error, results, fields) {
if(error) {
return primaryCallback(error);
}
// This is where you would put your recursive minion initialization
// The problem you are going to have is callback counting, using a library
// like async or step would make this party much much easier
that.minions = results; // bypass the db query after this
primaryCallback(null, results);
}
database.query('SELECT * FROM users WHERE owner = ' + data.id, aCallback);
};
var user = new User(someData);
user.getMinions(function(error, minions) {
if(error) {
throw error;
}
// Inside the function invoked by primaryCallback(...)
minions.forEach(function(minion) {
console.log('found this minion:', minion);
});
});
The biggest thing to note in this example are the callbacks. The database.query(...) is asynchronous and you don't want to tie up the event loop waiting for it to finish. This is solved by providing a callback, aCallback, to the query, which is executed when the results are ready. Once that callback fires and after you perform whatever processing you want to do on the records you can fire the primaryCallback with the final results.
Each Node.js process is single-threaded, so the line
while(Done != true){};
takes over the thread, and the callback that would have set Done to true never gets run because the thead is blocked on an infinite loop.
You need to refactor your program so that code that depends on the results of the query is included within the callback itself. For example, make MyQuery take a callback argument:
MyQuery = function(QueryString, callback){
Then call the callback at the end of your database.query callback -- or even supply it as the database.query callback.
The freezing is unfortunately correct behaviour, as Node is single-threaded.
You need a scheduler package to fix this. Personally, I have been using Fibers-promise for this kind of issue. You might want to look at this or another promise library or at async