I am sure this is a basic question, but I have been searching google for awhile and can't find a satisfactory answer..
I am used to programming MySQL select queries in PHP and simply grabbing a result, looping through each row, and within the loop doing further queries based on the column values of each individual row.
However, I'm working with javascript server side code now that relies on a SQL object where you pass the query and then a callback function that will be invoked after the query is run.
I'm confused with some of the scoping issues and how to best cleanly do this. For example, I don't want to do something like:
SQL.query("select * from blah", function(result) {
for(var i = 0; i < result.length; i++) {
SQL.query("select * from blah2 where i =" + result[i].property, function(result2) {
//now how do I access result from here? I know this isn't scoped correctly
});
}
});
What is the standard way to write this style of nested SQL query and not have scoping issues/messy code? Thanks!
This is very interesting... I've never heard of "server-side javascript"... but none the less this might help organize your code a bit. I use this method to organize my ajax request callbacks.
using your example it would look like this.
SQL.query("select * from some_table", function(result){ runNestedQuery(result); });
function runNestedQuery(result){
for(var i = 0; i < result.length; i++) {
SQL.query("select * from blah2 where i =" + result[i].property, function(result2){ nestedResult(result2); });
}
}
There are no scoping issues with your above code - but this is a nice way I like to organize this kind of thing.
result will be available in the second callback, that's how closures in JavaScript work, the functions has access to all variables in the outer scopes it was defined in.
function outer() {
var foo = 1;
function inner() { // inherits the scope of outer
var bla = 2;
console.log(foo); // works!
// another function in here well inherit both the scope of inner AND outer, and so on
}
inner();
console.log(bla); // doesn't work, raises "ReferenceError: bla is not defined"
}
outer();
Now, on to the problem, i will not point to the correct value, it too will be inherited to the second callback but it`s a reference and will therefore has the wrong value.
Fix is to create another closure:
SQL.query("select * from blah", function(result) {
for(var i = 0; i < result.length; i++) {
(function(innerResult) { // anonymous function to provide yet another scope
SQL.query("select * from blah2 where i =" + innerResult.property, function(result2) {
// innerResult has the correct value
});
})(result[i]); // pass the current result into the function
}
});
Or an extra function:
function resultThingy(result) {
SQL.query("select * from blah2 where i =" + result.property, function(result2) {
// result has the correct value
});
}
SQL.query("select * from blah", function(result) {
for(var i = 0; i < result.length; i++) {
resultThingy(result[i]);
}
});
Since you are using server-side Javascript, you can likely use forEach. Assuming that result instanceof Array == true:
SQL.query("select * from blah", function(result) {
result.forEach(function(item, index) {
SQL.query("select * from blah2 where i = " + item.property, function(result2) {
console.log(item, index, result); //works as intended
});
});
});
If result is merely array-like, then this
Array.prototype.forEach.call(result, function(item, index) { // etc...
should do the trick.
As others have pointed out result actually will be available all the way down in the nested callback.
But there is a very tricky part to this:
...Because the nested query runs asynchronously, your code will actually fire off a bunch of parallel queries -- one for each row in result -- all running at the same time (!). This is almost certainly not what you want; and unless result is very small indeed, all the simultaneous queries will use up all your available db connections rather quickly.
To remedy this, you might use something like this:
SQL.query("select * from blah", function(result) {
handleBlahRow( result, 0 );
});
function handleBlahRow( result, i ) {
if( !result || (i >= result.length)) return;
SQL.query("select * from blah2 where i =" + result[i].property, function(result2) {
// kick off the next query
handleBlahRow( result, i+1 );
// result, i, *and* result2 are all accessible here.
// do whatever you need to do with them
});
});
The above will run your nested queries 1-at-a-time. It's fairly easy to adapt the above to introduce limited parallelism (eg. 4-at-a-time), if you want it -- though it's probably not necessary.
Related
I have the following code (dbclient is Redis client for Node.js):
dbclient.zrange("cache", -1000000000000000, +1000000000000000, function(err, replies) {
logger.info("Replies: " + replies.length);
logger.info(err);
for (var i = 0; i < replies.length; i++) {
logger.info("I: " + i)
dbclient.hmget("doc:" + replies[i], "size", function(err, res) {
cache.set(replies[i], parseInt(res[0]));
logger.info(res[0]);
});
}
});
I notice a strange behavior:
The first output is: Replies: 195748, but in the for loop I notice that it always prints I: 195747 and the res[0] is always 198536. This run 195747 times.
It seems that it's stuck on the last index, and doesn't iterate over all items.
This is one of the most common errors in Javascript, in
function(err,res){
cache.set(replies[i], parseInt(res[0]));
logger.info(res[0])
}
you are using i, but that is always the one from the parent scope. Since the function is run as a callback asynchronously, it is always the value of the last loop iteration.
Change it to
(function(i) {
return function(err,res){
cache.set(replies[i], parseInt(res[0]));
logger.info(res[0])
};
})(i)
to bind i to the inner-most function's scope
further explanation: JavaScript closure inside loops – simple practical example
I feel this should be answered somewhere in the internet but I failed to find it, maybe because I'm not searching the correct terms but this is the problem: I have the following function:
function ParentFunction (DataBase, Parameters) {
for (k = 0; k < DataBase.length; k++){
var CalendarURL = "https://www.google.com/calendar/feeds/" + DataBase.cid;
$.ajax({
url: CalendarURL,
dataType: 'json',
timeout: 3000,
success: function( data ) { succesFunction(data, k, Parameters);},
error: function( data ) { errorFunction ("Error",Parameters); }
});
}
}
I was getting errors in succesFunction(data, k, Parameters) because 'k' was always evaluated with the latest value. What is happening is that, when the for loop runs k is correctly increased but, when the callback function successFunction was executed, typically several ms after the loop was finished, it was always been evaluated with the last value of k, not the value of the loop the $.ajax was called.
I fixed this by creating another function that contains the ajax call. It looks like this:
function ParentFunction (DataBase, Parameters) {
for (k = 0; k < DataBase.length; k++){
var CalendarURL = "https://www.google.com/calendar/feeds/" + DataBase.cid;
AjaxCall(CalendarURL, k, Parameters);
}
}
function AjaxCall(URL, GroupIndex, Parameters) {
$.ajax({
url: URL,
dataType: 'json',
timeout: 3000,
success: function( data ) { succesFunction(data, GroupIndex, Parameters);},
error: function( data ) { errorFunction ("Error",Parameters); }
});
}
and it works. I think when the function is called in the parentFunction a copy of the value of the arguments is created and when the callback executes sees this value instead of the variable k which by the time would have a wrong value.
So my question is, is this the way to implement this behaviour? Or is there more appropriate way to do it? I worry that either, different browsers will act differently and make my solution work in some situations and not work in others.
You are hitting a common problem with javascript: var variables are function-scoped, not block-scoped. I'm going to use a simpler example, that reproduces the same problem:
for(var i = 0; i < 5; i++) {
setTimeout(function() { alert(i) }, 100 * i);
}
Intuitively, you would get alerts of 0 through 4, but in reality you get 5 of 5, because the i variable is shared by the whole function, instead of just the for block.
A possible solution is to make the for block a function instead:
for(var i = 0; i < 5; i++) {
(function(local_i) {
setTimeout(function() { alert(local_i); }, 100 * i);
})(i);
}
Not the prettiest or easier to read, though. Other solution is to create a separate function entirely:
for(var i = 0; i < 5; i++) {
scheduleAlert(i);
}
function scheduleAlert(i) {
setTimeout(function() { alert(i); }, 100 * i);
}
In the (hopefully near) future, when browsers start supporting ES6, we're going to be able to use let instead of var, which has the block-scoped semantics and won't lead to this kind of confusion.
Another option – rather than creating a new named function – would be to use a partial application.
Simply put, a partial application is a function that accepts a function that takes n arguments, and m arguments that should be partially applied, and returns a function that takes (n - m) arguments.
A simple implementation of a left-side partial application would be something like this:
var partial = (function() {
var slice = Array.prototype.slice;
return function(fn) {
var args = slice.call(arguments,1);
return function() {
return fn.apply(this, args.concat(slice.call(arguments)));
}
}
}).call();
With this, then you can take a function that requires two arguments like:
function add(a,b) { return a + b; }
Into a function that requires only one argument:
var increment = partial(add, 1);
increment(1); // 2
increment(10); // 11
Or even a function that requires no arugments:
var return10 = partial(add, 5, 5);
return10(); // 10
This is a simple left-side only partial application function, however underscore.js provides a version that can partially apply an argument anywhere in the argument list.
For your example, instead of calling AjaxCall() to create a stable variable scope, you could instead do:
function ParentFunction (DataBase, Parameters) {
for (k = 0; k < DataBase.length; k++){
var CalendarURL = "https://www.google.com/calendar/feeds/" + DataBase.cid;
var onSuccess = _.partial(succesFunction, _, k, Parameters);
$.ajax({
url: CalendarURL,
dataType: 'json',
timeout: 3000,
success: onSuccess,
error: function( data ) { errorFunction ("Error",Parameters); }
});
}
}
Here, we are using _.partial() to transform a function with a signature of:
function(data, index, params) { /* work */ }
into a signature of:
function(data) { /* work */ }
Which is the signature that the success callback will actually be invoked with.
Though admittedly, this is all pretty much just syntactical sugar for the same underlying concepts already described, it can sometimes conceptually help to think about problems like these from as functional perspective than procedural one.
This has to do with closures in javascript. Your anonymous functions each reference a variable outside of their current scope, so each function's "k" is bound to the original looping variable "k." Since these functions are called some time after, each function looks back to see that "k" is sitting at its last value.
The most common way to get around this is exactly what you did. Instead of using "k" in a nested function definition (which forces a closure), you pass it as an argument to an external function, where no closure is needed.
Here are a few posts with similar issues:
How do JavaScript closures work?
JavaScript closure inside loops – simple practical example
Javascript infamous Loop issue?
so i have to calculate some share in a loop. In every iteration of that loop i have to get a variable called rent from an array. So i devided the calculate function from the database stuff.
var calculate = function() {
while(count < 100) {
var share = 50;
var shareArray = [];
for(var i = 0; i < 100; i++) {
var pension = share*2; // mathematical stuff
// Gets a rent from a database and returns it in a callback
getRent(modules, share, function(rent) {
share = rent*foo; // some fancy mathematical stuff going on here
// I need to get the share variable above out of its function scope
});
// I need the share variable right here
shareArray.push(share); // the value of share will be for i = 0: 50, i= 1: 50 ...
// This is not what i want, i need the share value from getRent()
}
count++;
}
}
Now as you may see i am presented with the following trouble. Because I'm working in node.js, the only way to get the rent variable from the modules array is through this callback function called getRent(). The thing is, i need the share value after this step but outside of getRent().
Is there any way i can do this?
This is the getRent() - Function:
var getRent = function(modules, share, callback) {
// Searching for a fitting rent in the modules array
// Just assume this is happening here
callback(rent);
};
So the question is: How can i "return" share:
getRent(modules, share, function(rent) {
share = rent*foo; // some fancy mathematical stuff going on here
// I need to get the share variable above out of its function scope
});
in any way?
If getRent is async there's no way to get the result synchronously. Fundamentally you don't know the value that getRent will end up supplying to its callback until it finally returns it. So it isn't a question of function scope, its a question of timing. You just have to wait for getRent to do its thing before you can get the value for rent. You need to refactor your code so that calculate is also async.
Something like:
// Refactor calculate to be async:
function calculate(cb) {
var data = [];
for ( var i=0; i<100; i++ ) {
getRent(function (rent) {
data.push(rent);
if ( data.length === 100 ) cb(data);
});
}
}
// And then use it async:
calculate(function (data) {
// data array arrives here with 100 elements
});
The above answer is perhaps similar to how you might achieve it with vanilla JS. Using the async library like miggs suggests is probably a good idea in the long run. But like I said, if you use vanilla JS, or the async lib, there's no getting away from the fact that you'll have to refactor towards asynchronicity both in this code and the code that calls it.
You want to use the whilst method of the async library (npm install async) to simplify this:
var count = 0;
var shareArray = [];
async.whilst(
function () {
return count < 100;
},
function (next) {
count++;
getRent(function(rent) {
// What does modules do anyway??
// Dont know where foo comes from...
shareArray.push(rent*foo); // some fancy mathematical stuff going on here
next();
});
},
function (err) {
console.log(shareArray);
// Do sth. with shareArray
}
);
If it was OK for you to request all 100 call in parallel, you could also use the parallel function.
The code might look more complicated than it needs to be I want to pass i into the balm function but it returns undefined because I'm doing it wrong.
Just extra information: This is for the server of a game I'm writing. Running in console for node.
for (i=30;i>=0;i--){
setTimeout(function balm(i){
this_sql ="UPDATE game_moblist SET hp = least(max_hp, hp +"+Math.round(i/2)+") WHERE id ="+mobid
connection.query(this_sql, function(err, rows, fields) {if (err) err=null});
console.log(this_sql)
this_sql ="SELECT hp, max_hp FROM game_moblist WHERE id ="+mobid; //emite catch all update pointint to hp..
connection.query(this_sql, function(err, rows, fields) {if (err) throw err;
socket.emit ('updatemisc',handler,rows);//eval handler using args
if (rows[0].hp==rows[0].max_hp){
i=0;
return i;
}
});
},(31-i)*333);
}
here is a simplified version just showing the concept
for (i=3;i>=0;i--){
setTimeout(function foo(i){
console.log(foo)
},1000*i);
I would expect the following output
"1" after 1000 ms
"2" after 2000 ms
"3" after 3000 ms
EDIT: It worked when I defined the function outside of setTimeout() and then called it like this
setTimeout(balm(i),...
You can't use a loop variable i declared outside the callback function and expect it to have the right value once the callback is actually executed - it'll have the last value assigned to it instead.
The code below shows the simplest (but not the shortest) solution:
function showNumber(n) {
return function() {
console.log(n);
}
}
for (i = 3; i >= 0; i--) {
setTimeout(showNumber(i), 1000 * i);
}
In other words, you call a function (that has its parameter "bound" to your loop variable) which then returns another function that is the one actually invoked by setTimeout().
There are other ways of doing this, typically with an Immediately Invoked Function Expression as shown in #Xander's answer, but the code above demonstrates the solution nicely.
i is 0 when the first callback executes and stays that way through the rest of the calls.
You could create a closure to capture the value of i at the moment of declaration:
for (i = 3; i >= 0; i--){
function(x) {
setTimeout(function foo(i){
console.log(i)
},1000 * x);
})(i);
}
Variables can not be passed into a function in its declaration.
for (i=3; i>=0; i--) {
fooLoop(i);
}
function fooLoop(iterator) {
setTimeout(function () {
console.log("value of i is" + iterator);
}, 1000*iterator);
}
Consider the following JavaScript code which has a recursive boom function
function foo() {
this.arr = [1, 2, 3, 4, 5];
this.output = "";
this.get_something = function(n, callback) {
this.output += "(" + n + ")";
$.get("...", function(result) {
// do something with 'result'
callback();
});
};
this.boom = function() {
if (this.arr.length > 0) {
this.get_something(this.arr.shift(), this.boom);
}
};
this.boom();
document.write("output = [" + this.output + "]");
}
var f = new foo();
When get_something is executed for the first time and callback() is called, this.arr is not available anymore (probably because this is pointing to something else now).
How would you solve this ?
The problem is in Ajax requests and function call context
The problem you're having is function call context when boom gets called after you've received data from the server. Change your code to this and things should start working:
this.get_something = function(n, callback) {
var context = this;
this.output += "(" + n + ")";
$.get("...", function(result) {
// do something with 'result'
callback.call(context);
});
};
Additional suggestion
I know that your array has only few items but this code of yours is probably just an example and in reality has many of them. The problem is that your recursion will be as deep as your array has elements. This may become a problem because browsers (or better said Javascript engines in them) have a safety check of recursion depth and you will get an exception when you hit the limit.
But you will also have to change your document write and put it in the boom function since these calls are now no more recursive (it they were at all in the first place).
this.boom = function() {
if (this.arr.length > 0) {
this.get_something(this.arr.shift(), this.boom);
}
else {
document.write("output = [" + this.output + "]");
}
};
After some deeper observation
Your code actually doesn't execute in recursion unless you use synchronous Ajax requests which is by itself a show stopper and everyone would urge you to keep the async if at all possible. So basically my first solution with providing function call context would do the trick just fine.
If you're using async Ajax requests your document.write shouldn't display correct result, because it would be called too early. My last suggested boom function body would solve this issue as well.
this.get_something(this.arr.shift(), this.boom.bind(this));
Now you've bound this to be this. The context inside the boom function will always be what you expect it to be.
.bind is not supported in non-modern browsers so use the compatibility implementation.
As a side note $.proxy can do most of what .bind offers. So it may be simpler for you to use
this.get_something(this.arr.shift(), $.proxy(this.boom.bind, this));
.bind is currently supported by IE9, FF4 and Chrome.
.bind, jQuery.proxy