Develop Reference

how to handle asynchronously: function C depending on function B depending on function A

I have three functions that all work with data from a global object. This global object gets filled with data from a local array in one function and with data from an ajax request with the second function. The third function relies on the data in the object, so the ajax request must have been completed.
I believe I am misunderstanding callbacks. Here's what I do:
var currentCharacter = {}
// this function gets the local data and then calls the second function
function loadData(getMarvelData) {
// do things to fill currentCharacter
getMarvelData(); // this is the callback to the next function (no ?)
}
// this function performs the ajax request, then calls the third function
function getMarvelData(getGoogleMap) {
// do ajax request and add stuff to currentCharacter
getGoogleMap(); // this is the callback to the final function (no ?)
}
function getGoogleMap() {
// do Google Map related stuff with data from currentCharacter
}
I thought setting a function as an argument of another function and then executing it would make the function dependent on the other before it continues. Clearly I still misunderstand callbacks after trying to make it work for a week now. As it is, the getMarvelData function doesn't even get called because I never see the alert popup and the currentCharacter object only has data from the loadData function.
Could somebody show the correct approach for my code, or if my approach of making these three functions is even the right one for this scenario.
The full repository is available at: https://github.com/ChaMbuna/Marvel-Map
V0.9 was actually working but the ajax call was set to run synchronous (it still is btw) Since then I've been overhauling my code to make it work asynchronously and to remove all jQuery (this is a Udacity project and removing jQuery was suggested by an instructor).
Appreciate the help

I have not enough reputation to put a comment, but a wild guess, you should remove the argument in loadData&getMarvelData or actually pass a function in calls to those function.

You have to pass the parameters correctly.
Try this:
var currentCharacter = {}
loadData(getMarvelData, getGoogleMap);
function loadData(f1, f2) {
// do sth.
f1(f2);
}
function getMarvelData(f2) {
// do sth.
f2();
}
function getGoogleMap() {
// do sth.
}
I havn't tested it, but it should work.

IndexedDB - Can't pass variable after onsuccess when opening database

I have been looking for assistance with setting up an IndexedDB for web storage, and I have run into a problem that I cannot find a good answer to. After I have successfully setup/opened the database I am having trouble passing the variable that contains the database information to access it later. Here is my code, I have been following a guide from MDN :
const DB_NAME = 'database-name';
const DB_VERSION = 2;
const DB_STORE_NAME = 'users';
var db;
function openDb() {
console.log('openDb ...')
var request = indexedDB.open(DB_NAME, DB_VERSION);
request.onsuccess = function(event) {
db = request.result;
console.log("openDb DONE");
};
request.onerror = function(event) {
console.log(request.errorCode);
};
request.onupgradeneeded = function(event) {
console.log('openDb.onupgradeneeded');
var store = event.currentTarget.result.createObjectStore(DB_STORE_NAME, { keyPath: 'id', autoIncrement: true });
store.createIndex('age', 'age', { unique: false });
};
}
function getObjectStore(store_name, mode) {
var tx = db.transaction(store_name, mode);
return tx.objectStore(store_name);
}
When getObjectStore is called the variable db is undefined. My knowledge of javascript is very limited and some concepts I don't get. The guide doesn't not show anything special being done and their demo works as is. Some other guides have mentioned implementing a callback, but they don't show how its done nor do I understand the concept of callbacks. Any help is appreciated. Thanks.

Unfortunately you need to learn about a relatively complicated concept usually referred to as asynchronous JavaScript before proceeding to use indexedDB. There are already several thousand questions related to AJAX on stackoverflow. I am trying to think of the most polite way to say this, but basically, the answer you are looking for is already provided by these other questions, and by many other websites. Nevertheless, here are some quick tips.
First, your approach is never going to work. You cannot skip over learning about async.
Second, do not use the setTimeout trick to get it to work. That is horrible advice.
Third, at a general level, a callback is simply a word used to describe a function when the function is used in a particular way. Specifically, a callback refers to a function that is passed as an argument to another function, where the other function then maybe calls the function at some later point in time. More specifically, a callback is generally a function that is called at the end of the function it is passed to, when the function has completed.
For example:
function a(b) { alert(b); }
function c(d) { d('hi'); }
c(a);
That might look a bit confusing at first but it is the simplest thing I can describe off the top of my head. In the example, the final line calls function c and passes in function a. The effect of the code is that you see 'hi' as a browser alert. In this example, the function a is passed as a parameter/argument to function c. Function c uses the name d for its one and only argument. c calls d with the string 'hi'. When describing this example, we would say that argument d represents a callback function passed to function c. We could also say that function a is the particular callback function used by function c. So that is basically it. When you pass a function in as an argument and the other function calls the passed in argument, you are using a callback.
Then things gets way more complicated, because you have to learn about how to read and write asynchronous code. Properly introducing it would take several pages. Here is an extreme crash course.
You have traditionally been writing synchronous code, even if you did not call it that. Synchronous code runs exactly when you expect it to, in the order that you write your statements. Here is a brief example of typical sync code:
function sum(a, b) { return a + b; }
alert(sum(1, 2));
Simple stuff. The next example is code that uses a callback, but is still synchronous.
function doOperation(op, num1, num2) { return op(num1, num2); }
function sumOperation(num1, num2) { return num1 + num2; }
var result = doOperation(sumOperation, 1, 2);
alert(result);
Here we passed the sumOperation function into the doOperation function. sumOperation is the callback function. It is the first argument with the name 'op'. Still pretty simple stuff. Now consider the next example. The point of the next example is to show how we pass control to the function to do something. Kind of like how goto/labels work.
function doOperation(op, num1, num2) {
var result = op(num1, num2);
alert(result);
return undefined;
}
function sumOperation(num1, num2) { return num1 + num2; }
doOperation(sumOperation, 1, 2);
Notice how doOperation no longer returns a value. It has the logic within its function body. So once we call doOp, the browser starts running the code inside doOperation. So we switched from the outer context into the body of the function. Also, because doOperation doesn't return anything, we cannot do anything with its return value. The logic is locked inside the body of the doOperation function. The code still works about the same, its just that now we are not returning anything from doOperation, and now the logic is inside doOperation instead of outside in the main/global context.
Now an example that uses setTimeout. This is completely unrelated to the suggestion of using setTimeout.
function doOperation(op, num1, num2) {
setTimeout(function runLater() {
var result = op(num1, num2);
alert(result);
return undefined;
}, 1000);
return undefined;
}
function sumOperation(num1, num2) { return num1 + num2; }
doOperation(sumOperation, 1, 2);
The point here is to understand that we use a callback (named runLater in this example), and that the code inside the callback does not run immediately. Therefore, we can no longer say it runs synchronously. We instead refer to the statements constituting the body of the callback function as asynchronous. So now an alert appears after 1 second. Notice how we cannot return anything from runLater. Also notice how we cannot return anything from doOperation. There is nothing to return. There is no way to get the value in the 'result' variable out of the scope of runLater. It is locked in there.
Let's try almost the same thing, but try to have runLater set a variable. Also, I am going to omit 'return undefined' because that is what every function without a explicit return statement returns.
var aGlobalResult = null;
function doOperation(op, num1, num2) {
setTimeout(function runLater() {
aGlobalResult = op(num1, num2);
}, 1000);
}
function sumOperation(num1, num2) { return num1 + num2; }
doOperation(sumOperation, 1, 2);
alert(aGlobalResult);
Hopefully you are catching on to the problem. First off, runLater does not return anything, so doOperation does not return anything, so we could not even try to do something like aGlobalResult = doOperation(...);, because that would not make any sense. Second, the result here is that you will see an alert 'undefined' because the alert statement executes prior to the statement that assigns a value to aGlobalResult. This is even though you wrote the assignment statement higher up (earlier) in the code, and the alert is later. This is the brick wall some newer developers run into right here. This is indeed confusing for some. aGlobalResult is undefined here because setTimeout does not set it until later. Even if we passed in 0 milliseconds to setTimeout, it is still 'later', meaning that the assignment happens at a later point in time, after the alert. The alert message is always going to be undefined. There is absolutely nothing you can do to avoid the way this works. Nothing. Period. Stop trying. Learn it, or give up entirely.
So, how does one typically write code that behaves or involves asynchronous stuff? By using callbacks. Which again means that you can no longer use return statements to assign values to outer scope variables. You instead want to write functions and pass around control to various functions. In other words, instead of:
function a() {}
function b() {}
function c() {}
a(); b(); c();
you write code like this:
function a(callback) {
var asdf = 1+2; // do some stuff in a
alert('a finished');
// a has now completed, call its callback function, appropriately named callback
callback();
}
function b(callback) {
var asdfasdfasdf = 3 + 4;
alert('b finished');
// call the callback
callback();
}
a( function(){ b(function() { alert('both a and b finished'); }); });
This is more formally known as continuation passing style, or CPS.
So, that is an example of the very basics of writing callback functions and basic asynchronous code. Now you can start to use indexedDB. The first thing you will notice is that the function indexedDB.open is documented as asynchronous. So, how can we use it? Like this:
var someGlobalVariable = null;
var openRequest = indexedDB.open(...);
openRequest.onsuccess = function openRequestOnSuccessCallbackFunction(event) {
// Get a reference to the database variable. If this function is ever called at some later
// point in time, the database variable is now defined as the 'result' property of the
// open request. There are multiple ways to access the result property. Any of the following
// lines works 100% the same way. Use whatever you prefer. It does not matter.
var databaseConnection = openRequest.result;
var databaseConnection = this.result;
var databaseConnection = event.target.result;
var databaseConnection = event.currentTarget.result;
// Now that we have a valid and defined databaseConnection variable, do something with it
// e.g.:
var transaction = databaseConnection.transaction(...);
var objectStore = transaction.objectStore(...);
// etc.
// DO NOT DO THE FOLLOWING, it does not work. Why? Review the early descriptions. First off
// this onsuccess callback function does not return anything. Second off this function gets
// called at some *later* point in time, who knows when. It could be a nanosecond later.
someGlobalVariable = databaseConnection;
};
Hopefully that sets you on the path.
Edit: I thought I would add a bit more of an intro. A related concept you need to learn that I did not explain clearly enough regarding control is the difference between imperative and declarative programming.
Imperative programming involves executing a series of statements in the order you wrote. You are the caller and are in control. Imperative code looks like this (fictional code):
var dbConn = dbFactory.newConnection('mydb');
var tx = dbConn.newTransaction();
var resultCode = tx.put(myObject);
if(resultCode == dbResultConstants.ERROR_PUT_KEY_EXISTS) {
alert('uhoh');
}
Declarative programming is subtly different. With a declarative approach, you write functions, and then you register (aka hook or bind) the functions to the JavaScript engine, and then at some point later, when it is appropriate, the engine runs your code. The engine is the caller and is in control, not you. Declarative programming involves callbacks and looks like this (fictional code):
dbFactory.newConnection(function onConnect(dbConn) {
dbConn.newTransaction(function onNewTransaction(tx) {
tx.put(myObject, function onPut(resultCode) {
if(resultCode == dbResultConstants.ERROR_PUT_KEY_EXISTS) {
alert('uhoh');
}
});
});
});
In this example, the only thing you called was the fictional dbFactory.newConnection function. You passed in a callback function. You did not call the callback function yourself. The engine calls the callback function. You cannot call the callback function yourself. This is kind of the whole idea behind why JavaScript engines can allow you to write asynchronous code. Because you don't get to control the order of execution of statements/functions. The engine gets to control it. All you get to do is write your functions, register them, and then start a chain of callbacks (the sole imperative line, the starting statement).
So this is why a function like getObjectStore in your question will not work. You are trying to call the function yourself, but that is backwards. You can only write a function and register it (somehow hook it up as a callback somewhere) and then the engine, not you, calls it at some later point in time.
Hopefully this is not more confusing, but you could actually write your function getObjectStore if you really wanted to by passing in the database variable to the function as its first argument. This leads to the logical next question, how to get a valid database variable to pass into the function. You cannot get one in a global context (reliably). Because the connection variable is only valid within the context of the onOpen callback function. So you would have to make your call to this function from within the onOpen function. Something like:
function getObjectStore(db, name, mode) {
var tx = db.transaction(name, mode);
var store = tx.objectStore(name);
return store;
}
var openRequest = indexedDB.open(...);
openRequest.onsuccess = function onOpen(event) {
// get the connection variable. it is defined within this (onOpen) function and open.
var db = this.result;
// call our simple imperative helper function to get the users store. only call it from
// within this onOpen function because that is the only place we can get the 'db' variable.
var usersStore = getObjectStore(db, 'users', 'readwrite');
// do something here with usersStore, inside this function only.
};

chaining javascript functions

I searched this topic on SO for more than an hour and could not find a case like mine.
I have a web app that I developed on Ruby on Rails. It is based on a form with multiple fields and buttons to execute actions.
These are examples (my real app uses more functions and more combinations of actions):
When Field1 is changed, I want to execute JS functions function1(), then function2(), then function3().
When Field2 is changed, I want to execute JS functions function2(), then function3().
When Field3 is changed, I want to execute JS function function3().
All these JS functions call specific actions defined in my controller, and they update the form using AJAX. I'd like function2() to start executing only after function1() has finished updating the form, otherwise the results will be wrong.
Because controller methods that are called by these functions send API calls to many other websites, like Google geocode or Google timezone, their response time is unpredictable, and could be anywhere from a few 100's of ms to a few seconds.
I tried this:
onchange="function1();function2();function3()"
but that did not work because function2() started executing before function1() has finished the updates via AJAX, and the same issue for function3() with regard to function2().
I also tried:
onchange="function1();setTimeout(function(){function2();setTimeout(function(){function3()},FUNCTION2_DELAY)},FUNCTION1_DELAY)"
but that did not work either because if FUNCTION1_DELAY is set too long, function2() will not start executing before FUNCTION1_DELAY has expired, whereas function1() might execute much faster than FUNCTION1_DELAY, and if FUNCTION1_DELAY is set too short, function2() will start executing before function1() has finished the updates that are needed for function2() to run. The same thing for function3() with regard to function2().
My question is : is there a smarter way to make function2() start executing only when function1() has finished updating the form via AJAX, and the same for function3() with regard to function2()?
I know I could create new controller methods where I merge these functions, like merging function1(), function2() and function3() into new_function1(), ..., etc., but that makes my code contain repetitive actions and will ultimately make it harder to maintain. So I'd rather look for a smarter way to handle this issue.

you need to use Promises.
Basically it works like that:
var MyFunction1 = new Promise(
function() {
//do stuff here
}
);
MyFunction1.then(Myfunction2 ...).then(Myfunction3 ...;
https://developer.mozilla.org/en/docs/Web/JavaScript/Reference/Global_Objects/Promise
http://api.jquery.com/promise/

setTimeout is asynchronous, so that won't work. You could call function 2 at the end of function 1, but still inside the function. Similarly for function 3:
function one(){
...logic...
function two();
}
function two(){
...logic...
function three();
}
function three(){
...logic...
}
Alternatively, you could create some sort of queue of functions, and only move to the next when the current one is finished:
var functionQueue = [
one,
two,
three
];
function next(){
var nextFn = functionQueue.shift();
if( nextFn ){
nextFn();
}
}
function one(){
...logic...
next();
}
// same for two & three
You can also use promises, which are quite powerful and make the code clearer to read, but may be overkill for this problem

How to write a node.js function that waits for an event to fire before 'returning'?

I have a node application that is not a web application - it completes a series of asynchronous tasks before returning 1. Immediately before returning, the results of the program are printed to the console.
How do I make sure all the asynchronous work is completed before returning? I was able to achieve something similar to this in a web application by making sure all tasks we completed before calling res.end(), but I haven't any equivalent for a final 'event' to call before letting a script return.
See below for my (broken) function currently, attempting to wait until callStack is empty. I just discovered that this is a kind of nonsensical approach because node waits for processHub to complete before entering any of the asynchronous functions called in processObjWithRef.
function processHub(hubFileContents){
var callStack = [];
var myNewObj = {};
processObjWithRef(samplePayload, myNewObj, callStack);
while(callStack.length>0){
//do nothing
}
return 1
}
Note: I have tried many times previously to achieve this kind of behavior with libraries like async (see my related question at How can I make this call to request in nodejs synchronous?) so please take the answer and comments there into account before suggesting any answers based on 'just use asynch'.

You cannot wait for an asynchronous event before returning--that's the definition of asynchronous! Trying to force Node into this programming style will only cause you pain. A naive example would be to check periodically to see if callstack is empty.
var callstack = [...];
function processHub(contents) {
doSomethingAsync(..., callstack);
}
// check every second to see if callstack is empty
var interval = setInterval(function() {
if (callstack.length == 0) {
clearInterval(interval);
doSomething()
}
}, 1000);
Instead, the usual way to do async stuff in Node is to implement a callback to your function.
function processHub(hubFileContents, callback){
var callStack = [];
var myNewObj = {};
processObjWithRef(samplePayload, myNewObj, callStack, function() {
if (callStack.length == 0) {
callback(some_results);
}
});
}
If you really want to return something, check out promises; they are guaranteed to emit an event either immediately or at some point in the future when they are resolved.
function processHub(hubFileContents){
var callStack = [];
var myNewObj = {};
var promise = new Promise();
// assuming processObjWithRef takes a callback
processObjWithRef(samplePayload, myNewObj, callStack, function() {
if (callStack.length == 0) {
promise.resolve(some_results);
}
});
return promise;
}
processHubPromise = processHub(...);
processHubPromise.then(function(result) {
// do something with 'result' when complete
});

The problem is with your design of the function. You want to return a synchronous result from a list of tasks that are executed asynchronously.
You should implement your function with an extra parameter that will be the callback where you would put the result (in this case, 1) for some consumer to do something with it.
Also you need to have a callback parameter in your inner function, otherwise you won't know when it ends. If this last thing is not possible, then you should do some kind of polling (using setInterval perhaps) to test when the callStack array is populated.
Remember, in Javascript you should never ever do a busy wait. That will lock your program entirely as it runs on a single process.

deasync is desinged to address your problem exactly. Just replace
while(callStack.length>0){
//do nothing
}
with
require('deasync').loopWhile(function(){return callStack.length>0;});

The problem is that node.js is single-threaded, which means that if one function runs, nothing else runs (event-loop) until that function has returned. So you can not block a function to make it return after async stuff is done.
You could, for example, set up a counter variable that counts started async tasks and decrement that counter using a callback function (that gets called after the task has finished) from your async code.

Node.js runs on A SINGLE threaded event loop and leverages asynchronous calls for doing various things, like I/O operations.
if you need to wait for a number of asynchronous operations to finish before executing additional code
you can try using Async -
Node.js Async Tutorial

You'll need to start designing and thinking asynchronously, which can take a little while to get used to at first. This is a simple example of how you would tackle something like "returning" after a function call.
function doStuff(param, cb) {
//do something
var newData = param;
//"return"
cb(newData);
}
doStuff({some:data}, function(myNewData) {
//you're done with doStuff in here
});
There's also a lot of helpful utility functions in the async library available on npm.

AJAX workflow: How do I order the execution of these functions?

I'm trying to figure the best way to get my functions executing in the correct order.
I have 3 functions
function 1 - squirts OPTIONs into a SELECT via JSON and marks them as selected
function 2 - squirts OPTIONS into a 2nd SELECT and marks them as selected
function 3 - gets the values from the above SELECTs along with some additional INPUT values, does an AJAX GET resulting in JSON data, which is read and populates a table.
With JQuery Onload, I execute:
function1();
function2();
function3();
I'm finding function3 is executing before the SELECTs have been populated with OPTIONS and hence the table has no results, because the values sent in the GET were blank.
I know this is probably a very simple problem and that there are probably a dozen ways to accomplish this, but basically I need the best way to code this so that function3 only runs if function1 and 2 are complete.
I've come into Javascript via the back door having learnt the basics of JQuery first!
Thanks for your assistance.

Javascript executes synchronously, which means that function3 must wait for function2 to complete, which must wait for function1 to complete before executing.
The exception is when you run code that is asynchronous, like a setTimeout, setInterval or an asynchronous AJAX request.
Any subsequent code that relies on the completion of such asynchronous code needs to be called in such a manner that it doesn't execute until the asynchronous code has completed.
In the case of the setTimeout, you could just place the next function call at the end of the function you're passing to the setTimeout.
In the case of an AJAX call, you can place the next function call in a callback that fires upon a completed request.
If you don't want the execution of the subsequent function to occur every time, you can modify your functions to accept a function argument that gets called at the end of the asynchronous code.
Something like:
function function1( fn ) {
setTimeout(function() {
// your code
// Call the function parameter if it exists
if( fn ) {
fn();
}
}, 200);
}
function function2() {
// some code that must wait for function1
}
onload:
// Call function1 and pass function2 as an argument
function1( function2 );
// ...or call function1 without the argument
function1();
// ...or call function2 independently of function1
function2();

I recommend you use a Promises library. You can hack simple solutions like other answers suggest, but as your application grows, you'll find you are doing more and more of these hacks. Promises are intended to solve these kinds of problems when dealing with asynchronous calls.
The CommonJS project has several Promises proposals which you should check out. Here is a question I asked on SO about Promises a while back with links to other solutions. Learn more about Promises in this Douglas Crockford video. The whole video is good, but skip to just past half way for promises.
I'm using the FuturesJS library currently as it suits my needs. But there are advantages to other implementations as well. It allows you to do sequences very easily:
// Initialize Application
Futures.sequence(function (next) {
// First load the UI description document
loadUI(next); // next() is called inside loadUI
})
.then(function(next) {
// Then load all templates specified in the description
loadTemplates(next); // next() is called inside loadTemplates
})
.then(function(next) {
// Then initialize all templates specified in the description
initTemplates();
});
Even more powerful is when you need to join async events together and do another action when all of the other async events have completed. Here's an example (untested) that will load a bunch of HTML files and then perform an action only once ALL of them have completed loading:
var path = "/templates/",
templates = ["one.html","two.html","three.html"],
promises = [];
$.each(templates, function(i,name) {
promises[i] = Futures.promise();
var $container = $("<div>");
$container.load(path+name, function(response,status,xhr) {
promises[i].fullfill();
}
});
Futures.join(promises, {timeout: 10000}) // Fail if promises not completed in 10 seconds
.when(function(p_arr) {
console.log("All templates loaded");
})
.fail(function(p_arr) {
console.log("Error loading templates");
});
This might be overkill for your application. But if the application is growing in complexity, using promises will help you in the long run.
I hope this helps!

invoke function2 inside of function1 and function3 inside of function2.

It's not clear why f1 and f2 are executing before f3.
Also, are you using the preferred $(document).ready() or some variation of onload?
It might be helpful if you provide a reproducible test case.

fun3() will only run after both are ready. It might run twice. You can fix this with a lock inside fun3() you would need a Singleton to guarantee it works correctly.
var select1ready = false, select2ready = false;
fun1()
{
// do stuff
select1ready = true;
fun3();
}
fun2()
{
// do stuff
select2ready = true;
fun3();
}
fun3()
{
if (select1ready && select2ready)
{
}
}
fun1();
fun2();