Develop Reference

Control Flow. Promise Logic. How to deviate without the deviation eloping with the control flow, leaving behind basically just an IOU (Promise)?

I'm having trouble understanding control flow with asynchronous programming in JS. I come from classic OOP background. eg. C++. Your program starts in the "main" -- top level -- function and it calls other functions to do stuff, but everything always comes back to that main function and it retains overall control. And each sub-function retains control of what they're doing even when they call sub functions. Ultimately the program ends when that main function ends. (That said, that's about as much as I remember of my C++ days so answers with C++ analogies might not be helpful lol).
This makes control flow relatively easy. But I get how that's not designed to handle event driven programming as needed on something like a web server. While Javascript (let's talk node for now, not browser) handles event-driven web servers with callbacks and promises, with relative ease... apparently.
I think I've finally got my head around the idea that with event-driven programming the entry point of the app might do little more than set up a bunch of listeners and then get out of the way (effectively end itself). The listeners pick up all the action and respond.
But sometimes stuff still has to be synchronous, and this is where I keep getting unstuck.
With callbacks, promises, or async/await, we can effectively build synchronous chains of events. eg with Promises:
doSomething()
.then(result => doSomethingElse(result))
.then(newResult => doThirdThing(newResult))
.then(finalResult => {
console.log(`Got the final result: ${finalResult}`);
})
.catch(failureCallback);
});
Great. I've got a series of tasks I can do in order -- kinda like more traditional synchronous programming.
My question is: sometimes you need to deviate from the chain. Ask some questions and act differently depending on the answers. Perhaps conditionally there's some other function you need to call to get something else you need along the way. You can't continue without it. But what if it's an async function and all it's going to give me back is a promise? How do I get the actual result without the control flow running off and eloping with that function and never coming back?
Example:
I want to call an API in a database, get a record, do something with the data in that record, then write something back to the database. I can't do any of those steps without completing the previous step first. Let's assume there aren't any sync functions that can handle this API. No problem. A Promise chain (like the above) seems like a good solution.
But... Let's say when I call the database the first time, the authorization token I picked up earlier for it has expired and I have to get a new one. I don't know that until I make that first call. I don't want to get (or even test for the need for) a new auth token every time. I just want to be able to respond when a call fails because I need one.
Ok... In synchronous pseudo-code that might look something like this:
let Token = X
Step 1: Call the database(Token). Wait for the response.
Step 2: If response says need new token, then:
Token = syncFunctionThatGetsAndReturnsNewToken().
// here the program waits till that function is done and I've got my token.
Repeat Step 1
End if
Step 3: Do the rest of what I need to do.
But now we need to do it in Javascript/node with only async functions, so we can use a promise (or callback) chain?
let Token = X
CallDatabase(Token)
.then(check if response says we need new token, and if so, get one)
.then(...
Wait a sec. That "if so, get one" is the part that's screwing me. All this asynchronicity in JS/node isn't going to wait around for that. That function is just going to "promise" me a new token sometime in the future. It's an IOU. Great. Can't call the database with an IOU. Well ok, I'd be happy to wait, but node and JS won't let me, because that's blocking.
That's it in a (well, ok, rather large) nutshell. What am I missing? How do I do something like the above with callbacks or Promises?
I'm sure there's a stupid "duh" moment in my near future here, thanks to one or more of you wonderful people. I look forward to it. 😉 Thanks in advance!

What you do with the .then call is to attach a function which will run when the Promise resolves in a future task. The processing of that function is itself synchronous, and can use all the control flows you'd want:
getResponse()
.then(response => {
if(response.needsToken)
return getNewToken().then(getResponse);
})
.then(() => /* either runs if token is not expired or token was renewed */)
If the token is expired, instead of directly scheduling the Promise returned by .then, a new asynchronous action gets started to retrieve a new token. If that asynchronous action is done, in a new task it'll resolve the Promise it returns, and as that Promise was returned from the .then callback, this will also then resolve the outer Promise and the Promise chain continues.
Note that these Promise chains can get complicated very quick, and with async functions this can be written more elegantly (though under the hood it is about the same):
do {
response = await getResponse();
if(response.needsToken)
await renewToken();
} while(response.needsToken)

Fist of all, I would recommend against using then and catch method to listen to Promise result. They tend to create a too nested code which is hard to read and maintain.
I worked a prototype for your case which makes use of async/await. It also features a mechanism to keep track of attempts we are making to authenticate to database. If we reach max attempts, it would be viable to send an emergency alert to administrator etc for notification purposes. This avoid the endless loop of trying to authenticate and instead helps you to take proper actions.
'use strict'
var token;
async function getBooks() {
// In case you are not using an ORM(Sequelize, TypeORM), I would suggest to use
// at least a query builder like Knex
const query = generateQuery(options);
const books = executeQuery(query)
}
async function executeQuery(query) {
let attempts = 0;
let authError = true;
if (!token) {
await getDbAuthToken();
}
while (attemps < maxAttemps) {
try {
attempts++;
// call database
// return result
}
catch(err) {
// token expired
if (err.code == 401) {
await getDbAuthToken();
}
else {
authError = false;
}
}
}
throw new Error('Crital error! After several attempts, authentication to db failed. Take immediate steps to fix this')
}
// This can be sync or async depending on the flow
// how the auth token is retrieved
async function getDbAuthToken() {
}

Keeping Node function parameters safe in an async environment

This is something that has been bothering me and I cant seem to find a straight answer.
Here is a form of Node function that I use a lot, it handles a web request and does a bit of IO:
function handleRequest(req, res) {
doSomeIo()
.then(function(result) {
res.send(result)
})
}
This function gets called and the res parameter is set to the current response object and it goes of into the land of IO, while its playing out there a second request comes through and sets the res to a new object? Now the first request comes back from IO, and uses the res object, is this now the first instance or the second i.e. does node essentially make a separate copy of everything each time the handleRequest function is called, with its own parameter values or is there only one instance of it and its parameters? Are the parameters in the the above function safe in an async environment or would it be better to do something like this:
function handleRequest(req, res) {
doSomeIo()
.then(function(res) {
return function(result) {
res.send(result)
}
}(res))
}
Or am I just completely ignorant of how Node and Java Script works, which seems pretty likely.

You don't have to worry at all about this case. The req object contains information about the HTTP request that the server received. Everything is sandboxed in per request basis. This will be helpful for you: What are "res" and "req" parameters in Express functions?

You can expect the current req and res object to remain the same among multiple events (I/O responses are essentially events) unless you do something to them, or you have other middleware that does something. There's no need to do anything like your second code snippet.
If you're new to JavaScript and/or the concept of closures, that's probably why you're uneasy with the syntax.

Each call to the function handleRequest() will not use the same variable values of previous calls to handleRequests(). An easy way to see this behavior would be to log the number of times the method was called, and the value of an incrementer in handleRequest().
Example:
In app.js (or whatever js file you initialize your server in), add the following:
var app = express(),
calls = 0; app.set('calls', calls);
Then in your handler add the following:
function handleRequest(req, res) {
doSomeIo()
.then(function(res) {
req.calls++;
return function(result) {
res.send(req.calls)
}
}(res))
}
You should notice that each call to the endpoint, no matter how quickly you make each call, count increases by 1 each time (get ready for race conditions).

Q library (javascript) - handling q.reject() in a promise with Q.all()

I'm building a blog site w/Express, and using Q for the first time, and I was hoping to tap into the knowledge of veteran Q users.
I'm making one request to my DB to load post data, and another request that hits the Instagram API (unless it's already cached) and returns some json. So I have something like:
Q.all([blogPromise, instagramPromise]).then(good, bad);
The issue/question I'm running into is that say my request fails in my instagramPromise and I call deferred.reject(), my bad function is called. However, I still want to load the page with the blog post data if my blogPromise resolves, but it seems I'm not getting any arguments when my bad function is called (e.g. I don't get the blogPromise data that was successfully fetched).
Given this, it seems my only option is to not call deferred.reject() when I have an error, and instead call deferred.resolve() with something like deferred.resolve({error: true}) which I can then use in my good function to handle what gets passed to my view.
So my question is, does this sound right? Is this not a misuse of Q using resolve when in fact I'm running into an error and should be using reject? Or am I missing something with Q that would allow a better approach?

If you want your promise to resolve when both blogPromise and instagramPromise either resolves or rejects, you need to use allSettled method. Here is an example from the documentation:
Q.allSettled([blogPromise, instagramPromise])
.then(function (results) {
var loaded = results.filter(function (result) {
return result.state === "fulfilled";
});
good(loaded);
});
Inside of allSettled's then callback you can filter successfully loaded results and pass them to the good function. Or handle failed results somehow with bad one.

Something like this perhaps?
Q.all([
blogPromise,
instagramPromise.catch(function() { return {error: true}; })
]).then(good, bad);
It's similar to the approach you mention, with the difference that the error suppression is done in the place where it's used, rather than in the place where the error originates.

How does next() work in Node.js?

I recently read a blog in Nodejitsu and I am wondering how this piece of code works.
var fs = require('fs'),
http = require('http'),
httpProxy = require('../lib/node-http-proxy');
module.exports = function (logging) {
// Code here is run when the middleware is initially loaded.
var logFile = fs.createWriteStream('./requests.log');
return function (request, response, next) {
// Code here is run on each request.
if (logging) {
logFile.write(JSON.stringify(request.headers, true, 2));
}
next();
}
}
And the explanation given for this piece of code is:
This middleware is for very simple logging - it will write the headers of each request to a log file.
the above module exported can be used as,
httpProxy.createServer(
require('./example-middleware')(true),
8000, 'localhost'
).listen(9000)
How is the code in the above method with next() invoked in every request? The usage is pretty simple: require the above module and it gets invoked every time.

I'll simplify the actual process but the gist of it:
When a request comes in, node passes the request and response object to the first middleware in the middleware stack. If that middleware sends a response or closes the connection in any way, then subsequent middleware are not called. Otherwise, that middleware has to tell node it's finished doing it's job to keep on moving through the middleware stack, so you call next() within your middleware to tell it to continue processing middleware.

Okay, so this is a pretty common thing. What this module contains is a single function, specified by this line, where we set module.exports to a function, module.exports = function (logging) {. The function returned by the module (and therefore returned by require()) returns another function, which is the middleware for the HTTP proxy (this middleware allows you to transforms the request). This middleware function gets called for every HTTP request made to the server. Quickredfox's answer provides a fairly good explanation of middlewares.
So the require('./example-middleware')(true) actually calls the function assigned to module.exports, but does not call the function inside that, which is returned immediately and passed as a middleware into the httpProxy.createServer function. This is a good way to set up some options for your middleware using closures. If you have any more questions, feel free to comment. :D

Extract Data from Anonymous Function Scope

Because of the complexity of this application, I have a need to wrap Facebook API calls, like so.
//In main file, read is always undefined
var read = fb_connect.readStream();
// In fb_wrapper.js
function readStream () {
var stream;
FB.api('/me/feed', {limit:10000}, function (response) {
stream = response.data;
});
return stream;
}
I know that due to the asynchronous nature of the call, the rest of the readStream() function will return stream (which has no value). I am having trouble finding a way of getting the data out of the callback function scope and back up to a higher scope. The FB API call is returning fine (I have debugged it a hundred times), but getting that response data has been the battle thus far.
If anyone has any suggestions, it would be much appreciated. I searched for Facebook jQuery plug-ins (as a pre-made wrapper, perhaps) with little luck.

Judging from your question, it seems that you are looking for a synchronous call. Which means that you'd want to use the data returned from the api call right after calling it. In that case, you'll need to check whether FB.api supports synchronous calls (mostly doesn't).
Otherwise, you'll need to understand that you are making an async call here. Which means that you should put your handling code INSIDE the callback function that you pass to FB.api. This is called the "continuation" style of writing code and is the standard way to use async calls.
FB.api('/me/feed', {limit:10000}, function (response) {
var stream = response.data;
// Do your processing here, not outside!!!
});
Or:
function handlerFunction(response) {
// Do your processing here
}
FB.api('/me/feed', {limit:10000}, handlerFunction);