I'm coming from a PHP background and I'm trying to learn NodeJS.
I know that everything in Node is async but I've found that i've been using the async / await combo quite a lot in my code and I wanted to make sure I wasn't doing something stupid.
The reason for it is that I have a lot of situations where I need the result of something before continuing (for example for a small ajax request). This small ajax request has no other purpose other than to do the set of things that I want it to and in the order that I specify so even if I do things the "async way" and write it using a callback i'm still having to wait for things to finish in the right order.
Right now whenever I find myself in this situation I just use await to wait for the result:
ie:
var result = await this.doSomething(data);
Opposed to using a callback
this.doSomething(data, function(callback) {
// code
callback();
});
To me the first example looks cleaner than the second one which is why I've been opting for that. But I'm worried that I might be missing something fundamental here. But in a situation where there is nothing else to process below the async call and the only way for things to progress is for it to follow a syncronous style, is there anything wrong with using the first style over the second?
But I'm worried that I might be missing something fundamental here.
Nope, you're not, that's exactly what you want to do, assuming this.doSomething(data) is asynchronous (and if it's an ajax call, one hopes it is async) and that it returns a promise (which all functions defined with the async keyword do implicitly). (If it's not asynchronous, you don't need the await, although it's allowed.)
You're probably being a bit confused (understandably) by the fact that things are going through a transition. Until recently, the overwhelming convention in Node APIs (the built-in ones and ones provided by third-party modules) was to use the "Node callback" pattern, which is that a function that will do asynchronous work expects its last argument to be a callback, which it will call with a first argument indicating success/failure (null = success, anything else is an error object) with subsequent arguments providing the result. (Your second example assumes doSomething is one of these, instead of being a function that returns a promise.)
Example: fs.readFile, which you use like this:
fs.readFile("/some/file", "utf-8", function(err, data) {
if (err) {
// ...handle the fact an error occurred..
return;
}
// ...use the data...
});
This style quickly leads to callback hell, though, which is one of the reasons promises (aka "futures") were invented.
But a lot of Node APIs still use the old pattern.
If you need to use "Node callback"-pattern functions, you can use util.promisify to create promise-enabled versions of them. For instance, say you need to use fs.readFile, which uses the Node callback pattern. You can get a promise version like this:
const readFilePromise = util.promisify(fs.readFile);
...and then use it with async/await syntax (or use the promise directly via then):
const data = await readFilePromise("/some/file", "utf-8");
There's also an npm module called promisify that can provide a promise-ified version of an entire API. (There's probably more than one.)
Just because promises and async/await replace the old Node callback style in most cases doesn't mean callbacks don't still have a place: A Promise can only be settled once. They're for one-off things. So callbacks still have a place, such as with the on method of EventEmitters, like a readable stream:
fs.createReadStream("/some/file", "utf-8")
.on("data", chunk => {
// ...do something with the chunk of data...
})
.on("end", () => {
// ...do something with the fact the end of the stream was reached...
});
Since data will fire multiple times, it makes sense to use a callback for it; a promise wouldn't apply.
Also note that you can only use await in an async function. Consequently, you may find yourself getting into the habit of a "main" module structure that looks something like this:
// ...Setup (`require` calls, `import` once it's supported, etc.)...
(async () => {
// Code that can use `await `here...
})().catch(err => {
// Handle the fact an error/promise rejection occurred in the top level of your code
});
You can leave the catch off if you want your script to terminate on an unhandled error/rejection. (Node doesn't do that yet, but it will once unhandled rejection detection matures.)
Alternately, if you want to use a promise-enabled function in a non-async function, just use then and catch:
this.doSomething()
.then(result => {
// Use result
})
.catch(err => {
// Handle error
});
Note: async/await is directly supported in Node 7.x and above. Be sure your target production environment supports Node 7.x or above if your'e going to use async/await. If not, you could transpile your code using something like Babel and then use the transpiled result on the older version of Node.
Related
If you use promise's then() syntax, you can truly run codes concurrently.
See this example.
async function asyncFunc() {
return '2';
}
console.log('1')
asyncFunc().then(result => console.log(result))
console.log('3')
this prints 1 -> 3 -> 2 because asyncFunction() runs in "async" and rest of code ('3') runs without delay.
But if you use async/await syntax, doesn't it work like plain ordinary "sync" code?
async function asyncFunc() {
return '2';
}
console.log('1')
const result = await asyncFunc()
console.log(result)
console.log('3')
this would print 1->2->3 because it "defers" rest of the code to "await" asyncFunction()
So I'm curious about the purpose of async/await syntax. I guess it allows you to use async functions like sync function. With async/await, I can easily see how my code runs because it has the same flow as plain sync code. But then, why not just use sync code in the first place?
But I'm sure I'm missing something here right? What is the purpose of async/await syntax?
The idea of async/await is as follows:
It is a syntactic sugar for promises. It allows to use async code (like Web Api's - AJAX, DOM api's etc.) in synchronous order.
You ask "But then, why not just use sync code in the first place?" - because in your example you forced a sync code to be returned in a promise, basically you made it async, but it could run as sync in the first place, but there are Web APIs as mentioned above, that work ONLY ASYNC (meaning you have to wait for them to be executed), that's why you cannot turn them in sync "in the first place" :)
When JavaScript encounters the await expression, it pauses the async function execution and waits until the promise is fulfilled (the promise successfully resolved) or rejected (an error has occurred), then it continues the execution where it pauses in that function... The promise can be code from JS engine (as in your case) or it can be from another place like browser apis (truly async).
Is it's possible to rewrite next code with try-catch? I have heard that modern JS support it.
{
// GET /someUrl
this.$http.get('/someUrl').then(response => {
// get body data
this.someData = response.body;
}, response => {
// error callback
});
}
Yes, by try catch I assume you are referring to async/await introduced in ES6. It's worth bearing in mind that async/await only works on promise objects and performs exactly the same function but just provides you a more synchronous way to write your code that is (in most situations) easier to read.
Using async await the promise will resolve and return that value and then proceed to the next line of code, any exceptions can be handled in a catch block like so:
const someFunc = async () => {
try {
const response = await this.$http.get('/someUrl');
console.log(response);
} catch(err){
// Error handling
throw new Error(err);
}
}
While many may point you to async/await so you can use the actual try/catch keywords, I think it's more important to understand the code you have and make an informed decision. Most importantly, promises already handle the try/catch for you. Lets dive in.
In this code, you are dealing with promises.
goDoSomething().then(callThisFunctionWithResultsWhenDone);
One of my favorite properties of Promises are that they already handle
try/catch for you.
goDoSomething()
.then(doSomethingThatThrows)
.catch(doSomethingWithYourError)
.then(otherwiseKeepGoing)
Notice I used the .catch instead of passing an error handler in the same line. The inline error handler doesn't do what you may expect so I avoid it. Using .catch() allows you to intelligently handle asynchronous errors by short-circuiting a series of operations or handling errors from a series of steps individually without breaking out of the pipeline.
goDoSomething()
.then(throwError)
.then(thisIsSkipped)
.then(thisToo)
.catch(handleErrorButDontRethrow)
.then(doThisWithOrWithoutError)
.then(finishUp)
I always like to think of this style as a pipeline. Each catch statement catches any errors above it going up to the previous catch.
So, in your case:
$http.get(...)
.catch(handleRequestError)
.then(process)
.catch(handleProcessingError)
This would handle all possible errors but usually to handle the request error you actually would skip the processing step so then:
$http.get(...)
.then(process)
.catch(handleProcessingError)
Likewise, whatever calls this block may just want to skip the next block if the processing failed. Then you are left with:
$http.get(...)
.then(process)
If you find yourself re-throwing then maybe you should remove the catch and let it bubble.
IMO this is much more idiomatic JS code. The style introduced with async/await is a bit more familiar if you learned other languages first but it brings a very imperative feeling with it. In order to have the flexibility to handle each async operation individually your async/await code would need to look like this:
let intermediateStateStorage;
try {
intermediateStateStorage = await goDoSomething();
} catch {
handleThis();
}
try {
intermediateStateStorage = await process(intermediateStateStorage);
} catch {
handleThisOne();
}
This seems verbose compared to:
goDoSomething()
.catch(handleThis)
.then(process)
.catch(handleThisOne)
Neither are bad and I obviously have preference but I thought you'd like to know what you are dealing with so you can make an informed decision.
I have a background process written in node.js that is using the EventEmitter like this:
var event = { returnValue: undefined };
eventEmitter.emit('name', event, argument);
return event.returnValue; // Example of doing things with event.returnValue.
Other end of the event:
eventEmitter.on('name', (event, argument) => {
var returnValue = await asyncMethod(); // <- This method returns a Promise, so I could use await if possible or some other solution.
event.returnValue = returnValue;
});
I've tried to find an elegant solution to use asynchronous methods, but since EventEmitter does not support asynchronous functions at all, I can't really use await in the function or something similar.
One solution could be to set returnValue to a Promise, but that quickly becomes impractical, specially since some of my functions iterate through multiple asynchronous functions.
I need all the code to complete properly before the first block of code continues and tries to use the event.returnValue.
Is my only solution to use some third party "synchronize", or hasn't anyone at node.js thought of this problem?
There isn't really any problem for me if the thread is blocked while executing, since this is a background process without any GUI.
Since it doesn't really seem to exist any intuitive way to wait for a Promise to resolve synchronously without callbacks, or to use the EventEmitter with async callbacks, I wrote my own simple EventEmitter containing an array, and custom on and emit methods.
It handles asynchronous functions used as parameters properly so I can use it like this:
asyncEmitter.on('name', async (event, argument) => {
event.argument = await asyncMethod(); // Nice and clean code!
});
Of course I could develop it further create a once function and so on, but for me I really only depend on on and emit.
I wonder why the node team hasn't solved this yet?
Could you please help me to understand javascirpt async hell?
I think I am missing something important ☹ The thing is that js examples and most of the answers on the internet are related to just one part of code – a small snippet. But applications are much more complicated.
I am not going write it directly in JS since I am more interested of the design and how to write it PROPERLY.
Imagine these functions in my application:
InsertTestData();
SelectDataFromDB_1(‘USERS’);
SelectDataFromDB_2(‘USER_CARS’,’USERS’);
FillCollections(‘USER’,’USER_CARS’);
DoTheWork();
DeleteData();
I did not provide any description for the functions but I think it is obvious based on names. They need to go in THIS SPECIFIC ORDER. Imagine that I need to run a select into the db to get USERS and then I need run a select to get USER_CARS for these USERS. So it must be really in this order (consider the same for other functions). The thing is that need to call 6 times Node/Mysql which is async but I need results in specific order. So how can I PROPERLY make that happen?
This could work:
/* not valid code I want to present the idea and keep it short */
InsertTestData(
Mysql.query(select, data, function(err,success)
{
SelectDataFromDB_1(‘USERS’); -- in that asyn function I will call the next procedure
}
));
SelectDataFromDB_1 (
Mysql.query(select, data, function(err,success)
{
SelectDataFromDB_2(‘USERS’); -- in that asyn function I will call the next procedure
}
));
SelectDataFromDB_2 (
Mysql.query(select, data, function(err,success)
{
FillCollections (‘USERS’); -- in that asyn function I will call the next procedure
}
));
etc..
I can “easily” chain it but this looks as a mess. I mean really mess.
I can use some scheduler/timmers to schedule and testing if the previous procedure is done)
Both of them are mess.
So, what is the proper way to do this;
Thank you,
AZOR
If you're using a recent version of Node, you can use ES2017's async/await syntax to have synchronous-looking code that's really asynchronous.
First, you need a version of Mysql.query that returns a promise, which is easily done with util.promisify:
const util = require('util');
// ...
const query = util.promisify(Mysql.query);
Then, wrap your code in an async function and use await:
(async () => {
try {
await InsertTestData();
await SelectDataFromDB_1(‘USERS’);
await SelectDataFromDB_2(‘USER_CARS’,’USERS’);
await FillCollections(‘USER’,’USER_CARS’);
await DoTheWork();
await DeleteData();
} catch (e) {
// Handle the fact an error occurred...
}
})();
...where your functions are async functions, e.g.:
async InsertTestData() {
await query("INSERT INTO ...");
}
Note the try/catch in the async wrapper, it's essential to handle errors, because otherwise if an error occurs you'll get an unhandled rejection notice (and future versions of Node may well terminate the process). (Why "unhandled rejections"? Because async functions are syntactic sugar for promises; an async function returns a promise.) You can either do that with the try/catch shown, or alternate by using .catch on the result of calling it:
(async () => {
await InsertTestData();
// ...
})().catch(e => {
// Handle the fact an error occurred
});
I'm aware of the power of promises, however I have several old functions that are synchronous:
function getSomething() {
return someExternalLibrary.functionReturnsAValue()
}
console.log(getSomething()); // eg prints 'foo'
Unfortunately, when someExternalLibrary updated, it has removed functionReturnsAValue() and has lumped me with functionReturnsAPromise():
function getSomething() {
return someExternalLibrary.functionReturnsAPromise()
}
console.log(getSomething()); // now prints '[object]'
This of course, breaks absolutely everything written that depends on what used to be a simple value.
Obviously, I'd prefer two things:
ask the original library to keep a synchronous return value. (Not going to happen -- b/c they have refused)
A way to actually wait for a value
I have read numerous articles on why promises are great, ad nauseam, but the simple fact is: If I embrace promises, all I really do is shuffle promises onto some other part of the code, which then must deal with the promise of a value...
Is there a way (in nodejs) to actually wait for a promise to get itself together?
The best I can find is to use coroutines and yield, but really, it's still passing the buck. To be clear, I want the function getSomething to continue to return a value. Is there a way to do it?
Clearly, I fear I've misunderstood something about Promises...
The app is for non-browser implementations and runs purely from the command line. I've been trying to understand how bluebird's reflect() might help, to no avail.
(Yes, I'm aware this question has been asked many times in various formats, but I can't find a suitable answer to the core issue. If anything, I'm looking for the opposite of this question. The closest related (but unhelpful) question I can find is: Managing promise dependencies.)
There's the concept of generator functions. These are a special kind of function in both syntax (asterisk notation) and semantics. Unlike regular functions, generator functions return something that's also new to ECMAScript: iterators. Iterators happen to be objects made specifically to be iterated on, e.g. with the all new for...of loop. They can be also iterated on manually by calling their 'next' method. Each such call produces an object containing two properties: 'value' (iterator's current value) and 'done' (a boolean indicating whether we reached the last value of the iterable). However, the best thing about generator functions is their ability to suspend their execution each time a keyword 'yield' is encountered. Let's have a glimpse of how it all works together:
'use strict';
let asyncTask = () =>
new Promise((resolve, reject) => {
if (Math.random() > 0.5) {
resolve(1);
} else {
reject(new Error('Something went wrong'));
}
});
let makeMeLookSync = fn => {
let iterator = fn();
let loop = result => {
!result.done && result.value.then(
res => loop(iterator.next(res)),
err => loop(iterator.throw(err))
);
};
loop(iterator.next());
};
makeMeLookSync(function* () {
try {
let result = yield asyncTask();
console.log(result);
} catch (err) {
console.log(err.message);
}
});
The short answer
I am told repeatedly: You can't undo functions that have been promisified.
Edit: An upcoming solution
It appears that the ES2017 (although still draft), goes a long way in making promisified code easier to work with:
https://ponyfoo.com/articles/understanding-javascript-async-await
It seems that there is also a node library ready for this support too: https://github.com/normalize/mz.
Using this methodology, having apis converted to Promises won't be so bad (although it still appears that promises still poison the rest of the codebase):
const fs = require('mz/fs')
async function doSomething () {
if (await fs.exists(__filename)) // do something
}
The rest of this answer is just a general commentary on the problem.
Why we need a solution
Let's start with a sample piece of traditional synchronous code, in 3 flavours from more 'older-fashioned' to 'newer':
This is the traditional javascript way, requiring exception based programming to handle unexpected errors:
function getSomething() {
if (someproblem) throw new Error('There is a problem');
return 'foo';
}
However, adding try/ catch statements becomes very laborious and tedious, very quickly.
With the advent of node.js, callbacks were made popular, which nicely circumvented the issue, since each caller was explicitly forced to deal with error conditions in the same callback. This meant less errors in the caller's code:
function getSomething(callback) {
if (callback) {
if (someproblem)
callback(new Error('There is a problem'), null);
else
callback(null, 'foo');
}
return 'foo';
}
Then, the after some teething issues, node.js quickly proved itself for server-side communications, and people were amazed at the speed that asynchronous solutions provided. Node application frameworks like Express and Meteor grew, which focused on this.
Unfortunately, using the same callback scheme quickly became troublesome and the developers dealing in asynchronous code started using Promises in an effort to linearize the code, to make it readable, like the traditional (try/catch) code was.
The problem is that it got evangenlized too much. Everyone started thinking that Promises are the way to go. Personally, I call it a poison on a codebase. Once you have anything that uses Promises, your whole codebase must become asynchronous. This is not always a sensible nor a practical solution, IMHO.
The worst of all side effects is that the above function, even though it is completely synchronous, can be written in Promises too:
var bluebird = require('bluebird');
function getSomething() {
// IMHO, this is ridiculous code, but is increasingly popular.
if (someproblem) return Promise.reject(new Error('There is a problem'));
return Promise.resolve('foo');
}
For those who doubt this is a problem, perhaps should look at the SO question: How do I convert an existing callback API to promises?. Pay particular attention to #3, Node-style callback.
So, for anyone who cares, I would like to suggest that there needs to be a 'pill' for Promises. I urge that we need more than promises: we need results, and sometimes in a timely manner.
Take a look at the default node.js api. It does not use Promises. It also provides both synchronous and asynchronous calls to appropriate parts of the api (eg File System).
For those of you who feel tempted to downvote this answer: that is your prerogative, but there are clear issues on when Promises are not the answer, and I feel strongly that there are cases when we need to be able to re-synchronize decoupled code.
I also apologize for this 'blog-post' styled answer.