I have a strict JavaScript API naming scheme I need to follow, it looks like this:
var Items = function() {
this.items = [];
};
Items.prototype.get() {
db.items.find(function(err, items) {
this.items = items;
});
return this.items;
}
The problem is the async call (db.items.find..) that doesn't have time to finish before the get() method returns an empty this.items..
The client needs to make the calls like this:
items = new Items();
console.log(items.get());
What's best practice to handle async calls here while still strictly following the API naming scheme?
Is there some native way I can let get() wait for a return inside the callback or do I need some kind of async lib for this?
EDIT:
Apparently what you are looking for might be possible using wait.for (https://github.com/luciotato/waitfor). I have not used it though, so I am not sure how well it would suit your needs. The method you would need would be wait.forMethod.
Previous Answer:
There is no way you can write async code in a synchronous manner. Also, it is not a good idea to mix async and sync methods. You are trying to define a synchronous method Item.prototype.get, but inside that you are using an async method db.items.find, which makes Item.prototype.get an asynchronous function. In order to get this to work you will have to define Item.prototype.get as a proper async function with a callback.
Items.prototype.get(fn) {
db.items.find(function(err, items) {
return fn(err, items);
});
}
You could then call it as
items = new Items();
items.get(function(err, items){
console.log(items);
}
I managed to solve this by using SilkJS (http://www.silkjs.net/), which is similar to Node in that it is built atop of V8 but it runs in synchronous mode.
This way I managed to keep the given API spec.
Related
I am very new to NodeJS. I was trying to get a function written that can simply return a configuration value from DB. I might need to call it multiple times.
In PHP or other synchronous languages, I would use the following code for it
function getConfigValue($configKeyName)
{
// DB READ OPERATIONS
return $confguration_value_fetched_from_db
}
getConfigValue("key1");
getConfigValue("key2");
etc
But in NodeJS, I found it too difficult to do this operation because of the Asynchronous nature of the code. After asking some questions here, and after spending hours to learn Callbacks, Promises, Async/await keywords, being a beginner the below is best code I could reach.
// Below function defines the 'get' function
var get = async (key) => {
var result = await COLLECTIONNAME.findOne({key}); //MongoDB in backend
return result.value;
}
// Here I am forced to define another async function so that I can await for the get function.
function async anotherfunction()
{
var value_I_am_lookingfor1 = await get("key1");
var value_I_am_lookingfor2 = await get("key2");
}
anotherfunction();
While it might work, I am not fully happy with the result, mainly because I really don't want to do all my further coding based on the fetched value within this function anotherfunction(). All I want is to fetch a single value? Also I might need to easily call it from many places within the application, not just from here (I was planning to place it in a module)
Any better or easier methods? Or should I always get the value I want, and then nest it with a 'then.' to do the subsequent operation? I even doubt the fundamental approach I take on NodeJS coding itself may be wrong.
Can anyone guide me?
mainly because I really don't want to do all my further coding based on the fetched value within this function anotherfunction(). All I want is to fetch a single value?
Because the request is asynchronous, and your code depends on having the fetched values first, there's no option other than to wait for the values to be retrieved before continuing. Somewhere in the script, control flow needs to halt until the values are retrieved before other parts of the script continue.
Also I might need to easily call it from many places within the application, not just from here (I was planning to place it in a module)
You should have that module make the requests and export a Promise that resolves to the values needed. Rather than using await (which forces requests to be processed in serial), you should probably use Promise.all, which will allow multiple requests to be sent out at once. For example:
valuegetter.js
const get = key => COLLECTIONNAME.findOne({ key }).then(res => res.value);
export default Promise.all([
get('key1'),
get('key2')
]);
main.js
import prom from './valuegetter.js';
prom.then(([val1, val2]) => {
// do stuff with val1 and val2
})
.catch((err) => {
// handle errors
});
If other modules need val1 and val2, call them from main.js with the values they need.
I have a background process written in node.js that is using the EventEmitter like this:
var event = { returnValue: undefined };
eventEmitter.emit('name', event, argument);
return event.returnValue; // Example of doing things with event.returnValue.
Other end of the event:
eventEmitter.on('name', (event, argument) => {
var returnValue = await asyncMethod(); // <- This method returns a Promise, so I could use await if possible or some other solution.
event.returnValue = returnValue;
});
I've tried to find an elegant solution to use asynchronous methods, but since EventEmitter does not support asynchronous functions at all, I can't really use await in the function or something similar.
One solution could be to set returnValue to a Promise, but that quickly becomes impractical, specially since some of my functions iterate through multiple asynchronous functions.
I need all the code to complete properly before the first block of code continues and tries to use the event.returnValue.
Is my only solution to use some third party "synchronize", or hasn't anyone at node.js thought of this problem?
There isn't really any problem for me if the thread is blocked while executing, since this is a background process without any GUI.
Since it doesn't really seem to exist any intuitive way to wait for a Promise to resolve synchronously without callbacks, or to use the EventEmitter with async callbacks, I wrote my own simple EventEmitter containing an array, and custom on and emit methods.
It handles asynchronous functions used as parameters properly so I can use it like this:
asyncEmitter.on('name', async (event, argument) => {
event.argument = await asyncMethod(); // Nice and clean code!
});
Of course I could develop it further create a once function and so on, but for me I really only depend on on and emit.
I wonder why the node team hasn't solved this yet?
I have a javascript app saving all data on server, then use REST API communicate server and client.
They works fine, until we start have more and more nested async call or nested sync call which hiding async call. For example:
function asyncFoo(callback) {
callback();
}
function syncCallHidingAsyncCall(){
syncStuff();
asyncFoo(function(){
syncFoo()
});
}
function nestedAsyncCall(callback){
asyncFoo(function(){
anotherAsyncCall(callback);
})
}
// this make refactor code become so hard.
// if we want add step2() after nestedAsyncCall();
// instead of add one line of code
// we need first add callback param in every asyncCall, then pass step2 as a callback
And some unnecessary async call:
// we actually only verify this once.
function isLogin(callback){
if (!App._user) {
ServerApi.getCurUser(function(data){
App._user = data.user;
callback(App._user)
});
}
callback(App._user)
}
function syncCallNeedVerfifyLogin(callback){
// user only need login once, so in most case this is really only a sync call.
// but now I have to involve a unnecessary callback to get the return value
isLogin(function(){
callback(syncStuff())
})
}
So after the project become bigger and bigger, we start forgot their relationship, which one need wait, which one will do magic. And more and more function become async only because some very small thing need be verify on server.
So I start feel their must be some design problem in this project. I am looking for the best practice or design patter, or some rules need follow in this kind heavy communicate app.
Thanks for help.
They exist in several patterns to manage asynchronous data exchange and routine execution. They are called in different names as well:
Promises
EventEmitters
Deferred Objects/Deferreds
Control Flow Libraries
Futures
Callback aggregators
Observer / Publisher-Subscriber
A common implementation is jQuery's Deferred Objects which is also used in managing it's AJAX methods. In NodeJS, there is also AsyncJS and the native EventEmitter. There's even a 20-liner library made by some guy that implements EventEmitter which you could use.
As Bergi says in the comments, the pattern you're looking for is called deferred / promises. There's an implementation built into jQuery. From the docs:
a chainable utility object created by calling the jQuery.Deferred()
method. It can register multiple callbacks into callback queues,
invoke callback queues, and relay the success or failure state of any
synchronous or asynchronous function.
There are a variety of other implementations some of which are outlined in this stackoverflow question.
Make yourself a queue system, something like:
function Queue() {
this.queue = [];
}
Queue.prototype.i = -1;
Queue.prototype.add = function(fn) {
if (typeof fn !== "function")
throw new TypeError("Invalid argument");
this.queue.push(fn);
}
Queue.prototype.next = function() {
this.i++;
if (this.i < this.queue.length) {
this.queue[this.i].appy(this, arguments);
}
}
Queue.prototype.start = function() {
if (this.i !== -1)
throw new Error("Already running")
this.next.apply(this, arguments);
}
And use it like this:
var q = new Queue();
q.add(function() {
// do something async
// In the callback, call `this.next()`, passing
// any relevant arguments
})
q.add(function() {
// do something async
// In the callback, call `this.next()`, passing
// any relevant arguments
})
q.add(function() {
// do something async
// In the callback, call `this.next()`, passing
// any relevant arguments
})
q.start();
DEMO: http://jsfiddle.net/4n3kH/
I'm trying to write my own wrapper for the mongojs collection.find method that should return the collection items selected by the specified query (querying isn't implemented yet, it should simply select all results). The problem is that I'm not getting back an array of results. It seems like the find method does some kind of async callback. So how can I force a synchronous call or force my script to wait?
Collection.prototype.find = function () {
var result = new Array;
if (Bridge.isServer) {
db.collection(name).find(function(err, items) {
items.forEach(function(item) {
result.push(item);
});
});
}
return result;
}
I think you should consider making your functions asynchronous, but if you insist on writing synchronous functions, there is a github project for making async functions synchronous.
Here's another SO post dealing with the same topic: Convert an asynchronous function to a synchronous function
I want to turn an asynchronous function to the synchronous.
function fetch() {
var result = 'snap!';
$.getJSON("http://api.flickr.com/services/feeds/photos_public.gne?tags=cat&tagmode=any&format=json&jsoncallback=?", function messyCallback(data){
result = data;
});
return result;
}
document.write(fetch());
See in action
The result always will be 'snap!', because $.getJSON run after fetch() is done.
My first idea was:
function fetch() {
var result = 'snap!';
$.getJSON("http://api.flickr.com/services/feeds/photos_public.gne?tags=cat&tagmode=any&format=json&jsoncallback=?", function messyCallback(data){
result = data;
});
while (true) {
if (result != 'snap!') return result;
}
}
It doesn't work and also blow off the browser.
I read about generators and iterators in JS 1.7, but I have no idea how to apply it to my problem.
This question is not really about jQuery. Instead of $.getJSON could be any another asynchronous function.
See this question also: Halt JavaScript execution without locking up the browser
Doing exactly what you want doesn't work. You can't create synchronousness from asynchronousness (only the other way around!) in a single-threaded event-driven environment. You should embrace the async nature of the language, and develop your own consistent style around handling callbacks so that your code is readable/maintainable. You could, for instance, choose to never do any real work inside a callback closure, but simply call another top-level function/method for clarity.
You want to roll your own $.getSyncJSON that uses $.ajax({async:false}). See: http://api.jquery.com/jQuery.ajax/.
I must advise you against this course of action, however. You can easily lock up the browser from all input, putting your UI at the mercy of the network. But if you know what you are doing and are sure of your use case, go for it.
Instead of writing helper methods like your fetch that return a value, make them accept another function, a "receiver", to pass their result to:
function fetch(receiver) {
$.getJSON("blah...", function(data) {
receiver(data);
});
}
Obviously this is redundant because it's exactly how getJSON already works, but in a more realistic example the fetch function would process or filter the result somehow before passing it on.
Then instead of:
document.write(fetch());
You'd do:
fetch(function(result) { document.write(result); });
Generators can be used to make asynchronous code a lot more linear in style. Each time you needed some asynchronous result, you'd yield a function to launch it, and the generator would resume when the result was available. There'd be a bit of management code keeping the generator going. But this isn't much help because generators are not standard across browsers.
If you're interested, here's a blog post about using generators to tidy up asynchronous code.
There is an extension to the JavaScript language called StratifiedJS. It runs in every browser, and it allows you to do just that: handling asynchronous problems in a synchronous/linear way without freezing your browser.
You can enable Stratified JavaScript e.g. by including Oni Apollo in your webpage like:
<script src="http://code.onilabs.com/latest/oni-apollo.js"></script>
<script type="text/sjs"> your StratifiedJS code here </script>
And your code would look like:
function fetch() {
return require("http").jsonp(
"http://api.flickr.com/services/feeds/photos_public.gne?" +
"tags=cat&tagmode=any&format=json", {cbfield:"jsoncallback"});
}
document.write(fetch());
Or if you really want to use jQuery in StratifiedJS:
require("jquery-binding").install();
function fetch() {
var url = "http://api.flickr.com/?format=json&...&jsoncallback=?"
return $.$getJSON(url);
}
document.write(fetch());
The docs are on http://onilabs.com/docs
The TameJS library is designed to deal with this problem.
You might write something like (untested):
var result = 'snap!';
await {
$.getJSON("http://api.flickr.com/services/feeds/photos_public.gne?tags=cat&tagmode=any&format=json&jsoncallback=?", defer(result));
}
return result;
The lower-level $.ajax() function from jQuery has more options, including async: [true|false] (default is true).
Nevertheless, in most cases you should follow Ben's advice and "embrace the async nature of the language".
I know this a little late but you can avoid callbacks with promises and await in async function
deliverResult = (options) => (
new Promise( (resolve, reject) => {
$.ajax(options).done(resolve).fail(reject);
})
)
getResult = async () => {
let options = {
type: 'get',
url: 'http://yourUrl.com',
data: {param1: 'arg1'}
}
console.log('waiting ..... ');
let result = await deliverResult(options);
console.log('**..waiting ended..**');
console.log(result);
}
<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.3.1/jquery.min.js"></script>
<input type='button' onclick='getResult()' value='click me'/>