Develop Reference

Is there a way to tell the event loop to process next message in ES5?

I know one must not block the loop, I know use call backs, and I know ES6 await. The more I research this the more it reaffirms it.
But sometimes your hands are tied. Is there a way to tell JavaScript, please go check on your event queue, and service those, then come back here before continuing execution.
Something like inspired by the MDN docs:
if (queue.hasNextMessage()) {
queue.processNextMessage()
}
There are similar threads about use datetime to wait a duration, but I don't know how the long other event thread will take, been looking at polling the promise status, but it appears to be a dead end.
The context is I have to override a validation callback. The caller of the callback does not wait for a promise to resolve (That I cant change).
Here is the test setup showing the concept. I have made a few attempts, but none of them work because they are always stuck in the main loop.
// The validate function depends on a fetch call which takes time.
// Free to change this.
function validate() {
return fetch(url).then(response => response.json())
.then(data => {console.log(data); return true;})
.catch(msg => {console.log(msg); return false;})
}
// Cannot change this function, I am not in control of it
function CallValidate() {
console.log("Validation Result: ", Boolean(validate()));
}
// This is the setup for when test passes
let url = 'http://api.open-notify.org/astros.json';
CallValidate();
// This is the setup for when test fails
// This currently fails because the promise objects is being evaluated
// to true, instead of waiting for its response.
url = 'http://DUMMY.NOT.WORKING.URL';
CallValidate();

Is there a way to tell JavaScript, please go check on your event queue, and service those, then come back here before continuing execution.
No, but you can do something very similar: divide your computation into several tasks. In Node.js, you can use setImmediate() to queue a task. In the browser, you can use messages, as outlined by this answer.
Example:
function setImmediate(callback) {
const channel = new MessageChannel();
channel.port1.onmessage = () => {
callback();
};
channel.port2.postMessage("");
}
console.log("Task 1");
setImmediate( () => {
console.log("Task 2");
});

Boolean(validate()) will always evaluate to true, because validate returns a Promise which is a truthy value no matter what it resolves to.
If you can't change CallValidate, you might still be able to lift up the fetch call. But I'm not sure if that's an option for you.
async function validateUrl(url) {
const valid = await fetch(url).then(response => response.json())
.then(data => {console.log(data); return true;})
.catch(msg => {console.log(msg); return false;});
function validate() {
return valid;
}
function CallValidate() {
console.log("Validation Result: ", Boolean(validate()));
}
CallValidate();
}
// assuming you are in async function context
await validateUrl('http://api.open-notify.org/astros.json');
await validateUrl('http://DUMMY.NOT.WORKING.URL');
If moving the function is also no option I would ditch the CallValidate code and either write something yourself, or pick another library/helper/etc that does handle asynchronous functions (with promise return values).

Async Frustrations: Should I be using callbacks and how do I pass them through multiple modules?

I am attempting to make a simple text game that operates in a socket.io chat room on a node server. The program works as follows:
Currently I have three main modules
Rogue : basic home of rogue game functions
rogueParser : module responsible for extracting workable commands from command strings
Verb_library: module containing a list of commands that can be invoked from the client terminal.
The Client types a command like 'say hello world'. This triggers the following socket.io listener
socket.on('rg_command', function(command){
// execute the verb
let verb = rogueParser(command);
rogue.executeVerb(verb, command, function(result){
console.log(result);
});
});
Which then in turn invokes the executeVerb function from rogue..
executeVerb: function(verb, data, callback){
verb_library[verb](data, callback);
},
Each verb in verb_library should be responsible for manipulating the database -if required- and then returning an echo string sent to the appropriate targets representing the completion of the action.
EDIT: I chose 'say' when I posted this but it was pointed out afterward that it was a poor example. 'say' is not currently async but eventually will be as will be the vast majority of 'verbs' as they will need to make calls to the database.
...
say: function(data, callback){
var response = {};
console.log('USR:'+data.user);
var message = data.message.replace('say','');
message = ('you say '+'"'+message.trim()+'"');
response.target = data.user;
response.type = 'echo';
response.message = message;
callback(response);
},
...
My problem is that
1 ) I am having issues passing callbacks through so many modules. Should I be able to pass a callback through multiple layers of modules? Im worried that I'm blind so some scope magic that is making me lose track of what should happen when I pass a callback function into a module which then passes the same callback to another module which then calls the callback. Currently it seems I either end up without access to the callback on the end, or the first function tries to execute without waiting on the final callback returning a null value.
2 ) Im not sure if Im making this harder than it needs to be by not using promises or if this is totally achievable with callbacks, in which case I want to learn how to do it that way before I summon extra code.
Sorry if this is a vague question, I'm in a position of design pattern doubt and looking for advice on this general setup as well as specific information regarding how these callbacks should be passed around. Thanks!

1) Passing callback trough multiple layers doesn't sound like a good idea. Usually I'm thinking what will happen, If I will continiue doing this for a year? Will it be flexible enough so that when I need to change to architecture (let's say customer have new idea), my code will allow me to without rewriting whole app? What you're experiencing is called callback hell. http://callbackhell.com/
What we are trying to do, is to keep our code as shallow as possible.
2) Promise is just syntax sugar for callback. But it's much easier to think in Promise then in callback. So personally, I would advice you to take your time and grasp as much as you can of programming language features during your project. Latest way we're doing asynchronus code is by using async/await syntax which allows us to totally get rid of callback and Promise calls. But during your path, you will have to work with both for sure.
You can try to finish your code this way and when you're done, find what was the biggest pain and how could you write it again to avoid it in future. I promise you that it will be much more educative then getting explicit answear here :)

Asynchronousness and JavaScript go back a long way. How we deal with it has evolved over time and there are numerous applied patterns that attempt to make async easier. I would say that there are 3 concrete and popular patterns. However, each one is very related to the other:
Callbacks
Promises
async/await
Callbacks are probably the most backwards compatible and just involve providing a function to some asynchronous task in order to have your provided function be called whenever the task is complete.
For example:
/**
* Some dummy asynchronous task that waits 2 seconds to complete
*/
function asynchronousTask(cb) {
setTimeout(() => {
console.log("Async task is done");
cb();
}, 2000);
}
asynchronousTask(() => {
console.log("My function to be called after async task");
});
Promises are a primitive that encapsulates the callback pattern so that instead of providing a function to the task function, you call the then method on the Promise that the task returns:
/**
* Some dummy asynchronous task that waits 2 seconds to complete
* BUT the difference is that it returns a Promise
*/
function asynchronousTask() {
return new Promise(resolve => {
setTimeout(() => {
console.log("Async task is done");
resolve();
}, 2000);
});
}
asynchronousTask()
.then(() => {
console.log("My function to be called after async task");
});
The last pattern is the async/await pattern which also deals in Promises which are an encapsulation of callbacks. They are unique because they provide lexical support for using Promises so that you don't have to use .then() directly and also don't have to explicitly return a Promise from your task:
/*
* We still need some Promise oriented bootstrap
* function to demonstrate the async/await
* this will just wait a duration and resolve
*/
function $timeout(duration) {
return new Promise(resolve => setTimeout(resolve, duration));
}
/**
* Task runner that waits 2 seconds and then prints a message
*/
(async function() {
await $timeout(2000);
console.log("My function to be called after async task");
}());
Now that our vocabulary is cleared up, we need to consider one other thing: These patterns are all API dependent. The library that you are using uses callbacks. It is alright to mix these patterns, but I would say that the code that you write should be consistent. Pick one of the patterns and wrap or interface with the library that you need to.
If the library deals in callbacks, see if there is a wrapping library or a mechanism to have it deal in Promises instead. async/await consumes Promises, but not callbacks.

Callbacks are fine, but I would only use them if a function is dependent on some asynchronous result. If however the result is immediately available, then the function should be designed to return that value.
In the example you have given, say does not have to wait for any asynchronous API call to come back with a result, so I would change its signature to the following:
say: function(data){ // <--- no callback argument
var response = {};
console.log('USR:'+data.user);
var message = data.message.replace('say','');
message = ('you say '+'"'+message.trim()+'"');
response.target = data.user;
response.type = 'echo';
response.message = message;
return response; // <--- return it
}
Then going backwards, you would also change the signature of the functions that use say:
executeVerb: function(verb, data){ // <--- no callback argument
return verb_library[verb](data); // <--- no callback argument, and return the returned value
}
And further up the call stack:
socket.on('rg_command', function(command){
// execute the verb
let verb = rogueParser(command);
let result = rogue.executeVerb(verb, command); // <--- no callback, just get the returned value
console.log(result);
});
Of course, this can only work if all verb methods can return the expected result synchronously.
Promises
If say would depend on some asynchronous API, then you could use promises. Let's assume this API provides a callback system, then your say function could return a promise like this:
say: async function(data){ // <--- still no callback argument, but async!
var response = {};
console.log('USR:'+data.user);
var message = data.message.replace('say','');
response.target = data.user;
response.type = 'echo';
// Convert the API callback system to a promise, and use AWAIT
await respone.message = new Promise(resolve => someAsyncAPIWithCallBackAsLastArg(message, resolve));
return response; // <--- return it
}
Again going backwards, you would also change the signature of the functions that use say:
executeVerb: function(verb, data){ // <--- still no callback argument
return verb_library[verb](data); // <--- no callback argument, and return the returned promise(!)
}
And finally:
socket.on('rg_command', async function(command){ // Add async
// execute the verb
let verb = rogueParser(command);
let result = await rogue.executeVerb(verb, command); // <--- await the fulfillment of the returned promise
console.log(result);
});

How can I use Observables instead of Promises?

I have a service with some methods, most of them require a certain callback to be completed before it can do its stuff. With Promises, in pseudo, it is very easy to do this:
ready = http.get(stuff); // Returns a promise, resolves after a while
methodOne() { // methods sometimes called before promise resolves
this.ready.then(_ => {
// doStuff
});
}
methodTwo() {
return this.ready.then(d => {
// doOtherStuff
});
}
Basically I need to do the stuff, only when i'm sure the service is ready.
I actually only need to check if it's ready (what methodOne is doing, just illustrating with methodTwo, that it's easy to more stuff as well).
I want to try and go all in on Observables, but for this specific case, I find it really hard to compete with a similar solution for Observables.
Promises will remember the value and know if it got resolved. An Observable is somewhat more complex and it seems that creating this same flow is troublesome. I need whatever is subscribing to the Observable, to known when it's ready. Some times the method is called early - before the Observable emits and sometimes late, after the Observable already emitted.
I have this right now, but it doesn't seem to work:
this.ready$ = someObservable // Will fire after a litle while but never finish - i only need the first to check though.
.publishReplay(1).refCount(); // Trying to replay if subscription comes after emit.
this.ready$.subscribe(_ => {
// This will be called
});
methodOne() {
this.ready$.subscribe(_ => {
// Not called
});
};
Perhaps i misunderstood the use of publishReplay and refCount?

I think what you're looking for is AsyncSubject. It mimics the promises behavior very well. Here is the description:
The AsyncSubject is a variant where only the last value of the
Observable execution is sent to its observers, and only when the
execution completes.
Here is how it can be used in your case:
subject = new AsyncSubject();
ready = streamOfData(stuff).first().subscribe(subject);
methodOne() {
return this.subject.asObservable();
}
The subject subscribes to the underlying observable returned by the first operator and waits until it's complete. It collects all the subscribers but doesn't send any values to them. As soon as the underlying observable completes it remembers the value and sends it to the collected subscribers. All new future subscribers will be immediately passed this stored resolved value.
Here is the simple example that demonstrates that you can subscribe before or after the observable completes:
const subject = new AsyncSubject();
const o = subject.asObservable();
o.subscribe((v) => {
console.log(v);
});
interval(500).first().subscribe(subject);
setTimeout(() => {
o.subscribe((v) => {
console.log(v);
});
}, 2000);

Get return value from subcribe on Observable

Using RxJS 5.0.0-rc.1, I'm trying to communicate my Observer and Observable in a way similar to how generators/iterators work by exchanging data using yield and .next(). The intention is to get a hold of what a call to .subscribe returns and modify/update following values in my observable stream depending on that.
I'm not entirely sure if this is, at all, possible. Though, I found out that you can catch exceptions thrown on .subscribe callbacks. The following snippets prints out "Boom!":
var source = Observable.create((observer) => {
try {
observer.next(42);
} catch (e) {
// This will catch the Error
// thrown on the subscriber
console.log(e.message);
}
observer.complete();
});
source.subscribe(() => {
throw new Error('Boom!');
});
So, what if instead of throwing, the subscriber returns a value? Is there a way for the Observable to retrieve it? Perhaps I'm approaching this the wrong way. If so, what's the "reactive" way of doing things in this scenario?
Many thanks.
EDIT
One possible way I came up with is by providing a callback function on every item in the stream. Something like:
var source = Observable.create((observer) => {
// This will print "{ success: true }"
observer.next({ value: 42, reply: console.log });
observer.complete();
});
source.subscribe(({ value, reply }) => {
console.log('Got', value);
return reply({ success: true });
});
Any other thoughts?
EDIT 2
Since my original question brought some confusion on what I was trying to achieve, I'll describe my real world scenario. I'm writing the API of a module for managing messages through queues (much like a simplified, in memory, AMQP-RPC mechanism) and I though RxJS would be a good fit.
It works like you would expect: a Publisher pushes messages to a queue, which get delivered to a Consumer. In term, the Consumer can reply to the Publisher, which can listen to that response if it's interested.
In an ideal scenario, the API would look something like this:
Consumer().consume('some.pattern')
.subscribe(function(msg) {
// Do something with `msg`
console.log(msg.foo);
return { ok: true };
});
Publisher().publish('some.pattern', { foo: 42 })
// (optional) `.subscribe()` to get reply from Consumer
That example would print 42.
The logic for replying to the Publisher lies within the Consumer function. But the actual response comes from the .subscribe() callback. Which leads me to my original question: how should I go about fetching that returned value from the creator of the stream?
Think of Consumer#consume() as:
/**
* Returns an async handler that gets invoked every time
* a new message matching the pattern of this consumer
* arrives.
*/
function waitOnMessage(observer) {
return function(msg) {
observer.next(msg);
// Conceptually, I'd like the returned
// object from `.subscribe()` to be available
// in this scope, somehow.
// That would allow me to go like:
// `sendToQueue(pubQueue, response);`
}
}
return Observable.create((observer) => {
queue.consume(waitOnMessage(observer));
});
Does it make any more sense?

There are indeed similarities between generators and observables. As you can see here, observables (asynchronous sequence of values) are the asynchronous version of iterables (synchronous sequence of values).
Now, a generator is a function which returns an Iterable. However, Rxjs Observable encloses both a generator - a.k.a producer (that you execute/start by calling subscribe) and the produced asynchronous sequence of values (that you observe by passing an Observer object). And the subscribe call returns a Disposable which allows you to stop receiving values (disconnect). So while generators and observables are dual concepts, the APIs to use them differ.
You cannot do two-way communication by default with the rxjs observable API. You probably could manage to do it by constructing yourself the back channel through subjects (note that you MUST have an initial value to kick off the cycle).
var backChannel = Rx.Subject();
backChannel.startWith(initialValue).concatMap(generateValue)
.subscribe(function observer(value){
// Do whatever
// pass a value through the backChannel
backChannel.next(someValue)
})
// generateValue is a function which takes a value from the back channel
// and returns a promise with the next value to be consumed by the observer.
You could consider wrapping that with :
function twoWayObsFactory (yield, initialValue) {
var backChannel = Rx.BehaviorSubject(initialValue);
var next = backChannel.next.bind(backChannel);
return {
subscribe : function (observer) {
var disposable = backChannel.concatMap(yield)
.subscribe(function(x) {
observer(next, x);
});
return {
dispose : function (){disposable.dispose(); backChannel.dispose();}
}
}
}
}
// Note that the observer is now taking an additional parameter in its signature
// for instance
// observer = function (next, yieldedValue) {
// doSomething(yieldedValue);
// next(anotherValue);
// }
// Note also that `next` is synchronous, as such you should avoir sequences
// of back-and-forth communication that is too long. If your `yield` function
// would be synchronous, you might run into stack overflow errors.
// All the same, the `next` function call should be the last line, so order of
// execution in your program is the same independently of the synchronicity of
// the `yield` function
Otherwise, the behaviour you describe seems to be that of an asynchronous generator. I never used such, but as this is a proposal for some future version of javascript, I think you can
already start trying it out with Babel (cf. https://github.com/tc39/proposal-async-iteration).
EDIT :
If you are looking for a loop-back mechanism (less general purpose approach but could very well fits your use case, if what you want to do is simple enough), the expand operator could help. To understand its behaviour, please check the doc, and the following answers on SO for examples of use in concrete contexts:
RxJS: Backpressure with switchMap producing N values
Circular Dependencies with RxJS. Modeling spores
How to build an rx poller that waits some interval AFTER the previous ajax promise resolves?
Basically expand allows you to both emit a value downstream and feed that value back at the same time in your producer.

Node.js async, but only handle first positive/defined result

What is the best way to create parallel asynchronous HTTP requests and take the first result that comes back positive? I am familiar with the async library for JavaScript and would happy to use that but am not sure if it has exactly what I want.
Background - I have a Redis store that serves as state for a server. There is an API we can call to get some data that takes much longer than reaching the Redis store.
In most cases the data will already be in the Redis store, but in some cases it won't be there yet and we need to retrieve it from the API.
The simple thing to do would be to query Redis, and if the value is not in Redis then go to the API afterwards. However, we'll needlessly lose 20-50ms if the data is not yet in our Redis cache and we have to go to the API after failing to find the data with Redis. Since this particular API server is not under great load, it won't really hurt to go to the API simultaneously/in parallel, even if we don't absolutely need the returned value.
//pseudocode below
async.minimum([
function apiRequest(cb){
request(opts,function(err,response,body){
cb(err,body.result.hit);
}
},
function redisRequest(cb){
client.get("some_key", function(err, reply) {
cb(err,reply.result.hit);
});
}],
function minimumCompleted(err,result){
// this mimimumCompleted final callback function will be only fired once,
// and would be fired by one of the above functions -
// whichever one *first* returned a defined value for result.hit
});
is there a way to get what I am looking for with the async library or perhaps promises, or should I implement something myself?

Use Promise.any([ap, bp]).
The following is a possible way to do it without promises. It is untested but should meet the requirements.
To meet requirement of returning the first success and not just the first completion, I keep a count of the number of completions expected so that if an error occurs it can be ignored it unless it is the last error.
function asyncMinimum(a, cb) {
var triggered = false;
var completions = a.length;
function callback(err, data) {
completions--;
if (err && completions !== 0) return;
if (triggered) return;
triggered = true;
return cb(err, data);
}
a.map(function (f) { return f(callback); });
}
asyncMinimum([
function apiRequest(cb){
request(opts,function(err,response,body){
cb(err,body.result.hit);
}
},
function redisRequest(cb){
client.get("some_key", function(err, reply) {
cb(err,reply.result.hit);
});
}],
function minimumCompleted(err,result){
// this mimimumCompleted final callback function will be only fired once,
// and would be fired by one of the above functions -
// whichever one had a value for body.result.hit that was defined
});

The async.js library (and even promises) keep track of the number of asynchronous operations pending by using a counter. You can see a simple implementation of the idea in an answer to this related question: Coordinating parallel execution in node.js
We can use the same concept to implement the minimum function you want. Only, instead of waiting for the counter to count all responses before triggering a final callback, we deliberately trigger the final callback on the first response and ignore all other responses:
// IMHO, "first" is a better name than "minimum":
function first (async_functions, callback) {
var called_back = false;
var cb = function () {
if (!called_back) {
called_back = true; // block all other responses
callback.apply(null,arguments)
}
}
for (var i=0;i<async_functions.length;i++) {
async_functions[i](cb);
}
}
Using it would be as simple as:
first([apiRequest,redisRequest],function(err,result){
// ...
});

Here's an approach using promises. It takes a little extra custom code because of the non-standard result you're looking for. You aren't just looking for the first one to not return an error, but you're looking for the first one that has a specific type of result so that takes a custom result checker function. And, if none get a result, then we need to communicate that back to the caller by rejecting the promise too. Here's the code:
function firstHit() {
return new Promise(function(resolve, reject) {
var missCntr = 0, missQty = 2;
function checkResult(err, val) {
if (err || !val) {
// see if all requests failed
++missCntr;
if (missCntr === missQty) {
reject();
}
} else {
resolve(val);
}
}
request(opts,function(err, response, body){
checkResult(err, body.result.hit);
}
client.get("some_key", function(err, reply) {
checkResult(err, reply.result.hit);
});
});
}
firstHit().then(function(hit) {
// one of them succeeded here
}, function() {
// neither succeeded here
});
The first promise to call resolve() will trigger the .then() handler. If both fail to get a hit, then it will reject the promise.