I wrote some code below that uses promises and the easiest way I could find to write it was using a Deferred object instead of the usual Promise executor function because I need to resolve the promise from outside the executor. I'm wondering if there's an accepted design pattern based on the Promise executor function for a problem like this that doesn't use a deferred-like solution? Can it be done without having to resolve the promise from outside the promise executor?
Here are the details.
I have a project that uses a set of Worker Threads and various parts of the code that want to use a Worker Thread from time to time. To manage that, I've created a simple WorkerList class that keeps a list of the available Worker Threads. When someone wants to use one, they call get() on it and that returns a promise that resolves to a Worker Thread. If a worker thread is available immediately, the promise resolves immediately. If all worker threads are in use (and thus the list of available workers is empty), then the promise doesn't resolve until one is later put back into the available list via the add(worker) method.
This WorkerList class has only two methods, add(worker) and get(). You get() a worker and when you're done with it, you add(worker) it back. When you add(worker) it back, the class checks to see if there are any tasks waiting for an available Worker. If there, are, it resolves their promise with an available Worker. That resolving of someone else's promise is where the Deferred was used.
Here's the code for the WorkerList:
class WorkerList {
constructor() {
this.workers = [];
this.deferredQueue = [];
}
add(worker) {
this.workers.push(worker);
// if someone is waiting for a worker,
// pull the oldest worker out of the list and
// give it to the oldest deferred that is waiting
while (this.deferredQueue.length && this.workers.length) {
let d = this.deferredQueue.shift();
d.resolve(this.workers.shift());
}
}
// if there's a worker, get one immediately
// if not, return a promise that resolves with a worker
// when next one is available
get() {
if (this.workers.length) {
return Promise.resolve(this.workers.shift());
} else {
let d = new Deferred();
this.deferredQueue.push(d);
return d.promise;
}
}
}
And, here's the Deferred implementation:
function Deferred() {
if (!(this instanceof Deferred)) {
return new Deferred();
}
const p = this.promise = new Promise((resolve, reject) => {
this.resolve = resolve;
this.reject = reject;
});
this.then = p.then.bind(p);
this.catch = p.catch.bind(p);
if (p.finally) {
this.finally = p.finally.bind(p);
}
}
Maybe the below is just a poor man's approach to deferreds, and doesn't really get to the crux of the matter, but instead of a queue of deferreds, you could just keep a queue of resolver functions.
This saves a small amount of code over your approach and avoids explicitly using Deferreds.
I don't know if there is an established pattern for this, but this in itself seems like a reusable pattern for maintaining an asynchronous pool of objects, so rather than calling it WorkerList, you could name it AsyncPool, and then compose that as a reusable piece within your WorkerList:
class AsyncPool {
constructor() {
this.entries = [];
this.resolverQueue = [];
}
add(entry) {
console.log(`adding ${entry}`);
this.entries.push(entry);
// if someone is waiting for an entry,
// pull the oldest one out of the list and
// give it to the oldest resolver that is waiting
while (this.resolverQueue.length && this.entries .length) {
let r = this.resolverQueue.shift();
r(this.entries.shift());
}
}
// if there's an entry, get one immediately
// if not, return a promise that resolves with an entry
// when next one is available
get() {
return new Promise((r) =>
this.entries.length
? r(this.entries.shift())
: this.resolverQueue.push(r)
);
}
}
let pool = new AsyncPool();
pool.add('Doc');
pool.add('Grumpy');
pool.get().then(console.log);
pool.get().then(console.log);
pool.get().then(console.log);
pool.get().then(console.log);
// add more entries later
setTimeout(() => pool.add('Sneezy'), 1000);
setTimeout(() => pool.add('Sleepy'), 2000);
Here is one solution that doesn't expose the promise resolver function anywhere outside the promise executor function.
Following up on my comment to my own question about an event-based solution, here's what I came up with. It uses a triggered event and an event listener to cause an action inside the promise executor function.
class WorkerList extends EventEmitter {
constructor() {
this.workers = [];
}
add(worker) {
this.workers.push(worker);
// notify listeners that there's a new worker in town
this.emit('workerAdded');
}
// if there's a worker, get one immediately
// if not, return a promise that resolves with a worker
// when next one is available
get() {
if (this.workers.length) {
return Promise.resolve(this.workers.shift());
} else {
return new Promise(resolve => {
const onAdded = () => {
if (this.workers.length) {
this.off('workerAdded', onAdded);
resolve(this.workers.shift());
}
}
this.on('workerAdded', onAdded);
});
}
}
}
I was initially concerned about maintaining FIFO ordering so that the first one to call get() gets the next worker available. But, because eventListeners are called in the order they were added, I think this would actually achieve FIFO order. If there are multiple calls to get(), they will all get notified about the workerAdded, but after the first one handles the message and takes the worker, the others will just find no worker left for them so their listener will stay attached waiting for a future workerAdded message when there is a worker for them (when their listener gets to be first in line).
I don't think I necessarily like this better than the other options shown, but it is an alternative and doesn't use Deferreds or even expose the resolve handler outside the executor function.
As was suggested, this could also be done where the eventEmitter is an instance variable rather than a base class:
class WorkerList {
constructor() {
this.workers = [];
this.emitter = new EventEmitter();
}
add(worker) {
this.workers.push(worker);
// notify listeners that there's a new worker in town
this.emitter.emit('workerAdded');
}
// if there's a worker, get one immediately
// if not, return a promise that resolves with a worker
// when next one is available
get() {
if (this.workers.length) {
return Promise.resolve(this.workers.shift());
} else {
return new Promise(resolve => {
const onAdded = () => {
if (this.workers.length) {
this.emitter.off('workerAdded', onAdded);
resolve(this.workers.shift());
}
}
this.emitter.on('workerAdded', onAdded);
});
}
}
}
Related
I'd like to accomplish the following using promises: only execute further once the state of something is ready. I.e. like polling for an external state-change.
I've tried using promises and async-await but am not getting the desired outcome. What am I doing wrong here, and how do I fix it?
The MDN docs have something similar but their settimeout is called within the promise--that's not exactly what I'm looking for though.
I expect the console.log to show "This function is now good to go!" after 5 seconds, but instead execution seems to stop after calling await promiseForState();
var state = false;
function stateReady (){
state = true;
}
function promiseForState(){
var msg = "good to go!";
var promise = new Promise(function (resolve,reject){
if (state){
resolve(msg);
}
});
return promise;
}
async function waiting (intro){
var result = await promiseForState();
console.log(intro + result)
}
setTimeout(stateReady,5000);
waiting("This function is now ");
What you're doing wrong is the promise constructor executor function executes immediately when the promise is created, and then never again. At that point, state is false, so nothing happens.
Promises (and async/await) are not a replacement for polling. You still need to poll somewhere.
The good news: async functions make it easy to do conditional code with loops and promises.
But don't put code inside promise constructor executor functions, because of their poor error handling characteristics. They are meant to wrap legacy code.
Instead, try this:
var state = false;
function stateReady() {
state = true;
}
const wait = ms => new Promise(resolve => setTimeout(resolve, ms));
async function promiseForState() {
while (!state) {
await wait(1000);
}
return "good to go!";
}
async function waiting(intro) {
var result = await promiseForState();
console.log(intro + result)
}
setTimeout(stateReady,5000);
waiting("This function is now ");
Based on your comments that you are waiting for messages from a server it appears you are trying to solve an X/Y problem. I am therefore going to answer the question of "how do I wait for server messages" instead of waiting for global variable to change.
If your network API accepts a callback
Plenty of networking API such as XMLHttpRequest and node's Http.request() are callback based. If the API you are using is callback or event based then you can do something like this:
function myFunctionToFetchFromServer () {
// example is jQuery's ajax but it can easily be replaced with other API
return new Promise(function (resolve, reject) {
$.ajax('http://some.server/somewhere', {
success: resolve,
error: reject
});
});
}
async function waiting (intro){
var result = await myFunctionToFetchFromServer();
console.log(intro + result);
}
If your network API is promise based
If on the other hand you are using a more modern promise based networking API such as fetch() you can simply await the promise:
function myFunctionToFetchFromServer () {
return fetch('http://some.server/somewhere');
}
async function waiting (intro){
var result = await myFunctionToFetchFromServer();
console.log(intro + result);
}
Decoupling network access from your event handler
Note that the following are only my opinion but it is also the normal standard practice in the javascript community:
In either case above, once you have a promise it is possible to decouple your network API form your waiting() event handler. You just need to save the promise somewhere else. Evert's answer shows one way you can do this.
However, in my not-so-humble opinion, you should not do this. In projects of significant size this leads to difficulty in tracing the source of where the state change comes form. This is what we did in the 90s and early 2000s with javascript. We had a lot of events in our code like onChange and onReady or onData instead of callbacks passed as function parameters. The result was that sometimes it takes you a long time to figure out what code is triggering what event.
Callback parameters and promises forces the event generator to be in the same place in the code as the event consumer:
let this_variable_consumes_result_of_a_promise = await generate_a_promise();
this_function_generate_async_event((consume_async_result) => { /* ... */ });
From the wording of your question you seem to be wanting to do this instead;
..somewhere in your code:
this_function_generate_async_event(() => { set_global_state() });
..somewhere else in your code:
let this_variable_consumes_result_of_a_promise = await global_state();
I would consider this an anti-pattern.
Calling asynchronous functions in class constructors
This is not only an anti-pattern but an impossibility (as you've no doubt discovered when you find that you cannot return the asynchronous result).
There are however design patterns that can work around this. The following is an example of exposing a database connection that is created asynchronously:
class MyClass {
constructor () {
// constructor logic
}
db () {
if (this.connection) {
return Promise.resolve(this.connection);
}
else {
return new Promise (function (resolve, reject) {
createDbConnection(function (error, conn) {
if (error) {
reject(error);
}
else {
this.connection = conn; // cache the connection
resolve(this.connection);
}
});
});
}
}
}
Usage:
const myObj = new MyClass();
async function waiting (intro){
const db = await myObj.db();
db.doSomething(); // you can now use the database connection.
}
You can read more about asynchronous constructors from my answer to this other question: Async/Await Class Constructor
The way I would solve this, is as follows. I am not 100% certain this solves your problem, but the assumption here is that you have control over stateReady().
let state = false;
let stateResolver;
const statePromise = new Promise( (res, rej) => {
stateResolver = res;
});
function stateReady(){
state = true;
stateResolver();
}
async function promiseForState(){
await stateResolver();
const msg = "good to go!";
return msg;
}
async function waiting (intro){
const result = await promiseForState();
console.log(intro + result)
}
setTimeout(stateReady,5000);
waiting("This function is now ");
Some key points:
The way this is written currently is that the 'state' can only transition to true once. If you want to allow this to be fired many times, some of those const will need to be let and the promise needs to be re-created.
I created the promise once, globally and always return the same one because it's really just one event that every caller subscribes to.
I needed a stateResolver variable to lift the res argument out of the promise constructor into the global scope.
Here is an alternative using .requestAnimationFrame().
It provides a clean interface that is simple to understand.
var serverStuffComplete = false
// mock the server delay of 5 seconds
setTimeout(()=>serverStuffComplete = true, 5000);
// continue until serverStuffComplete is true
function waitForServer(now) {
if (serverStuffComplete) {
doSomethingElse();
} else {
// place this request on the next tick
requestAnimationFrame(waitForServer);
}
}
console.log("Waiting for server...");
// starts the process off
requestAnimationFrame(waitForServer);
//resolve the promise or whatever
function doSomethingElse() {
console.log('Done baby!');
}
I want to queue up DB calls that will be executed once it's connected. The DB object is created and stored as a member of a module when it's connected.
DB Module:
var db = {
localDb: null,
connectLocal: (dbName) => {
// Do stuff
this.localDb = new PouchDB(dbName) // has a allDocs() method
}
}
Adding calls to queue:
var dbQueue = []
function getDocs () {
dbQueue.push (
db.localDb.allDocs () // allDocs() not yet defined; returns promise
)
}
// Called when connected and queue is not empty:
function processQueue () {
Promise.all (dbQueue)
.then(...)
}
If getDocs() is called before db.connectLocal() sets db.localDb, then I get the following error (or similar) because db.localDb is not yet defined:
TypeError: Cannot read property 'then' of undefined
Is it possible to add an undefined method, that returns a promise, to an array to be resolved later in Promise.all()? Any other ideas as to how I can solve this issue?
Also, I'm using Vue.js and PouchDB.
You can make a promise in your db module instead of just localDb property:
let localDb = null;
let resolveLocalDb = null;
let localDbPromise = new Promise(function(resolve, reject) {
resolveLocalDb = resolve;
});
var db = {
getLocalDb: () {
return localDbPromise;
}
connectLocal: (dbName) => {
// Do stuff
localDb = new PouchDB(dbName) // has a allDocs() method
resolveLocalDb(localDb);
}
}
Then, exchange .localDb to getLocalDb(), which returns a promise.
dbQueue.push(
db.getLocalDb().then(db => db.allDocs())
)
I solved my queue issue, but it wasn't at all how I was trying to go about it.
My first problem was thinking that Promise.all() deferred calling my methods until it was called, but they are called when added to the array. This caused the error I mentioned in my question. So I needed to rethink how to populate the queue with methods that may not yet exist.
The solution was to add the calls to an array (the queue) as strings (e.g. "getDocs"), then loop through the array calling the methods using bracket notation (e.g. db["getDocs"]()).
My app is written in Vue.js so it's obviously different, but here's a simplified, working example:
// Dummy DB object
var db = {
docs: [1, 2, 3]
};
// Queue were the DB ops are stored
var dbQueue = [];
// Process the queue - called elsewhere once the DB is connected
// The processed array and Promise.all() aren't necessary as you could just call
// the method outright, but I want to log the results in order
async function processQueue() {
var processed = []; // Called queue methods
// Add valid methods to
dbQueue.forEach(method => {
if (typeof db[method] === "function") {
return processed.push(db[method]());
} else {
console.error(`"${method}" is not a name of a valid method.`);
}
});
// Log promise results
await Promise.all(processed).then(res => {
console.log("Processed:", res);
});
// Empty the queue
dbQueue = [];
}
// Add some calls to the queue of methods that don't yet exist
dbQueue.push("getDocs");
dbQueue.push("getDocs");
// Simulate adding the method
db.getDocs = function() {
return new Promise(resolve => {
resolve(this.docs);
});
};
// Process queue once conditions are met (e.g. db is connected); called elsewhere
processQueue();
And here's a fiddle with an example that allows arguments for the methods: https://jsfiddle.net/rjbv0284/1/
Note: My code happens to be in TypeScript, but this question applies to both TypeScript and JavaScript with a few syntactic changes.
I've got a class that subscribes to a few things (such as Autobahn messages). When I'm done with the object, I need to unsubscribe from everything. Generically, my code is something like this:
class Example {
private sub1: Subscription;
private sub2: Subscription;
private sub3: Subscription;
constructor() {
subscribeToThings();
}
async subscribeToThings() {
this.sub1 = await subscribeToThingOne();
this.sub2 = await subscribeToThingTwo();
// *** NOTE 1 ***
this.sub3 = await subscribeToThingThree();
}
cleanupOnDeath () {
this.sub1.unsubscribe();
this.sub2.unsubscribe();
this.sub3.unsubscribe();
}
}
When I no longer need this object, I call cleanupOnDeath.
However, there is a problem. If the subscriptions (which are asynchronous) take a long time, it's possible that the "cleanupOnDeath()" function will be called before all of sub1, sub2, and sub3 are set. For example, the code might be at the line marked *** NOTE 1 *** when cleanupOnDeath is called.
Of course I could check if each one is undefined or not, before calling unsubscribe, but that doesn't really solve things because then that third subscription gets processed after cleanupOnDeath is finished, and lives forever.
Is there a way to implement this code so that cleanupOnDeath() can be be called in the middle of subscribeToThings() without making the code incredibly complex?
The simplest solution I have been able to come up with is something like this (replacing the sections of the above code as appropriate):
constructor() {
this.subscribeAndUnsubscribe();
}
async subscribeAndUnsubscribe () {
await this.subscribeToThings();
await cleanupEvent.wait();
this.sub1.unsubscribe();
this.sub2.unsubscribe();
this.sub3.unsubscribe();
}
cleanupOnDeath() {
cleanupEvent.set();
}
Where cleanupEvent is some kind of synchronization primitive. But I don't think this is optimal, because it also requires all pending subscriptions to be completed before any can be unsubscribed. Ideally, I'd like to be able to abort subscribeToThings() early, without adding checks after every line of code. For example. it can be done like this:
completed:boolean = false;
async subscribeToThings() {
try {
this.abortIfDone();
this.sub1 = await subscribeToThingOne();
this.abortIfDone();
this.sub2 = await subscribeToThingTwo();
this.abortIfDone();
this.sub3 = await subscribeToThingThree();
} catch (abortedEarly) {
// Something here
}
}
abortIfDone() {
if (this.completed) throw 'Something';
}
cleanupOnDeath() {
this.completed = true;
if (this.sub1) this.sub1.unsubscribe();
if (this.sub1) this.sub2.unsubscribe();
if (this.sub1) this.sub3.unsubscribe();
}
But that is messy and complicated.
Edit - Solution
I ended up using the following code, based on #ChrisTavares's answer. The difference between this and his answer is that I am no longer awaiting anything or storing the subscriptions themselves. Before, I was awaiting them to get the underlying subscription objects, for the sole purpose of unsubscribing later. Since this code stores the promises and handles unsubscribe on those using then, there is no need to do that.
class Example {
private sub1Promise: Promise<Subscription>;
private sub2Promise: Promise<Subscription>;
private sub3Promise: Promise<Subscription>;
constructor() {
this.subscribeToThings();
}
subscribeToThings() {
// No longer async!
this.sub1Promise = subscribeToThingOne();
this.sub2Promise = subscribeToThingTwo();
this.sub3Promise = subscribeToThingThree();
}
cleanupOnDeath () {
this.sub1Promise.then(s => s.unsubscribe());
this.sub2Promise.then(s => s.unsubscribe());
this.sub3Promise.then(s => s.unsubscribe());
}
}
There's no standard way to cancel a promise - there's discussions going on in the various spec committees, but it's a hard problem for lots of reasons that don't help here.
I do question if this is a legitimate issue - are subscriptions really slow enough to worry about this issue? I'm not familiar with Autobahn.
But, assuming it is, one thing that could work would be to not await the stuff immediately, but instead to hold onto the actual promises. Then you can tack a .then handler to clean stuff up when needed. Something like this:
class Example {
private sub1: Subscription;
private sub2: Subscription;
private sub3: Subscription;
private sub1Promise: Promise<Subscription>;
private sub2Promise: Promise<Subscription>;
private sub3Promise: Promise<Subscription>;
constructor() {
subscribeToThings();
}
async subscribeToThings() {
// *** NOTE - no await here ***
this.sub1Promise = subscribeToThingOne();
this.sub2Promise = subscribeToThingTwo();
this.sub3Promise = subscribeToThingThree();
this.sub1 = await this.sub1Promise;
this.sub2 = await this.sub2Promise;
this.sub3 = await this.sub3Promise;
}
cleanupOnDeath () {
// Unsubscribe each promise. Don't need to check for null,
// they were set in subscribeToThings
this.sub1Promise.then((s) => s.unsubscribe());
this.sub2Promise.then((s) => s.unsubscribe());
this.sub3Promise.then((s) => s.unsubscribe());
}
}
Another solutions is to run logic of subscribeToThings method via synchronous executor nsynjs. It is not promise-basesd, therefore no need to cancel promises. But it tracks currently running slow functions with callbacks, and allows to free resources, occupied by them (e.g. issue xhr.abort, or clearTimeout) in case if caller function is requested to stop. Here is how it could be done:
Step 1. Wrap underlying function with callback into nsynjs-aware wrapper. See wait and ajaxGetJson functions in this example: https://github.com/amaksr/nsynjs/blob/master/examples/browser-ajax-seq.js
Step 2. Put subscription logic into separate function:
function syn_SubscribeToThings() {
function subscribeToThingOne() {
...
var data = ajaxGetJson(nsynjsCtx,'subscribe/url1').data;
...
};
function subscribeToThingOneTwo {
...
var data = ajaxGetJson(nsynjsCtx,'subscribe/url2').data;
...
};
function subscribeToThingOneThree {
...
var data = ajaxGetJson(nsynjsCtx,'subscribe/url3').data;
...
};
subscribeToThingOne();
subscribeToThingOneTwo();
subscribeToThingOneThree();
}
Step 3. Run that function via nsynjs form your subscribeToThings method:
subscribeToThings() {
this.subscribeCtx = nsynjs.run(syn_SubscribeToThings,{},function(){});
}
Step 4. Stop it whenever you need:
cleanupOnDeath () {
this.subscribeCtx.stop();
}
It will stop execution of syn_SubscribeToThings and cancel all active underlying callbacks automatically.
I was trying to use promises to force serialization of a series of Ajax calls. These Ajax calls are made one for each time a user presses a button. I can successfully serialize the operations like this:
// sample async function
// real-world this is an Ajax call
function delay(val) {
log("start: ", val);
return new Promise(function(resolve) {
setTimeout(function() {
log("end: ", val);
resolve();
}, 500);
});
}
// initialize p to a resolved promise
var p = Promise.resolve();
var v = 1;
// each click adds a new task to
// the serially executed queue
$("#run").click(function() {
// How to detect here that there are no other unresolved .then()
// handlers on the current value of p?
p = p.then(function() {
return delay(v++);
});
});
Working demo: http://jsfiddle.net/jfriend00/4hfyahs3/
But, this builds a potentially never ending promise chain since the variable p that stores the last promise is never cleared. Every new operation just chains onto the prior promise. So, I was thinking that for good memory management, I should be able to detect when there are no more .then() handlers left to run on the current value of p and I can then reset the value of p, making sure that any objects that the previous chain of promise handlers might have held in closures will be eligible for garbage collection.
So, I was wondering how I would know in a given .then() handler that there are no more .then() handlers to be called in this chain and thus, I can just do p = Promise.resolve() to reset p and release the previous promise chain rather than just continually adding onto it.
I'm being told that a "good" promise implementation would not cause accumulating memory from an indefinitely growing promise chain. But, there is apparently no standard that requires or describes this (other than good programming practices) and we have lots of newbie Promise implementations out there so I have not yet decided if it's wise to rely on this good behavior.
My years of coding experience suggest that when implementations are new, facts are lacking that all implementations behave a certain way and there's no specification that says they should behave that way, then it might be wise to write your code in as "safe" a way as possible. In fact, it's often less work to just code around an uncertain behavior than it is to go test all relevant implementations to find out how they behave.
In that vein, here's an implementation of my code that seems to be "safe" in this regard. It just saves a local copy of the global last promise variable for each .then() handler and when that .then() handler runs, if the global promise variable still has the same value, then my code has not chained any more items onto it so this must be the currently last .then() handler. It seems to work in this jsFiddle:
// sample async function
// real-world this is an Ajax call
function delay(val) {
log("start: ", val);
return new Promise(function(resolve) {
setTimeout(function() {
log("end: ", val);
resolve();
}, 500);
});
}
// initialize p to a resolved promise
var p = Promise.resolve();
var v = 1;
// each click adds a new task to
// the serially executed queue
$("#run").click(function() {
var origP = p = p.then(function() {
return delay(v++);
}).then(function() {
if (p === origP) {
// no more are chained by my code
log("no more chained - resetting promise head");
// set fresh promise head so no chance of GC leaks
// on prior promises
p = Promise.resolve();
v = 1;
}
// clear promise reference in case this closure is leaked
origP = null;
}, function() {
origP = null;
});
});
… so that I can then reset the value of p, making sure that any objects that the previous chain of promise handlers might have held in closures will be eligible for garbage collection.
No. A promise handler that has been executed (when the promise has settled) is no more needed and implicitly eligible for garbage collection. A resolved promise does not hold onto anything but the resolution value.
You don't need to do "good memory management" for promises (asynchronous values), your promise library does take care of that itself. It has to "release the previous promise chain" automatically, if it doesn't then that's a bug. Your pattern works totally fine as is.
How do you know when the promise chain has completely finished?
I would take a pure, recursive approach for this:
function extendedChain(p, stream, action) {
// chains a new action to p on every stream event
// until the chain ends before the next event comes
// resolves with the result of the chain and the advanced stream
return Promise.race([
p.then(res => ({res}) ), // wrap in object to distinguish from event
stream // a promise that resolves with a .next promise
]).then(({next, res}) =>
next
? extendedChain(p.then(action), next, action) // a stream event happened first
: {res, next:stream}; // the chain fulfilled first
);
}
function rec(stream, action, partDone) {
return stream.then(({next}) =>
extendedChain(action(), next, action).then(({res, next}) => {
partDone(res);
return rec(next, action, partDone);
});
);
}
var v = 1;
rec(getEvents($("#run"), "click"), () => delay(v++), res => {
console.log("all current done, none waiting");
console.log("last result", res);
}); // forever
with a helper function for event streams like
function getEvents(emitter, name) {
var next;
function get() {
return new Promise((res) => {
next = res;
});
}
emitter.on(name, function() {
next({next: get()});
});
return get();
}
(Demo at jsfiddle.net)
It is impossible to detect when no more handlers are added.
It is in fact an undecidable problem. It is not very hard to show a reduction to the halting (or the Atm problem). I can add a formal reduction if you'd like but in handwavey: Given an input program, put a promise at its first line and chain to it at every return or throw - assuming we have a program that solves the problem you describe in this question - apply it to the input problem - we now know if it runs forever or not solving the halting problem. That is, your problem is at least as hard as the halting problem.
You can detect when a promise is "resolved" and update it on new ones.
This is common in "last" or in "flatMap". A good use case is autocomplete search where you only want the latest results. Here is an [implementation by Domenic
(https://github.com/domenic/last):
function last(operation) {
var latestPromise = null; // keep track of the latest
return function () {
// call the operation
var promiseForResult = operation.apply(this, arguments);
// it is now the latest operation, so set it to that.
latestPromise = promiseForResult;
return promiseForResult.then(
function (value) {
// if we are _still_ the last value when it resovled
if (latestPromise === promiseForResult) {
return value; // the operation is done, you can set it to Promise.resolve here
} else {
return pending; // wait for more time
}
},
function (reason) {
if (latestPromise === promiseForResult) { // same as above
throw reason;
} else {
return pending;
}
}
);
};
};
I adapted Domenic's code and documented it for your problem.
You can safely not optimize this
Sane promise implementations do not keep promises which are "up the chain", so setting it to Promise.resolve() will not save memory. If a promise does not do this it is a memory leak and you should file a bug against it.
I tried to check if we can see the promise's state in code, apprantly that is only possible from console, not from code, so I used a flag to moniter the status, not sure if there is a loophole somewhere:
var p
, v = 1
, promiseFulfilled = true;
function addPromise() {
if(!p || promiseFulfilled){
console.log('reseting promise...');
p = Promise.resolve();
}
p = p.then(function() {
promiseFulfilled = false;
return delay(v++);
}).then(function(){
promiseFulfilled = true;
});
}
fiddle demo
You could push the promises onto an array and use Promise.all:
var p = Promise.resolve,
promiseArray = [],
allFinishedPromise;
function cleanup(promise, resolvedValue) {
// You have to do this funkiness to check if more promises
// were pushed since you registered the callback, though.
var wereMorePromisesPushed = allFinishedPromise !== promise;
if (!wereMorePromisesPushed) {
// do cleanup
promiseArray.splice(0, promiseArray.length);
p = Promise.resolve(); // reset promise
}
}
$("#run").click(function() {
p = p.then(function() {
return delay(v++);
});
promiseArray.push(p)
allFinishedPromise = Promise.all(promiseArray);
allFinishedPromise.then(cleanup.bind(null, allFinishedPromise));
});
Alternatively, since you know they are executed sequentially, you could have each completion callback remove that promise from the array and just reset the promise when the array is empty.
var p = Promise.resolve(),
promiseArray = [];
function onPromiseComplete() {
promiseArray.shift();
if (!promiseArray.length) {
p = Promise.resolve();
}
}
$("#run").click(function() {
p = p.then(function() {
onPromiseComplete();
return delay(v++);
});
promiseArray.push(p);
});
Edit: If the array is likely to get very long, though, you should go with the first option b/c shifting the array is O(N).
Edit: As you noted, there's no reason to keep the array around. A counter will work just fine.
var p = Promise.resolve(),
promiseCounter = 0;
function onPromiseComplete() {
promiseCounter--;
if (!promiseCounter) {
p = Promise.resolve();
}
}
$("#run").click(function() {
p = p.then(function() {
onPromiseComplete();
return delay(v++);
});
promiseCounter++;
});
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 7 years ago.
Improve this question
How is a promise/defer library like q implemented? I was trying to read the source code but found it pretty hard to understand, so I thought it'd be great if someone could explain to me, from a high level, what are the techniques used to implement promises in single-thread JS environments like Node and browsers.
I find it harder to explain than to show an example, so here is a very simple implementation of what a defer/promise could be.
Disclaimer: This is not a functional implementation and some parts of the Promise/A specification are missing, This is just to explain the basis of the promises.
tl;dr: Go to the Create classes and example section to see full implementation.
Promise:
First we need to create a promise object with an array of callbacks. I'll start working with objects because it's clearer:
var promise = {
callbacks: []
}
now add callbacks with the method then:
var promise = {
callbacks: [],
then: function (callback) {
callbacks.push(callback);
}
}
And we need the error callbacks too:
var promise = {
okCallbacks: [],
koCallbacks: [],
then: function (okCallback, koCallback) {
okCallbacks.push(okCallback);
if (koCallback) {
koCallbacks.push(koCallback);
}
}
}
Defer:
Now create the defer object that will have a promise:
var defer = {
promise: promise
};
The defer needs to be resolved:
var defer = {
promise: promise,
resolve: function (data) {
this.promise.okCallbacks.forEach(function(callback) {
window.setTimeout(function () {
callback(data)
}, 0);
});
},
};
And needs to reject:
var defer = {
promise: promise,
resolve: function (data) {
this.promise.okCallbacks.forEach(function(callback) {
window.setTimeout(function () {
callback(data)
}, 0);
});
},
reject: function (error) {
this.promise.koCallbacks.forEach(function(callback) {
window.setTimeout(function () {
callback(error)
}, 0);
});
}
};
Note that the callbacks are called in a timeout to allow the code be always asynchronous.
And that's what a basic defer/promise implementation needs.
Create classes and example:
Now lets convert both objects to classes, first the promise:
var Promise = function () {
this.okCallbacks = [];
this.koCallbacks = [];
};
Promise.prototype = {
okCallbacks: null,
koCallbacks: null,
then: function (okCallback, koCallback) {
okCallbacks.push(okCallback);
if (koCallback) {
koCallbacks.push(koCallback);
}
}
};
And now the defer:
var Defer = function () {
this.promise = new Promise();
};
Defer.prototype = {
promise: null,
resolve: function (data) {
this.promise.okCallbacks.forEach(function(callback) {
window.setTimeout(function () {
callback(data)
}, 0);
});
},
reject: function (error) {
this.promise.koCallbacks.forEach(function(callback) {
window.setTimeout(function () {
callback(error)
}, 0);
});
}
};
And here is an example of use:
function test() {
var defer = new Defer();
// an example of an async call
serverCall(function (request) {
if (request.status === 200) {
defer.resolve(request.responseText);
} else {
defer.reject(new Error("Status code was " + request.status));
}
});
return defer.promise;
}
test().then(function (text) {
alert(text);
}, function (error) {
alert(error.message);
});
As you can see the basic parts are simple and small. It will grow when you add other options, for example multiple promise resolution:
Defer.all(promiseA, promiseB, promiseC).then()
or promise chaining:
getUserById(id).then(getFilesByUser).then(deleteFile).then(promptResult);
To read more about the specifications: CommonJS Promise Specification. Note that main libraries (Q, when.js, rsvp.js, node-promise, ...) follow Promises/A specification.
Hope I was clear enough.
Edit:
As asked in the comments, I've added two things in this version:
The possibility to call then of a promise, no matter what status it has.
The possibility to chain promises.
To be able to call the promise when resolved you need to add the status to the promise, and when the then is called check that status. If the status is resolved or rejected just execute the callback with its data or error.
To be able to chain promises you need to generate a new defer for each call to then and, when the promise is resolved/rejected, resolve/reject the new promise with the result of the callback. So when the promise is done, if the callback returns a new promise it is bound to the promise returned with the then(). If not, the promise is resolved with the result of the callback.
Here is the promise:
var Promise = function () {
this.okCallbacks = [];
this.koCallbacks = [];
};
Promise.prototype = {
okCallbacks: null,
koCallbacks: null,
status: 'pending',
error: null,
then: function (okCallback, koCallback) {
var defer = new Defer();
// Add callbacks to the arrays with the defer binded to these callbacks
this.okCallbacks.push({
func: okCallback,
defer: defer
});
if (koCallback) {
this.koCallbacks.push({
func: koCallback,
defer: defer
});
}
// Check if the promise is not pending. If not call the callback
if (this.status === 'resolved') {
this.executeCallback({
func: okCallback,
defer: defer
}, this.data)
} else if(this.status === 'rejected') {
this.executeCallback({
func: koCallback,
defer: defer
}, this.error)
}
return defer.promise;
},
executeCallback: function (callbackData, result) {
window.setTimeout(function () {
var res = callbackData.func(result);
if (res instanceof Promise) {
callbackData.defer.bind(res);
} else {
callbackData.defer.resolve(res);
}
}, 0);
}
};
And the defer:
var Defer = function () {
this.promise = new Promise();
};
Defer.prototype = {
promise: null,
resolve: function (data) {
var promise = this.promise;
promise.data = data;
promise.status = 'resolved';
promise.okCallbacks.forEach(function(callbackData) {
promise.executeCallback(callbackData, data);
});
},
reject: function (error) {
var promise = this.promise;
promise.error = error;
promise.status = 'rejected';
promise.koCallbacks.forEach(function(callbackData) {
promise.executeCallback(callbackData, error);
});
},
// Make this promise behave like another promise:
// When the other promise is resolved/rejected this is also resolved/rejected
// with the same data
bind: function (promise) {
var that = this;
promise.then(function (res) {
that.resolve(res);
}, function (err) {
that.reject(err);
})
}
};
As you can see, it has grown quite a bit.
Q is a very complex promise library in terms of implementation because it aims to support pipelining and RPC type scenarios. I have my own very bare bones implementation of the Promises/A+ specification here.
In principle it's quite simple. Before the promise is settled/resolved, you keep a record of any callbacks or errbacks by pushing them into an array. When the promise is settled you call the appropriate callbacks or errbacks and record what result the promise was settled with (and whether it was fulfilled or rejected). After it's settled, you just call the callbacks or errbacks with the stored result.
That gives you aproximately the semantics of done. To build then you just have to return a new promise that is resolved with the result of calling the callbacks/errbacks.
If you're interested in a full explenation of the reasonning behind the development of a full on promise implementation with support for RPC and pipelining like Q, you can read kriskowal's reasonning here. It's a really nice graduated approach that I can't recommend highly enough if you are thinking of implementing promises. It's probably worth a read even if you're just going to be using a promise library.
As Forbes mentions in his answer, I chronicled many of the design decisions involved in making a library like Q, here https://github.com/kriskowal/q/tree/v1/design. Suffice it to say, there are levels of a promise library, and lots of libraries that stop at various levels.
At the first level, captured by the Promises/A+ specification, a promise is a proxy for an eventual result and is suitable for managing “local asynchrony”. That is, it is suitable for ensuring that work occurs in the right order, and for ensuring that it is simple and straight-forward to listen for the result of an operation regardless of whether it already settled, or will occur in the future. It also makes it just as simple for one or many parties to subscribe to an eventual result.
Q, as I have implemented it, provides promises that are proxies for eventual, remote, or eventual+remote results. To that end, it’s design is inverted, with different implementations for promises—deferred promises, fulfilled promises, rejected promises, and promises for remote objects (the last being implemented in Q-Connection). They all share the same interface and work by sending and receiving messages like "then" (which is sufficient for Promises/A+) but also "get" and "invoke". So, Q is about “distributed asynchrony”, and exists on another layer.
However, Q was actually taken down from a higher layer, where promises are used for managing distributed asynchrony among mutually suspicious parties like you, a merchant, a bank, Facebook, the government—not enemies, maybe even friends, but sometimes with conflicts of interest. The Q that I implemented is designed to be API compatible with hardened security promises (which is the reason for separating promise and resolve), with the hope that it would introduce people to promises, train them in using this API, and allow them to take their code with them if they need to use promises in secure mashups in the future.
Of course, there are trade-offs as you move up the layers, usually in speed. So, promises implementations can also be designed to co-exist. This is where the concept of a “thenable” enters. Promise libraries at each layer can be designed to consume promises from any other layer, so multiple implementations can coexist, and users can buy only what they need.
All this said, there is no excuse for being difficult to read. Domenic and I are working on a version of Q that will be more modular and approachable, with some of its distracting dependencies and work-arounds moved into other modules and packages. Thankfully folks like Forbes, Crockford, and others have filled in the educational gap by making simpler libraries.
First make sure you're understanding how Promises are supposed to work. Have a look at the CommonJs Promises proposals and the Promises/A+ specification for that.
There are two basic concepts that can be implemented each in a few simple lines:
A Promise does asynchronously get resolved with the result. Adding callbacks is a transparent action - independent from whether the promise is resolved already or not, they will get called with the result once it is available.
function Deferred() {
var callbacks = [], // list of callbacks
result; // the resolve arguments or undefined until they're available
this.resolve = function() {
if (result) return; // if already settled, abort
result = arguments; // settle the result
for (var c;c=callbacks.shift();) // execute stored callbacks
c.apply(null, result);
});
// create Promise interface with a function to add callbacks:
this.promise = new Promise(function add(c) {
if (result) // when results are available
c.apply(null, result); // call it immediately
else
callbacks.push(c); // put it on the list to be executed later
});
}
// just an interface for inheritance
function Promise(add) {
this.addCallback = add;
}
Promises have a then method that allows chaining them. I takes a callback and returns a new Promise which will get resolved with the result of that callback after it was invoked with the first promise's result. If the callback returns a Promise, it will get assimilated instead of getting nested.
Promise.prototype.then = function(fn) {
var dfd = new Deferred(); // create a new result Deferred
this.addCallback(function() { // when `this` resolves…
// execute the callback with the results
var result = fn.apply(null, arguments);
// check whether it returned a promise
if (result instanceof Promise)
result.addCallback(dfd.resolve); // then hook the resolution on it
else
dfd.resolve(result); // resolve the new promise immediately
});
});
// and return the new Promise
return dfd.promise;
};
Further concepts would be maintaining a separate error state (with an extra callback for it) and catching exceptions in the handlers, or guaranteeing asynchronity for the callbacks. Once you add those, you've got a fully functional Promise implementation.
Here is the error thing written out. It unfortunately is pretty repetitive; you can do better by using extra closures but then it get's really really hard to understand.
function Deferred() {
var callbacks = [], // list of callbacks
errbacks = [], // list of errbacks
value, // the fulfill arguments or undefined until they're available
reason; // the error arguments or undefined until they're available
this.fulfill = function() {
if (reason || value) return false; // can't change state
value = arguments; // settle the result
for (var c;c=callbacks.shift();)
c.apply(null, value);
errbacks.length = 0; // clear stored errbacks
});
this.reject = function() {
if (value || reason) return false; // can't change state
reason = arguments; // settle the errror
for (var c;c=errbacks.shift();)
c.apply(null, reason);
callbacks.length = 0; // clear stored callbacks
});
this.promise = new Promise(function add(c) {
if (reason) return; // nothing to do
if (value)
c.apply(null, value);
else
callbacks.push(c);
}, function add(c) {
if (value) return; // nothing to do
if (reason)
c.apply(null, reason);
else
errbacks.push(c);
});
}
function Promise(addC, addE) {
this.addCallback = addC;
this.addErrback = addE;
}
Promise.prototype.then = function(fn, err) {
var dfd = new Deferred();
this.addCallback(function() { // when `this` is fulfilled…
try {
var result = fn.apply(null, arguments);
if (result instanceof Promise) {
result.addCallback(dfd.fulfill);
result.addErrback(dfd.reject);
} else
dfd.fulfill(result);
} catch(e) { // when an exception was thrown
dfd.reject(e);
}
});
this.addErrback(err ? function() { // when `this` is rejected…
try {
var result = err.apply(null, arguments);
if (result instanceof Promise) {
result.addCallback(dfd.fulfill);
result.addErrback(dfd.reject);
} else
dfd.fulfill(result);
} catch(e) { // when an exception was re-thrown
dfd.reject(e);
}
} : dfd.reject); // when no `err` handler is passed then just propagate
return dfd.promise;
};
You might want to check out the blog post on Adehun.
Adehun is an extremely lightweight implementation (about 166 LOC) and very useful for learning how to implement the Promise/A+ spec.
Disclaimer: I wrote the blog post but the blog post does explain all about Adehun.
The Transition function – Gatekeeper for State Transition
Gatekeeper function; ensures that state transitions occur when all required conditions are met.
If conditions are met, this function updates the promise’s state and value. It then triggers the process function for further processing.
The process function carries out the right action based on the transition (e.g. pending to fulfilled) and is explained later.
function transition (state, value) {
if (this.state === state ||
this.state !== validStates.PENDING ||
!isValidState(state)) {
return;
}
this.value = value;
this.state = state;
this.process();
}
The Then function
The then function takes in two optional arguments (onFulfill and onReject handlers) and must return a new promise. Two major requirements:
The base promise (the one on which then is called) needs to create a new promise using the passed in handlers; the base also stores an internal reference to this created promise so it can be invoked once the base promise is fulfilled/rejected.
If the base promise is settled (i.e. fulfilled or rejected), then the appropriate handler should be called immediately. Adehun.js handles this scenario by calling process in the then function.
``
function then(onFulfilled, onRejected) {
var queuedPromise = new Adehun();
if (Utils.isFunction(onFulfilled)) {
queuedPromise.handlers.fulfill = onFulfilled;
}
if (Utils.isFunction(onRejected)) {
queuedPromise.handlers.reject = onRejected;
}
this.queue.push(queuedPromise);
this.process();
return queuedPromise;
}`
The Process function – Processing Transitions
This is called after state transitions or when the then function is invoked. Thus it needs to check for pending promises since it might have been invoked from the then function.
Process runs the Promise Resolution procedure on all internally stored promises (i.e. those that were attached to the base promise through the then function) and enforces the following Promise/A+ requirements:
Invoking the handlers asynchronously using the Utils.runAsync helper (a thin wrapper around setTimeout (setImmediate will also work)).
Creating fallback handlers for the onSuccess and onReject handlers if they are missing.
Selecting the correct handler function based on the promise state e.g. fulfilled or rejected.
Applying the handler to the base promise’s value. The value of this operation is passed to the Resolve function to complete the promise processing cycle.
If an error occurs, then the attached promise is immediately rejected.
function process() {
var that = this,
fulfillFallBack = function(value) {
return value;
},
rejectFallBack = function(reason) {
throw reason;
};
if (this.state === validStates.PENDING) {
return;
}
Utils.runAsync(function() {
while (that.queue.length) {
var queuedP = that.queue.shift(),
handler = null,
value;
if (that.state === validStates.FULFILLED) {
handler = queuedP.handlers.fulfill ||
fulfillFallBack;
}
if (that.state === validStates.REJECTED) {
handler = queuedP.handlers.reject ||
rejectFallBack;
}
try {
value = handler(that.value);
} catch (e) {
queuedP.reject(e);
continue;
}
Resolve(queuedP, value);
}
});
}
The Resolve function – Resolving Promises
This is probably the most important part of the promise implementation since it handles promise resolution. It accepts two parameters – the promise and its resolution value.
While there are lots of checks for various possible resolution values; the interesting resolution scenarios are two – those involving a promise being passed in and a thenable (an object with a then value).
Passing in a Promise value
If the resolution value is another promise, then the promise must adopt this resolution value’s state. Since this resolution value can be pending or settled, the easiest way to do this is to attach a new then handler to the resolution value and handle the original promise therein. Whenever it settles, then the original promise will be resolved or rejected.
Passing in a thenable value
The catch here is that the thenable value’s then function must be invoked only once (a good use for the once wrapper from functional programming). Likewise, if the retrieval of the then function throws an Exception, the promise is to be rejected immediately.
Like before, the then function is invoked with functions that ultimately resolve or reject the promise but the difference here is the called flag which is set on the first call and turns subsequent calls are no ops.
function Resolve(promise, x) {
if (promise === x) {
var msg = "Promise can't be value";
promise.reject(new TypeError(msg));
}
else if (Utils.isPromise(x)) {
if (x.state === validStates.PENDING){
x.then(function (val) {
Resolve(promise, val);
}, function (reason) {
promise.reject(reason);
});
} else {
promise.transition(x.state, x.value);
}
}
else if (Utils.isObject(x) ||
Utils.isFunction(x)) {
var called = false,
thenHandler;
try {
thenHandler = x.then;
if (Utils.isFunction(thenHandler)){
thenHandler.call(x,
function (y) {
if (!called) {
Resolve(promise, y);
called = true;
}
}, function (r) {
if (!called) {
promise.reject(r);
called = true;
}
});
} else {
promise.fulfill(x);
called = true;
}
} catch (e) {
if (!called) {
promise.reject(e);
called = true;
}
}
}
else {
promise.fulfill(x);
}
}
The Promise Constructor
And this is the one that puts it all together. The fulfill and reject functions are syntactic sugar that pass no-op functions to resolve and reject.
var Adehun = function (fn) {
var that = this;
this.value = null;
this.state = validStates.PENDING;
this.queue = [];
this.handlers = {
fulfill : null,
reject : null
};
if (fn) {
fn(function (value) {
Resolve(that, value);
}, function (reason) {
that.reject(reason);
});
}
};
I hope this helped shed more light into the way promises work.