I've been working on a small 2D game library for my own use, and I've run into a bit of a problem. There is a particular function in the library called loadGame that takes dependency info as input (resource files, and a list of scripts ot be executed). Here's an example.
loadGame({
"root" : "/source/folder/for/game/",
"resources" : {
"soundEffect" : "audio/sound.mp3",
"someImage" : "images/something.png",
"someJSON" : "json/map.json"
},
"scripts" : [
"js/helperScript.js",
"js/mainScript.js"
]
})
Each item in resources has a key that is used by the game to access that particular resource. The loadGame function converts the resources into an object of promises.
The problem is that it tries to use Promises.all to check for when they're all ready, but Promise.all accepts only iterables as inputs - so an object like what I have is out of the question.
So I tried to convert the object into an array, this works great, except each resource is just an element in an array and doesn't have a key to identify them.
Here's the code for loadGame:
var loadGame = function (game) {
return new Promise(function (fulfill, reject) {
// the root folder for the game
var root = game.root || '';
// these are the types of files that can be loaded
// getImage, getAudio, and getJSON are defined elsewhere in my code - they return promises
var types = {
jpg : getImage,
png : getImage,
bmp : getImage,
mp3 : getAudio,
ogg : getAudio,
wav : getAudio,
json : getJSON
};
// the object of promises is created using a mapObject function I made
var resources = mapObject(game.resources, function (path) {
// get file extension for the item
var extension = path.match(/(?:\.([^.]+))?$/)[1];
// find the correct 'getter' from types
var get = types[extension];
// get it if that particular getter exists, otherwise, fail
return get ? get(root + path) :
reject(Error('Unknown resource type "' + extension + '".'));
});
// load scripts when they're done
// this is the problem here
// my 'values' function converts the object into an array
// but now they are nameless and can't be properly accessed anymore
Promise.all(values(resources)).then(function (resources) {
// sequentially load scripts
// maybe someday I'll use a generator for this
var load = function (i) {
// load script
getScript(root + game.scripts[i]).then(function () {
// load the next script if there is one
i++;
if (i < game.scripts.length) {
load(i);
} else {
// all done, fulfill the promise that loadGame returned
// this is giving an array back, but it should be returning an object full of resources
fulfill(resources);
}
});
};
// load the first script
load(0);
});
});
};
Ideally I'd like for some way to properly manage a list of promises for resources while still mantaining an identifier for each item. Any help would be appreciated, thanks.
First of all: Scrap that Promise constructor, this usage is an antipattern!
Now, to your actual problem: As you have correctly identified, you are missing the key for each value. You will need to pass it inside each promise, so that you can reconstruct the object after having awaited all items:
function mapObjectToArray(obj, cb) {
var res = [];
for (var key in obj)
res.push(cb(obj[key], key));
return res;
}
return Promise.all(mapObjectToArray(input, function(arg, key) {
return getPromiseFor(arg, key).then(function(value) {
return {key: key, value: value};
});
}).then(function(arr) {
var obj = {};
for (var i=0; i<arr.length; i++)
obj[arr[i].key] = arr[i].value;
return obj;
});
Mightier libraries such as Bluebird will also provide this as a helper function, like Promise.props.
Also, you shouldn't use that pseudo-recursive load function. You can simply chain promises together:
….then(function (resources) {
return game.scripts.reduce(function(queue, script) {
return queue.then(function() {
return getScript(root + script);
});
}, Promise.resolve()).then(function() {
return resources;
});
});
If you use lodash library, you can achieve this by a one-liner function:
Promise.allValues = async (object) => {
return _.zipObject(_.keys(object), await Promise.all(_.values(object)))
}
I actually created a library just for that and published it to github and npm:
https://github.com/marcelowa/promise-all-properties
https://www.npmjs.com/package/promise-all-properties
The only thing is that you will need to assign a property name for each promise in the object...
here's an example from the README
import promiseAllProperties from 'promise-all-properties';
const promisesObject = {
someProperty: Promise.resolve('resolve value'),
anotherProperty: Promise.resolve('another resolved value'),
};
const promise = promiseAllProperties(promisesObject);
promise.then((resolvedObject) => {
console.log(resolvedObject);
// {
// someProperty: 'resolve value',
// anotherProperty: 'another resolved value'
// }
});
Here is a simple ES2015 function that takes an object with properties that might be promises and returns a promise of that object with resolved properties.
function promisedProperties(object) {
let promisedProperties = [];
const objectKeys = Object.keys(object);
objectKeys.forEach((key) => promisedProperties.push(object[key]));
return Promise.all(promisedProperties)
.then((resolvedValues) => {
return resolvedValues.reduce((resolvedObject, property, index) => {
resolvedObject[objectKeys[index]] = property;
return resolvedObject;
}, object);
});
}
Usage:
promisedProperties({a:1, b:Promise.resolve(2)}).then(r => console.log(r))
//logs Object {a: 1, b: 2}
class User {
constructor() {
this.name = 'James Holden';
this.ship = Promise.resolve('Rocinante');
}
}
promisedProperties(new User).then(r => console.log(r))
//logs User {name: "James Holden", ship: "Rocinante"}
Note that #Bergi's answer will return a new object, not mutate the original object. If you do want a new object, just change the initializer value that is passed into the reduce function to {}
Here is #Matt's answer, with some types and some renames, and using ECMA-2019 Object.fromEntries.
// delayName :: (k, Promise a) -> Promise (k, a)
const delayName = ([name, promise]) => promise.then((result) => [name, result]);
export type PromiseValues<TO> = {
[TK in keyof TO]: Promise<TO[TK]>;
};
// promiseObjectAll :: {k: Promise a} -> Promise {k: a}
export const promiseObjectAll = <T>(object: PromiseValues<T>): Promise<T> => {
const promiseList = Object.entries(object).map(delayName);
return Promise.all(promiseList).then(Object.fromEntries);
};
function resolveObject(obj) {
return Promise.all(
Object.entries(obj).map(async ([k, v]) => [k, await v])
).then(Object.fromEntries);
}
credit to Cyril Auburtin for that genius at https://esdiscuss.org/topic/modify-promise-all-to-accept-an-object-as-a-parameter
function delay(ms) {
return new Promise(resolve => setTimeout(resolve, ms));
}
async function test() {
console.time(1);
console.log(await resolveObject({
foo: delay(5).then(()=>1),
bar: delay(120).then(()=>2)
}));
console.timeEnd(1);
}
Using async/await and lodash:
// If resources are filenames
const loadedResources = _.zipObject(_.keys(resources), await Promise.all(_.map(resources, filename => {
return promiseFs.readFile(BASE_DIR + '/' + filename);
})))
// If resources are promises
const loadedResources = _.zipObject(_.keys(resources), await Promise.all(_.values(resources)));
Based off the accepted answer here, I thought I'd offer a slightly different approach that seems easier to follow:
// Promise.all() for objects
Object.defineProperty(Promise, 'allKeys', {
configurable: true,
writable: true,
value: async function allKeys(object) {
const resolved = {}
const promises = Object
.entries(object)
.map(async ([key, promise]) =>
resolved[key] = await promise
)
await Promise.all(promises)
return resolved
}
})
// usage
Promise.allKeys({
a: Promise.resolve(1),
b: 2,
c: Promise.resolve({})
}).then(results => {
console.log(results)
})
Promise.allKeys({
bad: Promise.reject('bad error'),
good: 'good result'
}).then(results => {
console.log('never invoked')
}).catch(error => {
console.log(error)
})
Usage:
try {
const obj = await Promise.allKeys({
users: models.User.find({ rep: { $gt: 100 } }).limit(100).exec(),
restrictions: models.Rule.find({ passingRep: true }).exec()
})
console.log(obj.restrictions.length)
} catch (error) {
console.log(error)
}
I looked up Promise.allKeys() to see if someone had already implemented this after writing this answer, and apparently this npm package does have an implementation for it, so use that if you like this little extension.
I've written function that recursively awaits promises within an object and returns the constructed object back to you.
/**
* function for mimicking async action
*/
function load(value) {
return new Promise((resolve, reject) => {
setTimeout(() => {
resolve(value);
}, Math.random() * 1000);
});
}
/**
* Recursively iterates over object properties and awaits all promises.
*/
async function fetch(obj) {
if (obj instanceof Promise) {
obj = await obj;
return fetch(obj);
} else if (Array.isArray(obj)) {
return await Promise.all(obj.map((item) => fetch(item)));
} else if (obj.constructor === Object) {
const keys = Object.keys(obj);
for (let i = 0; i < keys.length; i++) {
const key = keys[i];
obj[key] = await fetch(obj[key]);
}
return obj;
} else {
return obj;
}
}
// now lets load a world object which consists of a bunch of promises nested in each other
let worldPromise = {
level: load('world-01'),
startingPoint: {
x: load('0'),
y: load('0'),
},
checkpoints: [
{
x: load('10'),
y: load('20'),
}
],
achievments: load([
load('achievement 1'),
load('achievement 2'),
load('achievement 3'),
]),
mainCharacter: {
name: "Artas",
gear: {
helmet: load({
material: load('steel'),
level: load(10),
}),
chestplate: load({
material: load('steel'),
level: load(20),
}),
boots: load({
material: load('steel'),
level: load(20),
buff: load('speed'),
}),
}
}
};
//this will result an object like this
/*
{
level: Promise { <pending> },
startingPoint: {
x: Promise { <pending> },
y: Promise { <pending> }
},
checkpoints: [ { x: [Promise], y: [Promise] } ],
achievments: Promise { <pending> },
mainCharacter: {
name: 'Artas',
gear: {
helmet: [Promise],
chestplate: [Promise],
boots: [Promise]
}
}
}
*/
//Now by calling fetch function, all promise values will be populated
//And you can see that computation time is ~1000ms which means that all processes are being computed in parallel.
(async () => {
console.time('start');
console.log(worldPromise);
let world = await fetch(worldPromise);
console.log(world);
console.timeEnd('start');
})();
Missing Promise.obj() method
A shorter solution with vanilla JavaScript, no libraries, no loops, no mutation
Here is a shorter solution than other answers, using modern JavaScript syntax.
The middle line process = ... is recursive and handles deep objects.
This creates a missing Promise.obj() method that works like Promise.all() but for objects:
const asArray = obj => [].concat(...Object.entries(obj));
const process = ([key, val, ...rest], aggregated = {}) =>
rest.length ?
process(rest, {...aggregated, [key]: val}) :
{...aggregated, [key]: val};
const promisedAttributes = obj => Promise.all(asArray(obj)).then(process);
// Promise.obj = promisedAttributes;
Better not use the last line! A much better idea is that you export this promisedAttributes as a utility function that you reuse.
Edit: This question seems to be gaining a little traction lately, so I thought I'd add my current solution to this problem which I'm using in a couple projects now. It's a lot better than the code at the bottom of this answer which I wrote two years ago.
The new loadAll function assume its input is an object mapping asset names to promises, and it also makes use of the experimental function Object.entries, which may not be available in all environments.
// fromEntries :: [[a, b]] -> {a: b}
// Does the reverse of Object.entries.
const fromEntries = list => {
const result = {};
for (let [key, value] of list) {
result[key] = value;
}
return result;
};
// addAsset :: (k, Promise a) -> Promise (k, a)
const addAsset = ([name, assetPromise]) =>
assetPromise.then(asset => [name, asset]);
// loadAll :: {k: Promise a} -> Promise {k: a}
const loadAll = assets =>
Promise.all(Object.entries(assets).map(addAsset)).then(fromEntries);
So I've come up with the proper code based on Bergi's answer. Here it is if anyone else is having the same problem.
// maps an object and returns an array
var mapObjectToArray = function (obj, action) {
var res = [];
for (var key in obj) res.push(action(obj[key], key));
return res;
};
// converts arrays back to objects
var backToObject = function (array) {
var object = {};
for (var i = 0; i < array.length; i ++) {
object[array[i].name] = array[i].val;
}
return object;
};
// the actual load function
var load = function (game) {
return new Promise(function (fulfill, reject) {
var root = game.root || '';
// get resources
var types = {
jpg : getImage,
png : getImage,
bmp : getImage,
mp3 : getAudio,
ogg : getAudio,
wav : getAudio,
json : getJSON
};
// wait for all resources to load
Promise.all(mapObjectToArray(game.resources, function (path, name) {
// get file extension
var extension = path.match(/(?:\.([^.]+))?$/)[1];
// find the getter
var get = types[extension];
// reject if there wasn't one
if (!get) return reject(Error('Unknown resource type "' + extension + '".'));
// get it and convert to 'object-able'
return get(root + path, name).then(function (resource) {
return {val : resource, name : name};
});
// someday I'll be able to do this
// return get(root + path, name).then(resource => ({val : resource, name : name}));
})).then(function (resources) {
// convert resources to object
resources = backToObject(resources);
// attach resources to window
window.resources = resources;
// sequentially load scripts
return game.scripts.reduce(function (queue, path) {
return queue.then(function () {
return getScript(root + path);
});
}, Promise.resolve()).then(function () {
// resources is final value of the whole promise
fulfill(resources);
});
});
});
};
So for these promises
const obj = { foo: promise1, bar: promise2 }
if you want object of values then await the promises
const newObj = {}
for (let key in obj)
newObj[key] = await obj[key]
but if you want an object of settled promises await them all without reassigning.
await Promise.all(Object.values(obj))
I recommend Sindre Sorhus' p-props. His stuff is always excellent.
One simple and easiest way to do it is
Promise.all([yourObject]).then((result)=>{
yourObject={...result}
}).catch((error)=>{console.log(error)})
Create function:
const promiseAllFromObject = async promisesObject => (
Object.keys(promisesObject).reduce(async (acc, key) => {
const lastResult = await acc;
return Object.assign(lastResult, { [key]: await promisesObject[key] });
}, Promise.resolve({}))
);
Usage:
promiseAllFromObject({
abc: somePromise,
xyz: someOtherPromise,
});
Result:
{
abc: theResult,
xyz: theOtherResult,
}
Need this, including good TypeScript support?
combine-promises can mix object values of different types and infer a good return type.
https://github.com/slorber/combine-promises
const result: { user: User; company: Company } = await combinePromises({
user: fetchUser(),
company: fetchCompany(),
});
Related
Think of how Rails, e.g. allows you to define a property as associated with another:
class Customer < ActiveRecord::Base
has_many :orders
end
This does not set up a database column for orders. Instead, it creates a getter for orders, which allows us to do
#orders = #customer.orders
Which goes and gets the related orders objects.
In JS, we can easily do that with getters:
{
name: "John",
get orders() {
// get the order stuff here
}
}
But Rails is sync, and in JS, if in our example, as is reasonable, we are going to the database, we would be doing it async.
How would we create async getters (and setters, for that matter)?
Would we return a promise that eventually gets resolved?
{
name: "John",
get orders() {
// create a promise
// pseudo-code for db and promise...
db.find("orders",{customer:"John"},function(err,data) {
promise.resolve(data);
});
return promise;
}
}
which would allow us to do
customer.orders.then(....);
Or would we do it more angular-style, where we would automatically resolve it into a value?
To sum, how do we implement async getters?
The get and set function keywords seem to be incompatible with the async keyword. However, since async/await is just a wrapper around Promises, you can just use a Promise to make your functions "await-able".
Note: It should be possible to use the Object.defineProperty method to assign an async function to a setter or getter.
getter
Promises work well with getters.
Here, I'm using the Node.js 8 builtin util.promisify() function that converts a node style callback ("nodeback") to a Promise in a single line. This makes it very easy to write an await-able getter.
var util = require('util');
class Foo {
get orders() {
return util.promisify(db.find)("orders", {customer: this.name});
}
};
// We can't use await outside of an async function
(async function() {
var bar = new Foo();
bar.name = 'John'; // Since getters cannot take arguments
console.log(await bar.orders);
})();
setter
For setters, it gets a little weird.
You can of course pass a Promise to a setter as an argument and do whatever inside, whether you wait for the Promise to be fulfilled or not.
However, I imagine a more useful use-case (the one that brought me here!) would be to use to the setter and then awaiting that operation to be completed in whatever context the setter was used from. This unfortunately is not possible as the return value from the setter function is discarded.
function makePromise(delay, val) {
return new Promise(resolve => {
setTimeout(() => resolve(val), delay);
});
}
class SetTest {
set foo(p) {
return p.then(function(val) {
// Do something with val that takes time
return makePromise(2000, val);
}).then(console.log);
}
};
var bar = new SetTest();
var promisedValue = makePromise(1000, 'Foo');
(async function() {
await (bar.foo = promisedValue);
console.log('Done!');
})();
In this example, the Done! is printed to the console after 1 second and the Foo is printed 2 seconds after that. This is because the await is waiting for promisedValue to be fulfilled and it never sees the Promise used/generated inside the setter.
As for asynchronous getters, you may just do something like this:
const object = {};
Object.defineProperty(object, 'myProperty', {
async get() {
// Your awaited calls
return /* Your value */;
}
});
Rather, the problem arises when it comes to asynchronous setters.
Since the expression a = b always produce b, there is nothing one can do to avoid this, i.e. no setter in the object holding the property a can override this behavior.
Since I stumbled upon this problem as well, I could figure out asynchronous setters were literally impossible. So, I realized I had to choose an alternative design for use in place of async setters. And then I came up with the following alternative syntax:
console.log(await myObject.myProperty); // Get the value of the property asynchronously
await myObject.myProperty(newValue); // Set the value of the property asynchronously
I got it working with the following code,
function asyncProperty(descriptor) {
const newDescriptor = Object.assign({}, descriptor);
delete newDescriptor.set;
let promise;
function addListener(key) {
return callback => (promise || (promise = descriptor.get()))[key](callback);
}
newDescriptor.get = () => new Proxy(descriptor.set, {
has(target, key) {
return Reflect.has(target, key) || key === 'then' || key === 'catch';
},
get(target, key) {
if (key === 'then' || key === 'catch')
return addListener(key);
return Reflect.get(target, key);
}
});
return newDescriptor;
}
which returns a descriptor for an asynchronous property, given another descriptor that is allowed to define something that looks like an asynchronous setter.
You can use the above code as follows:
function time(millis) {
return new Promise(resolve => setTimeout(resolve, millis));
}
const object = Object.create({}, {
myProperty: asyncProperty({
async get() {
await time(1000);
return 'My value';
},
async set(value) {
await time(5000);
console.log('new value is', value);
}
})
});
Once you've set up with an asynchronous property like the above, you can set it as already illustrated:
(async function() {
console.log('getting...');
console.log('value from getter is', await object.myProperty);
console.log('setting...');
await object.myProperty('My new value');
console.log('done');
})();
The following allows for async setters in proxy handlers following the convention in Davide Cannizzo's answer.
var obj = new Proxy({}, asyncHandler({
async get (target, key, receiver) {
await new Promise(a => setTimeout(a, 1000))
return target[key]
},
async set (target, key, val, receiver) {
await new Promise(a => setTimeout(a, 1000))
return target[key] = val
}
}))
await obj.foo('bar') // set obj.foo = 'bar' asynchronously
console.log(await obj.foo) // 'bar'
function asyncHandler (h={}) {
const getter = h.get
const setter = h.set
let handler = Object.assign({}, h)
handler.set = () => false
handler.get = (...args) => {
let promise
return new Proxy(()=>{}, {
apply: (target, self, argv) => {
return setter(args[0], args[1], argv[0], args[2])
},
get: (target, key, receiver) => {
if (key == 'then' || key == 'catch') {
return callback => {
if (!promise) promise = getter(...args)
return promise[key](callback)
}
}
}
})
}
return handler
}
Here's another approach to this. It creates an extra wrapper, but in other aspects it covers what one would expect, including usage of await (this is TypeScript, just strip the : Promise<..> bit that sets return value type to get JS):
// this doesn't work
private get async authedClientPromise(): Promise<nanoClient.ServerScope> {
await this.makeSureAuthorized()
return this.client
}
// but this does
private get authedClientPromise(): Promise<nanoClient.ServerScope> {
return (async () => {
await this.makeSureAuthorized()
return this.client
})()
}
Here's how you could implement your get orders function
function get(name) {
return new Promise(function(resolve, reject) {
db.find("orders", {customer: name}, function(err, data) {
if (err) reject(err);
else resolve(data);
});
});
}
You could call this function like
customer.get("John").then(data => {
// Process data here...
}).catch(err => {
// Process error here...
});
I would like to create an RxJS Observable from an iterable like the following:
const networkIterableFactory = (resource: string) => {
let i = 0;
return {
[Symbol.iterator]() {
return {
next() {
return {
done: false,
value: fetch(resource, {
mode: 'cors',
}).then(async response => {
console.log('i = ', i);
await throttle(10000); // Do some stuff
i++;
return {i: 'i'};
}),
};
},
};
},
};
};
function throttle(ms: number) {
return new Promise(resolve => setTimeout(resolve, ms));
}
let networkIterable = networkIterableFactory('google.com');
let network$ = rxjs.from(networkIterable).pipe(rxjs.operators.take(5));
network$.subscribe(() => console.log('yo!'));
Issue is that i prints 5 times as 0. It seems as though the way that the iterable's iterator saves its state is through updating the outer closure. rxjs.from just takes the whole iterable as one emmission so a bunch of unresolved promises are returned, but I need the iterator state to be altered by logic within the promise callback. Is there a way to make the observable wait until the promise resolves before emitting the next item from the iterator? I would rather avoid using asyncIterable because I don't want to bring in IxRx.
Since the values of your iterable are returned asynchronously, then you should implement Symbol.asyncIterator instead of Symbol.iterator Try this instead:
const networkIterableFactory = (resource: string) => {
let i = 0;
return {
[Symbol.asyncIterator]() {
return {
next() {
return fetch(resource, { mode: 'cors' }).then(x => ({ done: false, value: x }));
},
};
},
};
};
function throttle(ms: number) {
return new Promise(resolve => setTimeout(resolve, ms));
}
let networkIterable = networkIterableFactory('google.com');
let network$ = rxjs.from(networkIterable).pipe(rxjs.operators.take(5));
network$.subscribe(() => console.log('yo!'));
Edit:
RxJS actually doesn't support Async iterators yet: https://github.com/ReactiveX/rxjs/issues/1624.
I also tried with an AsyncGenerator:
const { from } = require('rxjs');
async function* test() {};
const asyncGenerator = test();
from(asyncGenerator);
But it throws:
TypeError: You provided an invalid object where a stream was expected.
You can provide an Observable, Promise, Array, or Iterable.
So you won't be able to make it with this pattern, I actually believe that RxJS is not suited for "pulling" data like you do with take (If this pattern worked, it would end up in an infinite requests loop, even if you only take 5 results). It is rather designed to "push" things to whoever listens.
Background
I am trying to filter an array of objects. Before I filter, I need to convert them to some format, and this operation is asynchronous.
const convert = () => new Promise( resolve => {
setTimeout( resolve, 1000 );
});
So, my first try was to do something like the following using async/await:
const objs = [ { id: 1, data: "hello" }, { id: 2, data: "world"} ];
objs.filter( async ( obj ) => {
await convert();
return obj.data === "hello";
});
Now, as some of you may know, Array.protoype.filter is a function which callback must return either true or false. filter is synchronous. In the previous example, I am returning none of them, I return a Promise ( all async functions are Promises ).
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/filter
So as one can assume, the code before doesn't really work... That assumption is correct.
Problem
To make filter work with an async function, I checked stackoverflow and found this topic:
Filtering an array with a function that returns a promise
Unfortunately, the chosen answer is overly complex and uses classes. This won't do for me. I am instead looking for a more simple solution, using simple functions with a functional approach.
There is one solution at the very end, using a map with a callback to simulate a filter:
https://stackoverflow.com/a/46842181/1337392
But I was hoping to fix my filter function, not to replace it.
Questions
Is there a way to have an async function inside a filter?
If not, what is the simplest replacement I can do?
There is no way to use filter with an async function (at least that I know of).
The simplest way that you have to use filter with a collection of promises is to use Promise.all and then apply the function to your collection of results.
It would look something like this:
const results = await Promise.all(your_promises)
const filtered_results = results.filter(res => //do your filtering here)
Hope it helps.
Adapted from the article How to use async functions with Array.filter in Javascript by Tamás Sallai, you basically have 2 steps:
One that creates the conditions for an object to pass
One that receives the objects and returns true or false according to conditions
Here's an example
const arr = [1, 2, 3, 4, 5];
function sleep(ms) {
return new Promise((resolve) => setTimeout(resolve, ms));
}
const asyncFilter = async (arr, predicate) => {
const results = await Promise.all(arr.map(predicate));
return arr.filter((_v, index) => results[index]);
}
const asyncRes = await asyncFilter(arr, async (i) => {
await sleep(10);
return i % 2 === 0;
});
console.log(asyncRes);
// 2,4
Use Scramjet fromArray/toArray methods...
const result = await scramjet.fromArray(arr)
.filter(async (item) => somePromiseReturningMethod(item))
.toArray();
as simple as that - here's a ready example to copy/paste:
const scramjet = require('../../');
async function myAsyncFilterFunc(data) {
return new Promise(res => {
process.nextTick(res.bind(null, data % 2));
});
}
async function x() {
const x = await scramjet.fromArray([1,2,3,4,5])
.filter(async (item) => myAsyncFilterFunc(item))
.toArray();
return x;
}
x().then(
(out) => console.log(out),
(err) => (console.error(err), process.exit(3)) // eslint-disable-line
);
Disclamer: I am the author of scramjet. :)
Build a parallel array to your array which you want to call filter on.
Await all of the promises from your filter func, in my eg, isValid.
In the callback in filter, use the 2nd arg, index, to index into your parallel array to determine if it should be filtered.
// ===============================================
// common
// ===============================================
const isValid = async (value) => value >= 0.5;
const values = [0.2, 0.3, 0.4, 0.5, 0.6];
// ===============================================
// won't filter anything
// ===============================================
const filtered = values.filter(async v => await isValid(v));
console.log(JSON.stringify(filtered));
// ===============================================
// filters
// ===============================================
(async () => {
const shouldFilter = await Promise.all(values.map(isValid));
const filtered2 = values.filter((value, index) => shouldFilter[index]);
console.log(JSON.stringify(filtered2));
})();
This behavior makes sense since any Promise instance has a truthy value, but it's not intuitive at a glance.
This answer uses library iter-ops, which handles iterable objects, and supports async filtering:
import {pipeAsync, filter, toAsync} from 'iter-ops';
// your input data:
const objs = [{id: 1, data: 'hello'}, {id: 2, data: 'world'}];
const i = pipeAsync(
objs,
filter(async value => {
await convert(); // any async function
return value.data === 'hello'; // filtering logic
})
);
for await(const a of i) {
console.log(a); // filtered data
}
P.S. I'm the author of iter-ops.
Reduce method can mimic filter and can operate with promises.
const isPositiveNumberAsync = async (number) => number >= 0;
const filterPositiveNumbersAsync = async (numbers) => numbers?.reduce(async (accumulatorPromise, number) => {
const accumulator = await accumulatorPromise;
if (await isPositiveNumberAsync(number)) {
return [...accumulator, number];
}
return accumulator;
}, Promise.resolve([])) || [];
(async () => {
// no numbers argument provided
console.log(await filterPositiveNumbersAsync());
// an empty argument list provided
console.log(await filterPositiveNumbersAsync([]));
// ok, but no positive numbers provided
console.log(await filterPositiveNumbersAsync([-1,-2,-3]));
// ok, positive numbers filtered.
console.log(await filterPositiveNumbersAsync([0,1,-1,-3,2,-2]));
})();
Array.prototype.asyncFilter =function( filterFn) {
const arr = this;
return new Promise(function(resolve){
const booleanArr = [];
arr.forEach(function (e) {
booleanArr.push(filterFn(e))
})
Promise.all(booleanArr).then(function (booleanArr) {
const arr2 = arr.filter(function (e, i) {
return booleanArr[i]
})
resolve(arr2)
})
})
}
/** use it like this**/
const arr=[1,2,3]
arr.asyncFilter(async e=>{}).then(...)
You can use Promise.filter from Bluebird that works similarly to Array.filter but it supports async & await.
Add asyncFilter as an extension to Array:
#available(macOS 10.15.0, *)
extension Array where Element: Any {
public func asyncFilter(closure: (Element) async -> Bool) async -> Array {
var result = [Element]()
for item in self {
if await closure(item) {
result.append(item)
}
}
return result
}
}
Usage:
result = await result.asyncFilter { item in
if <item match> {
return true
}
}
So I have a function which reads from Firebase database and then creates an array of objects. I have defined variable key, but it is still unavailable.
var usersList = [];
const ref = firebase.database().ref()
function fetchUsers() {
ref.child('users').once('value').then(snap => {
var promises = [];
snap.forEach(childSnap => {
var key = childSnap.key
promises.push
(ref.child(`users/${key}/points`).once('value')
);
});
return Promise.all(promises);
}).then(function(snapshots) {
return snapshots.map(snapper => {
var points = snapper.val()
return {uid: key, points: points};
})
}).then(function(usersList) {
console.log(usersList)
})
}
And this is the error I get...
(node:11724) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 1): ReferenceError: key is not defined
If i just do: key = childSnap.key, then every object's uid is the same.
key is only defined within the forEach callback. Naturally you cannot then reference it from within the second then callback. Moreover, which key would it be? For the first entry? The second? Third?
Instead, you need to return the key with the value:
function fetchUsers() {
ref.child('users').once('value').then(snap => {
var promises = [];
snap.forEach(childSnap => {
var key = childSnap.key;
promises.push(
ref.child(`users/${key}/points`).once('value').then(
snapper => ({key, snapper}) // **
)
);
});
return Promise.all(promises);
}).then(function(snapshots) {
return snapshots.map(({key, snapper}) => { // **
var points = snapper.val();
return {uid: key, points: points};
});
}).then(function(usersList) {
console.log(usersList);
});
}
On the first ** line above, we're transforming the result from once so it returns an object with both the key and "snapper".
On the second ** line, we're using destructured parameters to receive those objects as the distinct key and snapper parameters.
FWIW, if you want to aggressively use concise arrows:
function fetchUsers() {
ref.child('users').once('value').then(snap =>
Promise.all(snap.map(childSnap => {
const key = childSnap.key;
return ref.child(`users/${key}/points`)
.once('value')
.then(snapper => ({key, snapper}));
}))
).then(snapshots =>
snapshots.map(({key, snapper}) => ({uid: key, points: snapper.val()}))
).then(usersList => {
console.log(usersList);
});
}
Also note the use of map rather than declaring an array and pushing to it from a forEach callback. :-)
Think of how Rails, e.g. allows you to define a property as associated with another:
class Customer < ActiveRecord::Base
has_many :orders
end
This does not set up a database column for orders. Instead, it creates a getter for orders, which allows us to do
#orders = #customer.orders
Which goes and gets the related orders objects.
In JS, we can easily do that with getters:
{
name: "John",
get orders() {
// get the order stuff here
}
}
But Rails is sync, and in JS, if in our example, as is reasonable, we are going to the database, we would be doing it async.
How would we create async getters (and setters, for that matter)?
Would we return a promise that eventually gets resolved?
{
name: "John",
get orders() {
// create a promise
// pseudo-code for db and promise...
db.find("orders",{customer:"John"},function(err,data) {
promise.resolve(data);
});
return promise;
}
}
which would allow us to do
customer.orders.then(....);
Or would we do it more angular-style, where we would automatically resolve it into a value?
To sum, how do we implement async getters?
The get and set function keywords seem to be incompatible with the async keyword. However, since async/await is just a wrapper around Promises, you can just use a Promise to make your functions "await-able".
Note: It should be possible to use the Object.defineProperty method to assign an async function to a setter or getter.
getter
Promises work well with getters.
Here, I'm using the Node.js 8 builtin util.promisify() function that converts a node style callback ("nodeback") to a Promise in a single line. This makes it very easy to write an await-able getter.
var util = require('util');
class Foo {
get orders() {
return util.promisify(db.find)("orders", {customer: this.name});
}
};
// We can't use await outside of an async function
(async function() {
var bar = new Foo();
bar.name = 'John'; // Since getters cannot take arguments
console.log(await bar.orders);
})();
setter
For setters, it gets a little weird.
You can of course pass a Promise to a setter as an argument and do whatever inside, whether you wait for the Promise to be fulfilled or not.
However, I imagine a more useful use-case (the one that brought me here!) would be to use to the setter and then awaiting that operation to be completed in whatever context the setter was used from. This unfortunately is not possible as the return value from the setter function is discarded.
function makePromise(delay, val) {
return new Promise(resolve => {
setTimeout(() => resolve(val), delay);
});
}
class SetTest {
set foo(p) {
return p.then(function(val) {
// Do something with val that takes time
return makePromise(2000, val);
}).then(console.log);
}
};
var bar = new SetTest();
var promisedValue = makePromise(1000, 'Foo');
(async function() {
await (bar.foo = promisedValue);
console.log('Done!');
})();
In this example, the Done! is printed to the console after 1 second and the Foo is printed 2 seconds after that. This is because the await is waiting for promisedValue to be fulfilled and it never sees the Promise used/generated inside the setter.
As for asynchronous getters, you may just do something like this:
const object = {};
Object.defineProperty(object, 'myProperty', {
async get() {
// Your awaited calls
return /* Your value */;
}
});
Rather, the problem arises when it comes to asynchronous setters.
Since the expression a = b always produce b, there is nothing one can do to avoid this, i.e. no setter in the object holding the property a can override this behavior.
Since I stumbled upon this problem as well, I could figure out asynchronous setters were literally impossible. So, I realized I had to choose an alternative design for use in place of async setters. And then I came up with the following alternative syntax:
console.log(await myObject.myProperty); // Get the value of the property asynchronously
await myObject.myProperty(newValue); // Set the value of the property asynchronously
I got it working with the following code,
function asyncProperty(descriptor) {
const newDescriptor = Object.assign({}, descriptor);
delete newDescriptor.set;
let promise;
function addListener(key) {
return callback => (promise || (promise = descriptor.get()))[key](callback);
}
newDescriptor.get = () => new Proxy(descriptor.set, {
has(target, key) {
return Reflect.has(target, key) || key === 'then' || key === 'catch';
},
get(target, key) {
if (key === 'then' || key === 'catch')
return addListener(key);
return Reflect.get(target, key);
}
});
return newDescriptor;
}
which returns a descriptor for an asynchronous property, given another descriptor that is allowed to define something that looks like an asynchronous setter.
You can use the above code as follows:
function time(millis) {
return new Promise(resolve => setTimeout(resolve, millis));
}
const object = Object.create({}, {
myProperty: asyncProperty({
async get() {
await time(1000);
return 'My value';
},
async set(value) {
await time(5000);
console.log('new value is', value);
}
})
});
Once you've set up with an asynchronous property like the above, you can set it as already illustrated:
(async function() {
console.log('getting...');
console.log('value from getter is', await object.myProperty);
console.log('setting...');
await object.myProperty('My new value');
console.log('done');
})();
The following allows for async setters in proxy handlers following the convention in Davide Cannizzo's answer.
var obj = new Proxy({}, asyncHandler({
async get (target, key, receiver) {
await new Promise(a => setTimeout(a, 1000))
return target[key]
},
async set (target, key, val, receiver) {
await new Promise(a => setTimeout(a, 1000))
return target[key] = val
}
}))
await obj.foo('bar') // set obj.foo = 'bar' asynchronously
console.log(await obj.foo) // 'bar'
function asyncHandler (h={}) {
const getter = h.get
const setter = h.set
let handler = Object.assign({}, h)
handler.set = () => false
handler.get = (...args) => {
let promise
return new Proxy(()=>{}, {
apply: (target, self, argv) => {
return setter(args[0], args[1], argv[0], args[2])
},
get: (target, key, receiver) => {
if (key == 'then' || key == 'catch') {
return callback => {
if (!promise) promise = getter(...args)
return promise[key](callback)
}
}
}
})
}
return handler
}
Here's another approach to this. It creates an extra wrapper, but in other aspects it covers what one would expect, including usage of await (this is TypeScript, just strip the : Promise<..> bit that sets return value type to get JS):
// this doesn't work
private get async authedClientPromise(): Promise<nanoClient.ServerScope> {
await this.makeSureAuthorized()
return this.client
}
// but this does
private get authedClientPromise(): Promise<nanoClient.ServerScope> {
return (async () => {
await this.makeSureAuthorized()
return this.client
})()
}
Here's how you could implement your get orders function
function get(name) {
return new Promise(function(resolve, reject) {
db.find("orders", {customer: name}, function(err, data) {
if (err) reject(err);
else resolve(data);
});
});
}
You could call this function like
customer.get("John").then(data => {
// Process data here...
}).catch(err => {
// Process error here...
});