So the easiest way to explain this is that I am trying to search your package.json for all your packages, then use the npm registry to see if there are new updates. I have most of that done. Accept for the following for loop:
import request from 'request'
export const findAllUpdates = (packageInfo) => {
for(prop in packageInfo) {
request('https://registry.npmjs.org/' + prop, function(err, resp) {
json = JSON.parse(resp.body);
if (json['dist-tags'].latest !== packageInfo[prop].version) {
// Do something, we are not the same version so we should
// update some counter.
}
});
}
}
Your packageInfo is a object of key=>value which represents either a dependency or a dev dependency from the package.json and the package-lock.json or the yarn.lock
The important part is we look at, in the above function, what you have installed, we then use the registry to get that package info and compare the latest version on the registry to that of what you have installed and then we want, ideally, to update the state of the component with the total count.
how ever the issue is that we have entered callback hell, especially with a for loop, looping over every package, making a request.
I cannot create a variable and store the response in there because it cannot be accessed after the request is made. I cannot see how using events here would work because you might have 40 packages installed as either dep or dev and thats a lot of events to fire.
And finally the proper solution might be to use promises, but the whole concept of promises is yet more callbacks with the .then(() => {}), .catch(() => {}) which puts me back at square one.
The goal is to call this in a component and have the state of that component be updated with the total amount of packages that have to be updated (or at least have new versions)
Any ideas? Am I going about this all wrong?
You're right to be looking at promises.
The first thing is to give yourself a promise-enabled version of request. There's a promisify function in util that takes a Node-callback-style function and returns a promise-enabled version. So:
import util from 'util';
// ...
const prequest = util.promisify(request);
(There's also an npm module, promisify, that can do entire APIs all at once.)
Then, do your requests, gathering the promises in an array (and using then on them to do any post-processing; then returns a new promise, so we're still good), and use Promise.all to wait until they all resolve (or any of them rejects);
So all together:
import request from 'request';
import util from 'util';
const prequest = util.promisify(request);
export const findAllUpdates = (packageInfo) => {
const updates = []
for (const prop in packageInfo) {
updates.push(prequest('https://registry.npmjs.org/' + prop).then(resp => {
const json = JSON.parse(resp.body);
if (json['dist-tags'].latest !== packageInfo[prop].version) {
// Do something, we are not the same version so we should
// update some counter.
}
// Perhaps return something here, it will be the resolution
// for the promise for this request; otherwise, the promise
// will resolve with `undefined`.
}));
}
return Promise.all(updates);
};
That overall function's promise will resolve to an array of the results of each promise in the array (in order), or (again) reject if any of them rejects.
I want to be able to suggest async/await, but it doesn't currently bring much to the table for a bunch of parallel promises. (There's been the occasional talk of await.all to do what Promise.all does, but it hasn't caught on [yet?].)
Having said that, though, if we break things up a bit it is a bit nicer with async/await:
import request from 'request';
import util from 'util';
const prequest = util.promisify(request);
const checkOne = async (prop) => {
const resp = await prequest('https://registry.npmjs.org/' + prop);
const json = JSON.parse(resp.body);
if (json['dist-tags'].latest !== packageInfo[prop].version) {
// Do something, we are not the same version so we should
// update some counter.
}
// Perhaps return something here, it will be the resolution
// for the promise for this request; otherwise, the promise
// will resolve with `undefined`.
};
export const findAllUpdates = (packageInfo) => {
const updates = []
for (const prop in packageInfo) {
updates.push(checkOne(prop);
}
return Promise.all(updates);
};
And of course, if all the properties in packageInfo are "own" properties (not inherited), findAllUpdates gets a lot simpler:
export const findAllUpdates = (packageInfo) => {
return Promise.all(Object.keys(packageInfo).map(checkOne));
};
Side note: I've added a few missing declarations above.
I would recommend using Promise. You can make several async calls, and then wait for them using Promise.all.
If you also return a Promise from your "findAllUpdates" method, then it is easy for the caller to just update its own state when the information is available.
In React that would be something like:
findAllUpdates(..)
.then(result => this.setState({
packagesNeedUpdate: result.filter(p => !p.latest).length
}));
I've created a simple example below that uses promises. It fakes the requests, but otherwise should be pretty accurate.
// Fake requests. Will return version '1.0.0' after one second.
const makeRequest = url => {
return new Promise(resolve => {
setTimeout(() => {
resolve(JSON.stringify({
'dist-tags': {
'latest': '1.0.0'
}
}, null, 2));
}, 1000);
});
}
const findAllUpdates = (packageInfo) => {
return new Promise(resolve => {
const promises = Object.keys(packageInfo)
.map(prop => makeRequest('https://registry.npmjs.org/' + prop).then(JSON.parse));
Promise.all(promises) // Wait for all promises
.then(jsons => {
const results = Object.keys(packageInfo)
.map((prop, i) => {
const latestInNpm = jsons[i]['dist-tags'].latest;
const current = packageInfo[prop].version
return {
prop,
latest: latestInNpm === current
}
});
resolve(results); // Return result to caller
});
});
}
function run() {
console.log('Loading..');
const packages = {
'react': {version: '0.0.1'},
'angular': {version: '1.0.0'},
'ember': {version: '0.5.0'},
'mithril': {version: '0.9.0'}
};
findAllUpdates(packages).then(result => {
const needUpdates = result.filter(p => !p.latest).length;
console.log('Need updates: ', needUpdates);
});
}
run();
Related
In node.js I am trying to get a list of bid and ask prices from a trade exchange website (within an async function). Within the foreach statement I can console.info() the object data(on each iteration) but when I put all of this into an array and then return it to another function it passes as 'undefined'.
const symbolPrice = async() => {
let symbolObj = {}
let symbolList = []
await bookTickers((error, ticker) => {
ticker.forEach(symbol => {
if (symbol.symbol.toUpperCase().startsWith(starts.toUpperCase())) {
symbolObj = {
symbol: symbol.symbol,
bid: symbol.bidPrice,
ask: symbol.askPrice
}
console.info(symbolObj);
}
symbolList.push(symbolObj)
});
const results = Promise.all(symbolList)
return results;
});
}
const symbolPriceTest = async() => {
const res = await symbolPrice(null, 'ETH', true);
console.log(res)
}
I have tried pretty much everything I can find on the internet like different awaits/Promise.all()'s. I do admit I am not as familiar with async coding as I would like to be.
So, if the basic problem here is to call bookTickers(), retrieve the asynchronous result from that call, process it and then get that processed result as the resolved value from calling symbolPrice(), then you can do that like this:
const { promisify } = require('util');
const bookTickersP = promisify(bookTickers);
async function symbolPrice(/* declare arguments here */) {
let symbolList = [];
const ticker = await bookTickersP(/* fill in arguments here */);
for (let symbol of ticker) {
if (symbol.symbol.toUpperCase().startsWith(starts.toUpperCase())) {
symbolList.push({
symbol: symbol.symbol,
bid: symbol.bidPrice,
ask: symbol.askPrice
});
}
}
return symbolList;
}
async function symbolPriceTest() {
const res = await symbolPrice(null, 'ETH', true);
console.log(res)
}
Things to learn from your original attempt:
Only use await when you are awaiting a promise.
Only use Promise.all() when you are passing it an array of promises (or an array of a mixture of values and promises).
Don't mix plain callback asynchronous functions with promises. If you don't have a promise-returning version of your asynchronous function, then promisify it so you do (as shown in my code above with bookTickersP().
Do not guess with async and await and just throw it somewhere hoping it will do something useful. You MUST know that you're awaiting a promise that is connected to the result you're after.
Don't reuse variables in a loop.
Your original implementation of symbolPrice() had no return value at the top level of the function (the only return value was inside a callback so that just returns from the callback, not from the main function). That's why symbolPrice() didn't return anything. Now, because you were using an asynchronous callback, you couldn't actually directly return the results anyway so other things had to be redone.
Just a few thoughts on organization that might be reused in other contexts...
Promisify book tickers (with a library, or pure js using the following pattern). This is just the api made modern:
async function bookTickersP() {
return new Promise((resolve, reject) => {
bookTickers((error, ticker) => {
error ? reject(error) : resolve(ticker);
})
});
}
Use that to shape data in the way that the app needs. This is your app's async model getter:
// resolves to [{ symbol, bid, ask }, ...]
async function getSymbols() {
const result = await bookTickersP();
return result.map(({ symbol, bidPrice, askPrice }) => {
return { symbol: symbol.toUpperCase(), bid: bidPrice, ask: askPrice }
});
}
Write business logic that does things with the model, like ask about particular symbols:
async function symbolPriceTest(search) {
const prefix = search.toUpperCase();
const symbols = await getSymbols();
return symbols.filter(s => s.symbol.startsWith(prefix));
}
symbolPriceTest('ETH').then(console.log);
I'm new to react. I have a user list that fetched once when the component mounts.
and before the results dispatched to the reducer, it needs to loop inside the userlist and get user fullname/title from another endpoint, then add a new property to userlist object.
I cant figure out how to wait for all promises (getUserById() function) to finish before calling a dispatch. I tried the solution here, but failed:
How to return many Promises and wait for them all before doing other stuff
code below just to illustrate what I want:
const {
listUsers,
fetchUserData
} = useContext(GlobalContext);
const getUserById = async (userId) => {
return sp.web.siteUsers.getById(userId).get().then(user => user.Title);
}
useEffect(() => {
sp.web.lists.getById("8C271450-D3F9-489C-B4FC-9C7470594466").items.get()
.then(userLists => {
userLists = userLists.map(list => {
if (list.Person_x0020_ResponsibleId) {
getUserById(list.Person_x0020_ResponsibleId).then(username => {
list['Person_Responsible'] = username; // -> fetch user fullname and title
})
} else { // -> if id is null
list['Person_Responsible'] = '-';
}
return list
});
fetchListSuccess(userLists); // -> dispatch result to reducer
});
}, []);
You can accomplish this using Promise.all. First you need an array of promises from your second API calls. Then we'll give this array to Promise.all, and it will wait until they all resolve.
I've rewritten using async/await syntax. It works the same as using .then with the promises, but when you're working with a promise chain that's this complex it's easier to follow with async/await.
useEffect(async () => {
const userLists = await sp.web.lists.getById('8C271450-D3F9-489C-B4FC-9C7470594466').items.get();
const promises = userLists.map(async (list) => {
if (list.Person_x0020_ResponsibleId) {
const username = await getUserById(list.Person_x0020_ResponsibleId);
list.Person_Responsible = username; // -> fetch user fullname and title
} else { // -> if id is null
list.Person_Responsible = '-';
}
return list;
});
await Promise.all(promises);
fetchListSuccess(userLists); // -> dispatch result to reducer
}, []);
A few notes:
You don't really need to reassign userLists in the map, because you're just adding a property to existing objects. This will happen without the map.
Now the map is being used to return an array of promises for your second API calls. This is used by the Promise.all to wait until all those promises resolve.
The function below is called to create a post
const createPost = (newPost) => {
app.database().ref('posts').push(newPost);
}
This is another version of the function
const postRef = app.database().ref('posts');
const createPost = (newPost) => {
postRef.push(newPost);
}
Which is preferable and why
Both will add data to the database, but the second is better:
const postRef = app.database().ref('posts');
const createPost = (newPost) => {
postRef.push(newPost);
}
Since postRef is referring to a root node in the database which you might use later in the js file.
push returns a promise which resolves when the write to the database is complete. If you don't wait for it to resolve you will not know if it fails as your function will have returned successfully. Instead you will get an Unhandled rejection. Also, anything that runs after the functions has returned will receive less CPU and memory in firebase. So rewrite it like this:
const createPost = async (newPost) => {
await app.database().ref('posts').push(newPost);
}
As to your original question you should not declare a variable if only used once, so I would opt for the first variation.
I know that promise.all() fails when even 1 of the promise is failed. I only want to try for failed promises and don't want to run promise.all() again.
Any recommendations on how I can achieve this in minimal way?
Promises are eager construct and model a value obtained asynchronously,
a Promise is produced using some kind of producer, like fetch for instance.
If you retain a reference to this producer then you can replay the nmechanism
that produced the Promise in the first place.
// producer function
function getData (arg) {
const result = new Promise();
return result.then(value => {
return { ok:true, value };
}, error => {
return {
ok: false,
value: error,
// retry is a function which calls the producer with the same arguments
retry: () => getData(arg)
};
})
}
Then if you have something like:
const data = [];
// Promise<{ok: boolean, value: any, retry?: function}>
// No promises will fail in this array
const asyncResults = data.map(getResults);
Promise.all(asyncResults)
.then((results) => {
const successes = results.filter(res => res.ok);
const retrys = results.filter(res => !res.ok).map(res => res.retry()); // retry all failed promises
})
Memory leaks, stack overflow: because I retain a reference to original arguments in order to retry and the algorithm is recursive there could be a memory leak. However the algorithm cannot "stack overflow":
getData calls do not get "deeper" over time (see retry definition)
the asyncrhonicity of the algorithm prevent this behaviour if a promise was never resolved
old data is properly discarded when accessing the results as const resultData = results.filter(res => res.ok).map(res => res.value);
However the algorithm could take a long time to settle if a promise keep on not getting resolved and prevent access to the rest of the values.
In an alternative I suggest you take a look at another async primitive, not yet part of the language (maybe some day) : Observables which are designed for this kind of tasks: lazy, retry-able async operations.
You may use async package and wrap all promise calls with closure with done argument.
Then simply resolve results.
const async = require('async');
const promiseAll = promises => {
return new Promise((resolve) => {
// preparing array of functions which has done method as callback
const parallelCalls = promises.map(promise => {
return done => {
promise
.then(result => done(null, result)
.catch(error => {
console.error(error.message);
done();
});
}
});
// calling array of functions in parallel
async.parallel(
parallelCalls,
(_, results) => resolve(results.filter(Boolean))
);
});
};
router.get('/something', async (req, res) => {
...
const results = await promiseAll(promises);
...
});
Or we can simply do Promise.all without using async package:
router.get('/something', async (req, res) => {
...
const results = (
await Promise.all(
promises.map(promise => {
return new Promise(resolve => {
promise.then(resolve).catch(e => resolve());
});
});
)
).filter(Boolean);
...
});
I'm trying to solve an issue which involves waiting for ES6 imports. I'm using localstorage to set the values inside of a object properties. First off they all start out as false, then as they get imported, they're individually set to true. This operation takes less than a second. The problem is, if the code continues execution, it does so before some components are loaded.
I've tried a few different things to only call a callback (to actually continue execution) once all values are truthy, but i just can't seem to come up with a feasible clean solution.
Basically i need something like this:
let exampleObj = {
value1: true,
value2: true,
value3: false
}
function check() {
for (let key in exampleObj) {
if (!key) {
// exampleObj[key] is false, so rerun this whole check
check()
}
}
}
function start() {
check()
// if check() doesn't rerun, it means no `key` was still set to false.
// therefor carry on and mount the app
mountSomething()
}
start()
I can't wrap my head around how to achieve this.
Any help would be much appreciated.
Edit:
Some more details to help everyone a bit more.
Basically the premise of this is to enable/disable features on the client side (built with Vue). So it goes something like this:
Vue router is setup with only a few routes
App starts by Fetching the feature flags from an API
If a given feature is set to true, I run a helper method to import the component tied to that feature, then use vue router's addRoutes feature to include that route
In the import then() method, i set a localstorage value to true, since i know that the component has indeed been imported successfully
Quick snapshot:
const features = {}
const featureChecker = (feature, activity = false) => {
features[feature] = activity
localStorage.setItem('Features', JSON.stringify(features))
}
export const configureSomeFeatureRoute = (router) => {
featureChecker('SomeComponent')
import('#/path/to/SomeComponent.vue')
.then(Home => {
router.addRoutes([{
name: 'some-component',
path: '/some-path',
component: SomeComponent.default
}])
featureChecker('SomeComponent', true)
})
}
After this, a start() method is called which is meant to then loop through my localstorage Features object and check that every single prop is set to true before continuing execution and mounting my Vue app. This is the part i'm struggling with.
Just use Promise.all.
let importsContainer = [];
let import1 =
import ('#/path/to/SomeComponent.vue')
.then(Home => {
router.addRoutes([{
name: 'some-component',
path: '/some-path',
component: SomeComponent.default
}]);
});
importsContainer.push(import1);
let import2 =
import ('#/path/to/SomeComponent.vue')
.then(Home => {
router.addRoutes([{
name: 'some-component',
path: '/some-path',
component: SomeComponent.default
}]);
});
importsContainer.push(import2);
Promise.all(importsContainer).then((imports) => {
// everything is imported, do something or
});
Try putting it wrapping it in Promises and use a setTimeout to wait for the variables to be set. Then use Promise.all, this waits for all the Promise to be resolve. Then you want do your logic in the .then function. Of course in your example you won't res(1) but resolve with the values of your localstorage.
const promise1 = new Promise((res, rej) => setTimeout(() => res(1), 2000))
const promise2 = new Promise((res, rej) => setTimeout(() => res(1), 2000))
const promise3 = new Promise((res, rej) => setTimeout(() => res(1), 2000))
Promise.all([promise1, promise2, promise3]).then((values) => {
console.log(values)
// your start function here.
});