I am trying to to make several requests that are asynchronous, then push the data obtained into an array by using for loop. However, I encountered a problem that forloop itself doesnt return value which is required to resolve promise.. Also, I would like to use the array for rendering for GET method.
So How do I complete promise within for loop and make sure that the array is filled before I render it for GET method. Sorry for my bad English. I am still new to the asynchronous concept, please teach me in easy words if possible. Also, if there are any better ways to solve the problem I would like to know.
const currencyName = ["btc", "eth", "xrp"];
let dataCollections = [];
for (i = 0; i < currencyName.length; i ++) {
dataCollections.push(new Promise(function(resolve, reject) {
request.get(`https://apiv2.bitcoinaverage.com/indices/global/ticker/${currencyName[i]}jpy`, function(error, response, body) {
if (error) {
reject(error);
} if (dataCollections.length === 3) {
dataColletions = [];
} else {
resolve(JSON.parse(body));
}
});
}));
}
//Promise.all(??).then(??)
//For rendering dataCollections
app.get("/", function(req, res) {
res.render("home", {dataCollections: dataCollections});
}
1) consider naming your array currencyNames plural so that you can name your iterator currencyName.
2) Your for loop is reinventing the array.map function. Consider using array.map instead.
3) Your for loop has an impossible if clause (if (dataCollections.length === 3)) which also means the else clause is on the wrong if statement. I assume this is a typo.
Now to the core of your question, yes Proimse.all(Array<Promise>) is what you want; it resolves when all elements in its param resolve.
const currencyNames = ["btc", "eth", "xrp"];
let dataCollections = currencyNames.map(currencyName => new Promise((resolve, reject) =>
request.get(`https://apiv2.bitcoinaverage.com/indices/global/ticker/${currencyName}jpy`, (error, response, body) => {
if (error)
reject(error);
else
resolve(JSON.parse(body));
})));
//For rendering dataCollections
app.get("/", async (req, res) =>
res.render("home", {dataCollections: await Promise.all(dataCollections)}));
Make the endpoint async:
app.get("/", async function(req, res) {
res.render("home", {dataCollections: await Promise.all(dataCollections) });
});
Related
I'm trying to limit the number of requests I send to an API.
I'm using Limiter and it's working just like I need, the only issue is that I can't find a way to use it with await (I need all the responses before rendering my page)
Can someone give me a hand with it?
Btw the Log returns a boolean.
const RateLimiter = require('limiter').RateLimiter;
const limiter = new RateLimiter(50, 5000)
for (let i = 0; i < arrayOfOrders.length; i++) {
const response = limiter.removeTokens(1, async (err, remainingRequests) => {
console.log('request')
return await CoreServices.load('updateOrder', {
"OrderNumber": arrayOfOrders[i],
"WorkFlowID": status
})
})
console.log('response', response)
}
console.log('needs to log after all the request');
this is loggin:
response true
response true
response false
needs to log after all the request
request
request
request
...
Promisifying .removeTokens will help, see if this code works
const RateLimiter = require('limiter').RateLimiter;
const limiter = new RateLimiter(50, 5000);
const tokenPromise = n => new Promise((resolve, reject) => {
limiter.removeTokens(n, (err, remainingRequests) => {
if (err) {
reject(err);
} else {
resolve(remainingRequests);
}
});
});
(async() => { // this line required only if this code is top level, otherwise use in an `async function`
const results = await Promise.all(arrayOfOrders.map(async (order) => {
await tokenPromise(1);
console.log('request');
return CoreServices.load('updateOrder', {
"OrderNumber": order,
"WorkFlowID": status
});
}));
console.log('needs to log after all the request');
})(); // this line required only if this code is top level, otherwise use in an `async function`
explanation
Firstly:
const tokenPromise = n => new Promise((resolve, reject) => {
limiter.removeTokens(n, (err, remainingRequests) => {
if (err) {
reject(err);
} else {
resolve(remainingRequests);
}
});
});
promisifies the limiter.removeTokens to use in async/await - in nodejs you could use the built in promisifier, however lately I've had too many instances where that fails - so a manual promisification (I'm making up a lot of words here!) works just as well
Now the code is easy - you can use arrayOfOrders.map rather than a for loop to create an array of promises that all run parallel as much as the rate limiting allows, (the rate limiting is done inside the callback)
await Promise.all(... will wait until all the CoreServices.load have completed (or one has failed - you could use await Promise.allSettled(... instead if you want)
The code in the map callback is tagged async so:
await tokenPromise(1);
will wait until the removeTokens callback is called - and then the request
return CoreServices.load
is made
Note, this was originally return await CoreServices.load but the await is redundant, as return await somepromise in an async function is just the same as return somepromise - so, adjust your code too
I am trying to load a certain page from my website only after the data is loaded from the database. However, it prints the results too early, giving an empty array, and it doesn't group the names at all. I believe this may be solved with async and await functions or callbacks, but I'm new to those and despite what I've searched, I still can't find the solution. So far, I've just been messing around with wrapping certain loops or functions in async functions and awaiting them. Other solutions have been focused on one .find() or other loop. Any help is appreciated! Thank you!
Routes.js:
app.get("/test", async (req, res) => {
var result = [];
users.findById(mongo.ObjectID("IDString"), async (err, res) => {
result = await res.getGroups(users, messages);
});
console.log(result);
//this should print [["name1"]["name1", "name2"],["name3", "name1"]]
res.redirect("/");
});
getGroups:
UserSchema.methods.getGroups = async function(users, messages) {
var result = [];
await messages.find({Group: this._id}, (err, group) => {
for(var i = 0; i < group.length; i++){
var members = group[i].Group;
var subArray = [];
members.forEach((memberID) => {
users.findById(memberID, (err, req) => {
console.log(req.name);
subArray.push(req.name);
console.log(subArray);
});
});
//After function finishes:
console.log(subArray);
result.push(subArray);
subArray = [];
}
});
return result;
};
In your first code snippet, you must set
console.log(result);
//this should print [["name1"]["name1", "name2"],["name3", "name1"]]
res.redirect("/");
in the callback function of findById. Like this:
app.get("/test", async (req, res) => {
var result = [];
users.findById(mongo.ObjectID("IDString"), async (err, res) => {
result = await res.getGroups(users, messages);
console.log(result);
//this should print [["name1"]["name1", "name2"],["name3", "name1"]]
res.redirect("/");
});
});
This is because of asynchronous nature of Javascript. Read the article below
https://medium.com/better-programming/is-javascript-synchronous-or-asynchronous-what-the-hell-is-a-promise-7aa9dd8f3bfb
If you want to use async/await paradigm, you can write this code like this:
app.get("/test", async (req, res) => {
var result = [];
result = await users.findById(mongo.ObjectID("IDString"));
console.log(result);
//this should print [["name1"]["name1", "name2"],["name3", "name1"]]
res.redirect("/");
});
In your second code snippet, you should use await OR callback. Just as explained above.
I'm trying to write some code that makes a call to an API, which will then return some URLs, from each of which I'll then need to make a new call. To avoid nesting loads of callbacks everywhere I've tried using async/await, but it doesn't seem to be working for some reason despite the extremely simple setup I've tried.
I've tried to create two async functions, one of which makes the call to any url and returns the response, and another one which will then call the first function several times depending on the response (i.e. how many urls are returned).
const request = require('request');
init();
async function init() {
const username = "x";
const archiveUrl = "https://someurl.com";
const archiveResponse = await apiCall(archiveUrl)
const archives = archiveResponse.archives;
console.log(archives);
}
async function apiCall(url) {
request(url, { json: true }, (err, res, body) => {
if (err) { return console.log(err); }
console.log(body);
return body;
});
}
This is literally my entire code at the moment and I'm not sure where it's going wrong. The error I'm getting is that it can't read .archives from undefined, and after that error message it's then logging the body from the apiCall function (making me fairly sure that the function isn't awaiting as expected).
That said I can't see what I'm doing wrong. Any tips on general best practice would be much appreciated, I've used async/await before but it's always been hacky and self-taught so I'm sure there's a better way of doing this.
In the apiCall function you are using callback instead of Promise, You need to promisify that function so that const archiveResponse = await apiCall(archiveUrl) actually works:
function apiCall(url) {
return new Promise((res, rej) => {
request(url, { json: true }, (err, res, body) => {
if (err) { return rej(err)}
console.log(body);
return res(body);
});
})
}
If you are using async-await please handle errors by enclosing this in try..catch
Note: Or you can use request-promis or axios they support promise out of the box.
You are close!
let response = await fetch('https://someurl.com');
let data = await response.json();
console.log(response);
decode the JSON body of the response with .json() and it should work
Need to parse some XML files from mass array with file_path values.
Try to use async, fs, xml2js.
When use single string file_path all works perfect. But when I use aync.filter() with array I can't understand how I can return result from xml.parseString()
const fs = require('fs');
const xml2js = require('xml2js');
const async = require('async');
var mass=['/file1.xml','/fil2.xml','/file3.xml',...]
async.filter(mass, async function(file_path, callback){
if(fs.statSync(file_path)['size']>0){
fs.readFileSync(file_path, 'utf8', function(err, data) {
xml.parseString(data, function (err, result) {
console.log(Object.keys(result)[0]);
return result; //need get this result to results array
})
})
}
}, function(err, results) {
console.log(results)
});
Who can understand how it works and what I need to change in my code.
Thanks a lot!
You are trying to map and filter at the same time. Since your filter condition is synchronously available, use the array filter method for that, and then pass that to async.map.
You should then call the callback function, that async.map provides to you, passing it the result. So don't return it, but call the callback.
The readFileSync method does not take a callback like its asynchronous counterpart. It just returns the data.
Also, drop the async keyword, as you are not using the await keyword at all.
async.map(mass.filter((file_path) => fs.statSync(file_path).size > 0),
function(file_path, callback){
var data = fs.readFileSync(file_path, 'utf8');
xml.parseString(data, function (err, result) {
console.log(Object.keys(result)[0]);
callback(null, result);
})
}, function(err, results) {
console.log(results)
});
It should be noted however, that since Node now comes with the Promise API, and even the async/await extension to that, the async module has become much less interesting. Consider using Promises.
const promises = mass.filter(file_path => {
return fs.statSync(file_path).size > 0
}).map(function(file_path) {
return new Promise(resolve => {
const data = fs.readFileSync(file_path, 'utf8');
xml.parseString(data, function (err, result) {
console.log(Object.keys(result)[0]);
resolve(result);
});
});
});
Promise.all(promises).then(results => {
console.log(results);
});
I'm having issues trying to work with api "pagination", I need to merge multiple objects into one. However, I can't seem to figure out how to do this with promises and I'm not sure how to work with async/await to get the best results.
I have a class called selly, which works with the Selly API.
getAllProducts(page = 1) {
return this.makeRequest('products', { page: page });
}
makeRequest(endpoint, params = {}) {
return axios
.get(`https://selly.gg/api/v2/${endpoint}`, {
headers: {
Authorization: `Basic ${this.genApiToken(this.apiKey, this.apiEmail)}`,
'User-Agent': `username - localhost`,
},
params: params,
})
.catch((err) => {
console.log(err);
});
}
Which worked great, until I realized I need to fetch multiple pages and combine all of the results into a single object. Here's my attempt:
app.get('/api/products', (req, res) => {
res.setHeader('Content-Type', 'application/json');
selly.getAllProducts().then((request) => {
const pages = request.headers['x-total-pages'];
let products = request.data;
if (pages > 1) {
let i = 2;
while (pages >= i) {
selly.getAllProducts(i).then((nextPageRequest) => {
nextPageRequest.data.forEach((item) => {
products.push(item);
});
});
i++;
}
}
res.send(JSON.stringify(products));
});
});
I can't seem to push nextPageRequest to products object. Obviously because res.send runs before the promise selly.getAllProducts(i) is finished.
I understand that the best optimization for this is using async and await, however I just can't get the basic concept of it, or well, at-least how would I refactor my code into using async and await.
How can I get this working?
You can make the whole app.get handler async, then await the inital request to figure out how many pages there are, then use a for loop and await additional calls of selly.getAllProducts, pushing the result to the products array. Make sure to try/catch so you can handle errors:
app.get('/api/products', async (req, res) => {
res.setHeader('Content-Type', 'application/json');
try {
const request = await selly.getAllProducts();
const pages = request.headers['x-total-pages'];
const products = request.data;
for (let i = 2; i <= pages; i++) {
const { data } = await selly.getAllProducts(i);
products.push(...data);
}
res.send(JSON.stringify(products));
} catch(e) {
// handle errors
}
});