I am sending batch requests to a moralis avalanche node which (half the time) returns token prices, but it only returns prices to the callback, I can't see the token addresses or anything related to the request I sent. Is there anyway to have some context sent to the callback? Or maybe a better way to send batch requests?
here is the callback:
const callbackMethod = (error, result) => {
if (error) {
console.log("error");
} else {
result.map((res) => {
console.log(res.toString())
})
}
}
and the requests:
const generateBatch = () => {
var batch = new web3.BatchRequest();
for (let i = 0; i < tokens.length; i++) {
for (let j = 0; j < tokens.length; j++) {
if (i != j) {
batch.add(contract.methods
.function(params)
.call.request({from: web3.eth.accounts.wallet.accounts["0"].address}, callbackMethod));
}
}
}
return batch;
}
My implementation was to write a wrapper around the function I was using (I was already going to deploy a contract for this project) and make it return both the prices and the path.
Related
I am trying to use recursive promises to fetch all children categories inside categories of scraped website. It seems that my code should work just fine but it only works if i use "reject" instead resolve. Somethings stopping promise from beeing fullfilled but i cannot manage to find the problem cause.
export async function ScrapeCategory(category) {
return new Promise(async (reject, resolve) => {
let promises = [];
var i = 1;
var ilen = category.pageCount;
while (i <= ilen) {
var res = await GetResponse(baseURL + category.link + "?pageId=" + i);
var $ = cheerio.load(res.data);
let productTiles = $(".product-link-ui");
var y = 0;
var ylen = productTiles.length;
while (y < ylen) {
promises.push(await FetchProducts(productTiles[y], category, i));
y++;
}
i++;
}
await Promise.all(promises).then(async (prodResult) => {
category.products = prodResult;
if (category.children) {
let childPromises = [];
var z = 0;
var zlen = category.children.length - 1;
while (z < zlen) {
childPromises.push(await ScrapeCategory(category.children[z]));
z++;
}
await Promise.all(childPromises).then((childResults) => {
console.log(childResults);
category.children = childResults;
});
}
});
resolve(category);
});
}
This is the promise that i try to fulfill. I appreaciate any kind of help.
It should work in that way that first it fetches products for the category, than when it is fulfilled it checks if category have child categories - if yes than it starts to fetch those categories. In the end it resolves main category with child categories inside as well.
Best regards
I tried to move code outside "then" function of promise all statement but there was no change. Problem seems to be with recursive statements that are never fullfilled (comes with undefined as return).
When i change resolve to reject the code works fine but it bothers me cause i know something is wrong and it cannot be the right sollution.
ChatGPT answered my question. So it seems like it should not return a promise at all (it's not needed) so code works fine when i changed it to this:
export async function ScrapeCategory(category) {
let promises = [];
var i = 1;
var ilen = category.pageCount;
while (i <= ilen) {
var res = await GetResponse(baseURL + category.link + "?pageId=" + i);
var $ = cheerio.load(res.data);
let productTiles = $(".product-link-ui");
var y = 0;
var ylen = productTiles.length;
while (y < ylen) {
promises.push(await FetchProducts(productTiles[y], category, i));
y++;
}
i++;
}
await Promise.all(promises).then(async (prodResult) => {
category.products = prodResult;
if (category.children) {
let childPromises = [];
var z = 0;
var zlen = category.children.length;
while (z < zlen) {
childPromises.push(await ScrapeCategory(category.children[z]));
z++;
}
await Promise.all(childPromises).then((childResults) => {
category.children = childResults;
});
}
});
return category;
}
I want to update multiple documents so I have to use loop but below code is broken because it has multiple res.json.
for (i = 0; i < dateArray.length; i ++ ) {
Trucks.update({ 'data.date': dateArray[i] }, {'$set': update}, {'multi':true}, function(err,response){
res.json(response);
})
}
I can put res.end() but I want to know when the operation is done.
How can I use async module to improve it? I want to know when the operation is done.
I don't whether it's good solution. It's just a hack.
I think it serves your need. Good luck.
var docsUpdated = 0;
var length = dateArray.length;
for (i = 0; i < length; i++) {
Trucks.update({
'data.date': dateArray[i]
}, {
'$set': update
}, {
'multi': true
}, function(err, response) {
if(err) res.end();
docsUpdated += response;
if( i === length-1){
res.json(docsUpdated);
}
})
}
I'm querying a mongo database to retrieve the tiles for the display in rougelike game. This is the function I use:
function get_display(){
var collections = ['austinsroom'];
var db = mongojs(uri, collections);
var temphtml = '';
for(var j = 0; j < 3; j++) {
console.log("y=" + String(j));
db.austinsroom.find({"y": j}, {}).sort({"x": 1}, function(err, records) {
if(err) {
console.log("There was an error executing the database query.");
return;
}
var i = records.length;
while(i--) {
temphtml += records[i].display;
}
temphtml += '<br>';
//console.log(temphtml);
//return temphtml;
//THE ONLY WAY I CAN GET ANYTHING TO PRINT IN THE CONSOLE IS IF I PUT IT INSIDE THE LOOP HERE
});
//console.log(temphtml);
//return temphtml;
//THIS DOES NOTHING
}
//console.log(temphtml);
//return temphtml;
//THIS DOES NOTHING
}
get_display();
If I put the console.log(temphtml) inside the loop, it prints out three times which isn't what I want. I only want the final string (i.e. ...<br>...<br>...<br>. Also I can't ultimately return the temphtml string, which is actually the important thing. Is this some quirk of javascript? Why would it not execute statements after the loop?
Also: is there a better way to retrieve every element of a grid that's stored in a mongo database, in order, so it can be displayed properly? Here's what the mongo documents look like:
{
"_id": {"$oid": "570a8ab0e4b050965a586957"},
"x": 0,
"y": 0,
"display": "."
}
Right now, the game is supposed to display a "." in all empty spaces using the x and y values for the coordinates. The database is indexed by "x" values.
See async.whilst. You want flow control of the for loop, for which this provides a callback to control each loop iteration.
var temphtml = "",
j = 0;
async.whilst(
function() { return j < 3 },
function(callback) {
db.austinsroom.find({"y": j }, {}).sort({"x": 1}, function(err, records)
temphtml += records.map(function(el) {
return el.display;
}).join("") + '<br>';
j++;
callback(err);
});
},
function(err) {
if (err) throw err;
console.log(temphtml);
}
)
Either that or use Promise.all() on collected promises to return "one big result". But you would also need to switch to promised-mongo from mongojs, as the nearest equivalent, since there are more mongodb drivers that actually support promises. That one is just the direct fork from mongojs:
var temphtml = "",
j = 0,
promises = [];
for ( var j=0; j < 3; j++ ) {
promises.push(db.austinsroom.find({"y": j }, {}).sort({"x": 1}).toArray());
promises.push('<br>'); // this will just join in the output
)
Promise.all(promises).then(function(records) {
temphtml += records.map(function(el) {
return el.display;
}).join("");
})
Not exactly the same thing, since it's one list output and not three, but the point is that the Promise objects defer until actually called to resolve, so you can feed the paramters in the loop, but execute later.
I do not use MongoDB but from what I am reading it is asynchronous. So what is happening is your db.austinsroom.find call fires another "thread" and returns to the for loop to continue the next iteration.
One way to do what you want is have a check at the end of your db.austinsroom.find function to see if you're done with the for loop. Something like:
function get_display()
{
var collections = ['austinsroom'];
var db = mongojs(uri, collections);
var temphtml = '';
var doneCounter = 0;
for(var j = 0; j < 3; j++)
{
console.log("y = " + String(j));
db.austinsroom.find({"y": j}, {}).sort({"x": 1}, function(err, records)
{
if(err)
{
console.log("There was an error executing the database query.");
return;
}
var i = records.length;
while(i--)
{
temphtml += records[i].display;
}
temphtml += '<br>';
// we're done with this find so add to the done counter
doneCounter++;
// at the end, check if the for loop is done
if(doneCounter == 3)
{
// done with all three DB calls
console.log(temphtml);
}
});
}
}
This is probably not the best way as I know nothing about MongoDB but it should put you on the right path.
db.listCollections().toArray(function(err, collections){
for(i = 1; i < collections.length; i++){
if(i < parseInt(collections.length)){
var collect = db.collection(collections[i].name.toString(),function(){
collect.count(function(err,result){
console.log(collections[i].name.toString());
corrCount = corrCount + result;
});
});
}else{
collect = collections[i].name;
}
}
});
So my problem is that collect ends up being undefined and thus one can't count the amount of entries in the collection. Something tells me that I should solve this with callback but I keep failing. Also I don't understand why the console prints out 2 times before it shots of the error. I'm using nodejs with mongodb native driver.
You need to move the rest of the code out of the callback, like so:
db.listCollections().toArray(function(err, collections) {
for (i = 1; i < collections.length; i++) {
if (i < parseInt(collections.length)) {
var collect = db.collection(collections[i].name.toString());
collect.count(function(err, result) {
console.log(collections[i].name.toString());
corrCount = corrCount + result;
});
} else {
collect = collections[i].name;
}
}
});
I have this code. I want to loop through all Users in a database and for each user get a list of their Portfolios (which in turn is a collection of stocks):
calculatePortfolios: function(callback) {
var thisuser;
module.exports.getAllUsers(function(err, users) {
/* Loop through all users with i */
for(var i = 0; i < users.length; i++) {
console.log('i = ' + i);
console.log(users.length);
thisuser = users[i]; // Store current user.
/* Loop through all held stocks in current user's portfolio. */
module.exports.getPortfolio(thisuser.username, function(err, portfolio) {
for(var x = 0; x < portfolio.length; x++) {
console.log('x = ' + x);
var total = parseFloat((portfolio[x].held * portfolio[x].unitprice) + thisuser.cash);
console.log(total);
console.log(thisuser.username);
module.exports.updateInvestor(thisuser.username, total, function(err, result) {
console.log(thisuser.username + ' ' + total);
});
}
});
}
callback(err, 'OK');
});
},
The result I get is that all i indices (users) are looped through before all x indices (portfolios). Shouldn't x be an inner loop of i?
Is this something related to how Node.JS works?
Any help is much appreciated. Thank you.
the getPortfolio() is most likely an asynchronous operation, and depending on the internal implementation it may either queue the calls or translate them to http requests for example that may take time, when these requests are completed only then your callback function that operates with x will be called.
But meanwhile getPortfoli() call will return. That is exactly why you see it iterating fast over all your i's, and only then you callback starts getting called.
Lack of JOIN's is really confusing to users who come with SQL background to node.js (event-driven) and NoSQL databases.
Consider this flow for your task:
Load all users.
Iterate through each user and record field which is used for collection relationship (username in your case), usually it is _id.
2b. (Optional) make Object, where key - will be field of relationship, in your case username, and value - will be object it self.
Make another query to related collection with similar query: { _id: { $in: ids } } where ids will be array of user _id's. In your case you need list of usernames.
Iterate through each portfolio item and add it to user object. If 2b did - then no need of two iterations.
Users are done, and have portfolio data and ready to be used.
#PSL: Got it working with async.waterfall. Probably not the most elegant solution, but it works for now.
/**
* #description Calculate the value of an investor's portfolio.
* #function
*/
calculatePortfolios: function(callback) {
async.waterfall([
function(callback) {
var allUsers = [];
module.exports.getAllUsers(function(err, users) {
for(var i = 0; i < users.length; i++) {
allUsers.push(users[i]);
}
});
callback(null, allUsers);
},
function(allUsers, callback) {
var totals = [];
for(var i = 0; i < allUsers.length; i++) {
module.exports.getPortfolio(allUsers[i].username, function(err, portfolio) {
for(var x = 0; x < portfolio.length; x++) {
totals.push(parseFloat(portfolio[x].held * portfolio[x].unitprice));
}
});
}
callback(null, allUsers, totals);
},
function(allUsers, totals, callback) {
for(var i = 0; i < allUsers.length; i++) {
module.exports.updateInvestor
(allUsers[i].username, (totals[i] + allUsers[i].cash), function(err, result) {
console.log('Updated an investor!');
});
}
callback(null, 'OK');
}
]);
callback(null, 'OK');
},