Develop Reference

Create loop in Javascript to use with imap-simple nodejs package

Firstly, some background as to what my test script will cover.
Pressing a button on a website will fire off an email to a test mailbox.
This email can take anything between 10 and 30 minutes to arrive in the test mailbox.
So using the following code from imap-simple ;
'get new email info': function(browser) {
imaps.connect(config).then(function (connection) {
return connection.openBox('INBOX').then(function () {
var searchCriteria = ['UNSEEN'];
var fetchOptions = {
bodies: ['HEADER', 'TEXT'],
markSeen: false
};
return connection.search(searchCriteria, fetchOptions).then(function (results) {
var subjects = results.map(function (res) {
return res.parts.filter(function (part) {
return part.which === 'HEADER';
})[0].body.subject[0];
});
console.log(subjects);
Correctly comes back with a blank subjects array, as the email hasn't been received by the test mailbox yet.
Adding a 30 minutes pause to the beginning of the script 'works', as after the 30 minutes the subjects array is populated as the email is (99.99% of the time) sent within a 30 minute window.
However, it is definitely far from ideal as the email might be received within 15 minutes, meaning the test is 'wasting' 15 minutes.
So what I'd ideally like to do is write some form of loop (?) that tests if the subjects array is populated or not.
So if the array is populated, carry on with the rest of the test script (which entails testing that the array contains a certain text).
If the array is not populated, wait for another minute before trying again.
Continue this trying every minute until the array is populated.
I've tried setInterval, For loops, While loops, etc but I can't seem to get them to work and I'm out of ideas to be honest.
Any advice, help, references would be greatly appreciated and any more info can be promptly added if required.

One way to do that could be using recursion.
const createPromise = ms => new Promise((resolve, reject) => {
setTimeout(() => resolve(ms), ms)
});
function findUnseenEmails(connection) {
return connection.openBox('INBOX').then(function () {
var searchCriteria = [
'UNSEEN'
];
var fetchOptions = {
bodies: ['HEADER', 'TEXT'],
markSeen: false
};
return connection.search(searchCriteria, fetchOptions).then(function (results) {
var subjects = results.map(function (res) {
return res.parts.filter(function (part) {
return part.which === 'HEADER';
})[0].body.subject[0];
});
console.log(subjects);
return subjects.length > 0 ? subjects : createPromise(5000).then(function() { return findUnseenEmails(connection)});
});
});
}
imaps.connect(config).then(function (connection) {
return findUnseenEmails(connection);
}).then((subjects) => console.log('finished', subjects));
Of course there is a possibility and danger of stack overflow, but in such scenario feel free to come back to stack overflow to find here with the help of our community non-recursive solution.
Result:
EDIT:
Answering your question regarding closing connection:
I'd do it like this (in findUnseenEmails function)
if (subjects.length > 0) {
connection.end();
return subjects;
} else {
return createPromise(5000).then(function() { return findUnseenEmails(connection)});
}

Issues updating firestore document by committing a batch after an async function

I'm writing a cloud function that uses request-promise and cheerio to scrape a website and then check that information against a user document.
I am not entirely familiar with Javascript and Cloud Functions.
I've come so far that I managed to extract the information I need and navigate to the user's document and compare the data. Now the last piece of this function is to give the user points for each matching data point, so I need to update a map inside the user document.
This function has to loop through all users and change their document if the data point matches. I'm not sure the way I've written the code is the most optimal in terms of performance and billing if the userbase gets huge... Any pointers to how I could minimize the impact on the task would be of great help, as im new with JS.
So this is the code:
exports.getV75Results = functions.pubsub.schedule('every 2 minutes').onRun(async (context) => {
let linkMap = new Map();
const url = `https://www.example.com`
const options = {
uri: url,
headers: { 'User-Agent': 'test' },
transform: (body) => cheerio.load(body)
}
await rp(options)
.then(($) => {
for(let i = 1; i <= 7; i++)
{
//Find player from game
const lopp1 = $(`#mainContentHolder > div > div.mainContentStyleTrot > div > div.panel-body > table:nth-child(1) > tbody > tr:nth-child(${i}) > td:nth-child(2) > span`).text()
const lopp1StrR1 = lopp1.replace("(", "");
const lopp1StrR2 = lopp1StrR1.replace(")", "");
const lopp1StrR3 = lopp1StrR2.replace(" ", "");
linkMap.set(i, lopp1StrR3.toUpperCase());
}
console.log(linkMap);
return linkMap;
}).then(async () => {
//Start lookup users
let usersRef = db.collection('fantasyfotball').doc('users');
usersRef.listCollections().then(collections => {
collections.forEach( collection => {
var user = collection.doc(collection.id);
let batch = new admin.firestore().batch();
user.get().then(function(doc) {
let json = doc.data();
//Look in users collection if players document exist
Object.keys(json).forEach((name) => {
if(name != null) {
//Document with users active fotball players
if(name == 'players') {
let i = 0;
Object.values(json[name]).forEach((value) => {
i++;
if(value.localeCompare(linkMap.get(i)) == 0) {
//Loop through user keys and find owned players if user has the correct player
Object.keys(json).forEach((map) => {
if(map != null)
{
//Document with a map of player owned fotball players, each respective player has a key = 'fotball player' and value = '[price, points]'
if(map == 'ownedplayers')
{
Object.entries(json[map]).forEach((players) => {
if(players[0].localeCompare(value) == 0) {
console.log(players[1][1]);
//Add points to respective player field
//PROBABLY NOT HOW TO CHANGE A DOCUMENT FILED, THIS DOESNT WORK..
players[1][1]++;
}
});
//EACH TIME THIS RUNS IT SAYS: "Cannot modify a WriteBatch that has been committed"
batch.update(user, {'ownedplayers': json[map]});
}
}
});
}
});
}
} else {
console.log('user does not have a playermode document.');
}
});
});
return batch.commit().then(function () {
console.log("Succesfully commited changes.");
return null;
});
});
});
}).catch((err) => {
return err;
});
});
The issues i get in the console are "Cannot modify a WriteBatch that has been committed." and I fail to modify and add points to the player field inside the users document.
This is the console:
This is the firestore document structure:
I'm completely stuck on this.. Feels like I've tried all different approaches, but I think i dont fully understand cloud functions and javascript, so i would gladly recive feedback and help on how to make this work.
Cheers,

Finally.... i managed to update the document successfully. I put the commit outside another ".then()". Thought I tried that, but yay I guess :P
}).then(() => {
return batch.commit().then(function () {
console.log("Succesfully commited changes.");
return null;
});
The problem now is that it commits every loop. I think the most optimal here would be to batch update ALL users before committing?
And again, is there a more optimal way to do this, in terms of minimizing the operation and impact? I'm afraid I go too deep with for loops instead of directly navigating to the document, but haven't found an easier way to do that.
Any thoughts?

NodeJS and Mongoose Callback inside a For Loop

So here's the problem. I have a REST API that handles a booking creation, however, before saving the booking inside mongo it validates if there is a clash with another booking.
exports.create = function(req, res) {
var new_type = new Model(req.body);
var newBooking = new_type._doc;
//check if the new booking clashes with existing bookings
validateBooking.bookingClash(newBooking, function(clash){
if(clash == null) // no clashes, therefore save new booking
{
new_type.save(function(err, type) {
if (err)
{
res.send(err); //error saving
}
else{
res.json(type); //return saved new booking
}
});
}
else //clash with booking
{
//respond with "clashDate"
}
});
};
Here you have the validation function to check if there is a clash with bookings on the same day:
exports.bookingClash = function (booking, clash) {
//find the bookings for the same court on the same day
var courtId = (booking.courtId).toString();
Model.find({courtId: courtId, date: booking.date}, function(err, bookings) {
if(err == null && bookings == null)
{
//no bookings found so no clashes
clash(null);
}
else //bookings found
{
//for each booking found, check if the booking start hour falls between other booking hours
for(var i = 0; i<bookings.length ; i++)
{
//here is where I check if the new booking clashes with bookings that are already in the DB
{
//the new booking clashes
//return booking date of the clash
clash(clashDate); //return the clashDate in order to tell the front-end
return;
}
}
//if no clashes with bookings, return null
clash(null);
}
});
};
So, ALL of this works with one single new booking. However, now I want to be able to handle a recursive booking (booking that is made weekly). I have recreated the "create" function and call the validateBooking.bookingClash function inside a for loop.
Unfortunately, when I run this, it calls the bookingClash function perfectly, but when it reaches the line making the search in the database:
Model.find({courtId: courtId, date: booking.date}, function(err, bookings)
It does not wait for the callback and before handling the response "clash", makes i++ and continues.
How can I make it work and wait for the callback?
var array = req.body;
var clashes = [];
for(var i = 0; i<array.length;i++)
{
validateBooking.bookingClash(array[i], function(clash)
{
if(clash)
{
clashes.push(clash);
}
else{
console.log("no clash");
}
}
}

Seems like a basic async call problem, for loops do not wait for callbacks to be called.
You could use async 'series' function for exmaple instead of the for loop. This way each find will get called after the previous one.
Mongoose also has a promise based syntax which can help you : http://mongoosejs.com/docs/promises.html

You Can use async eachSeries
async.eachSeries(users, function iterator(user, callback) {
if(something) {
//thing you want to do
callback();
} else {
callback();
}
}

Since you are using callback functions there are two ways you could try to solve this:
1) use some external library that allows you to perform an asynchronous map operation and run all the checks for each clash. Once they are done check the combined results for a clash and proceed accordingly
I would suggest using the async library
your code would look something like:
async.map(array,(entry,callback) => validateBooking.bookingClash(entry,callback),(error,mappingResults)=>{...})
2) you could try to change this function to a recursive one
`function recursiveValidation(arrayToCheck,mainCallback){
if(arrayToCheck.length === 0) {
return cb(null} // end of array without errors
}
validateBooking.bookingClash(_.head(arrayToCheck), function(clash)
{
if(clash)
{
return mainCallback(clash);
}
return recursiveValidation(_.tail(arrayToCheck),mainCallback);
}
}`
The above code is just a mockup but it should show the point.
The _ is lodash

No need to changing anything in your code except the declaration use let instead of var and your loop should work.
var array = req.body;
var clashes = [];
`
for(**let** i = 0; i<array.length;i++)
{
validateBooking.bookingClash(array[i], function(clash)
{
if(clash)
{
clashes.push(clash);
}
else{
console.log("no clash");
}
}
}`
You have to understand the difference between let and var. Also why var cannot be used for running async code inside a loop.
Learn about let: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/let

I found the way to get this done after trying all of your answers.
What I had to do was this:
validateBooking.singleBooking(new_type._doc, newBookingClubId, function (clash) {
if (clash == null) // no clash
{
validatorArray.push(0);
if(validatorArray.length == array.length) //has received everything from mongo
{
console.log("Clashes: " + clashes.toString());
if(validatorArray.indexOf(1) > -1) //contains a clash
{
var error = {
code: 409,
message: "409 Conflict",
clashes: clashes
};
errorsHandler.handleError(error, res);
}
This way, I created an array called "validatorArray" that was called every time I received something back from Mongo.
This way I could easily compare the length of the array of bookings and the validatorArray length. When they were equal, it meant that it had received everything back from mongo and could send back the response.
Thanks for the help!

Proper way to findOne document in Template Event?

I am trying to findOne document in my Template.admin.events code. I have a form and onClick I want to verify if the ID of the ObjectID entered is an existing document in my collection and fetch that result to show it on the template.
My event code on the client :
Template.admin.events({
'click #btnAjouterObjet'(event) {
let objetIdInput = $('#object_id').val().toString();
Meteor.subscribe('objetsFindOne', objetIdInput, {
onReady: function () {
let obj = Objets.findOne();
if (obj) {
console.log("found");
console.log(obj);
objetsArr.push(objetIdInput);
}
else {
console.log("not found");
console.log(obj);
}
}
});
}
});
In my Objets api :
Meteor.publish('objetsFindOne', function objetsFindOne(param_id){
return Objets.find({_id : param_id});
})
I have verified and my objetIdInput always change on click when a different Id is entered but the subscribe always returns the first id entered. I also added the onReady because otherwise it returned undefined.
I am new to Meteor and I have also tried to subscribe to all the collection and doing the find on the client but I don't think it is the best idea as my collection has about 22000 documents.

Just to elaborate a little bit on the first answer, as to how to change this pattern:
(1) you should place your Meteor.subscribe() call in your Template.admin.onCreated() function.
(2) the subscription reads from a reactive value, for example, new ReactiveVar().
(3) now, anytime the reactive value changes, the subscription will re-run. So, in your template event, you set the reactive value to the id and let the subscription handle the rest.
Discover Meteor and other resources should be helpful on any details.

You are going about this all wrong. I suggest you take a look at Template-Level Subscriptions

I opted for the use of a method :
Client side:
'click #btnAjouterObjet'(event) {
let objetIdInput = $('#object_id').val().toString();
let result = Meteor.call('findObj', objetIdInput, function (error, result) {
if (error) {
console.log(error.reason);
return;
}
console.log(result);
});
}
On the server side :
Meteor.methods({
findObj: function (param_id) {
console.log(Objets.find({ _id: param_id }).fetch());
return Objets.find({ _id: param_id }).fetch();
},
});

Meteor Users collection and Deps.autorun problems

I'm still struggling to understand how to access Meteor.users as a foreign key from another collection query. I understand that only the current user is published by default so I have a publication on the server as
Meteor.publish('itemOwner', function(userId) {
check(userId, String);
var user = Meteor.users.find({id: userId});
return user;
// return Meteor.users.find({id: userId}, {
// fields: {'profile': 1} });
});
I then have a Deps.autorun on the client..
Deps.autorun(function () {
var itemOwnerId = Session.get("itemOwnerID");
if (itemOwnerId) {
debugger
var ID = Session.get("itemOwnerID");
Meteor.subscribe('itemOwner', Session.get("itemOwnerID"));
}
});
I set the session ID on a modal form load, and display it in the template by calling the ownerProfile helper (or try to)
Template.showQuoteModalInner.helpers({
getQuote: function () {
// Get the quote ID from the session var
var quote = Session.get("quoteID");
if(quote) {
debugger;
var ID = quote.user._id;
Session.set("itemOwnerID", quote.user._id);
return quote;
}
},
ownerProfile: function() {
debugger;
var quote = Session.get("quoteID");
if(quote) {
var ID = quote.user._id;
var theUser = Meteor.users.find({_id: quote.user._id});
return theUser;
};
}
});
Now, I can trace the user ID at each stage and see it getting correctly passed to the autorun and the helpers. If I stop the program at the debugger in the ownerProfile helper and in the console put in Meteor.user.fetch({_id: "the id here"}).fetch() I get the correct user back.. but, in the handler itself the Meteor.users.find returns null??? What am I missing?

Two possibilities I noticed.
First, you are missing an underscore in the find in your publish function.
.find({id: userId}) should be .find({_id: userId}).
But this probably isn't the issue if you are seeing the user (other than the logged in user) in the console.
Second, if you are not seeing the user from your Template.showQuoteModalInner.ownerProfile helper, it is probably because you are returning a find() instead of a findOne().
find() returns a cursor whereas findOne() returns the record. Try findOne() if you want to display that single user's attributes.