Develop Reference

Mongoose on node, collection.find() always executes last no matter where i put the code

I'm new to all of this web development programming. I'm currently learning for 1 or 2 months now. and trying to create a local database system using node and mongodb. I'm having trouble understanding the sequence of mongoose collection.find(). I want to run the collection.find() to get some data then pass it to another function but it seems the collection.find() always executes last no matter where i put the code.
My code is something like this:
function getInitial() {
ticket.find(function (err, tickets) {
totalTickets = tickets.length;
console.log("Done verifying tickets");
});
gentInitial();
function lastFunction() {
console.log("this should be the last to execute");
}
lastFunction();
The output of this code is:
"this should be the last to execute"
"Done verifying tickets"
Desired output is:
"Done verifying tickets"
"this should be the last to execute"

Find runs async, so the callback in getInitial() gets executed after everything else in the OP code. Fixing the code to account for the fact that the find runs asynchronously...
async function getInitial() {
let tickets = await ticket.find();
totalTickets = tickets.length;
console.log("Done verifying tickets");
}
function lastFunction() {
console.log("this should be the last to execute");
}
async function test() {
await gentInitial();
lastFunction();
}
test();

like you have discovered, callbacks are not synchronous. (ticket.find takes time to return data)
if you want to execute something after find has returned data then execute it from inside the callback.
function getInitial() {
ticket.find(function (err, tickets) {
totalTickets = tickets.length;
console.log("Done verifying tickets");
// execute the rest of the code here
lastFunction();
});
}
function lastFunction() {
console.log("this should be the last to execute");
}
getInitial();
Please note that this way of coding from callbacks to callbacks leads the the callback hell : http://callbackhell.com/
the only way to have clean code and avoid thinking in callbacks too much is my using promises and using the await syntax.
example :
// now it works sequentially without callbacks
const data = await Character.find();
console.log(data);

This is actually straight forward. You're using callback which run syncronously. While lastfunction() directly calls console.log().
Mongoose call goes through a few step before it can actually return the result of the query you run and because nodeJS is a single threaded language, it'll move on to the next code (lastfunction in your case) while keeping the syncronous call in memory. Then gives you the result once the data have been fetched.
You can use asyn...await to make Mongoose give you the result before moving on to the next code.
Your code should be like this:
const getInitial = async() => {
try{
const tickets = await ticket.find({});
const totalTickets = tickets.length;
console.log("Done verifying tickets");
} catch(e){
console.log(e)
}
}
getInitial();
function lastFunction() {
console.log("this should be the last to execute");
}
lastFunction();

It is an asynchronous function because it will take time to query database and report it back to you.
Also try leaning about event loop in node.js.
There is plenty sources online.
Img for reference:-

Is there any way to break out of a function by running another function in JS

I am working on a simple project and I would like to create a simple helper function that checks for a error in a callback. If there is a error then it should break the whole function that called it. Code example:
//Makes call to database and tries to insert element
db.collection("data").insertOne(
{
key: 'some-data'
}, (error, result) => {
//Return error if something goes wrong - else error is empty
checkError(error, "Unable to load database");
console.log("Succes item added")
}
);
Note: Yes this is node.js but this whole principle could be repeated in js with other callbacks - very simple repeatable error principle.
So in the insertOne function the first argument is some data I am adding to the database. The second argument is the callback function that is called after this async operation is finished. It returns a error which I could just handle by adding this if statement to the callback:
if (error) {
console.error(error);
return;
}
Buuut thats disrespecting the dry principle (bc I write the exact same if statement everywhere with no syntax being changed except the message) and is also distracting when reading the callback function. Now my issue is in the function checkError() even tho I can just print the error with the message or throw the error, I dont actually have a way to break the original callback so that it doesnt cause any more havoc in my database. I will go on to promisify this callback which is a solution. BUT I want to know if there is a way to this in the way I presented it here. Note: I dont want to use the try catch block bc thats replacing a if statement with another two blocks.
My checkError function:
const checkError = function (error, msg = "Something went wrong") {
if (error) console.error(`${msg}: error`);
//Break original block somehow ¯\_(ツ)_/¯
};
If I were to compress my question it would be: how to break a function with another function. Is there any way to achieve this?

I don't think this is possible. But you could achieve something similar with this:
function checkError (error, msg = "Something went wrong") {
if (!error) return false;
console.error(`${msg}: error`);
return true;
};
db.collection("data").insertOne(
{
key: 'some-data'
}, (error, result) => {
//Return error if something goes wrong - else error is empty
if (checkError(error, "Unable to load database")) return;
console.log("Succes item added")
}
);

Things become easier when you use promises.
Often asynchronous APIs provide a promise interface, and this is also the case for mongodb/mongoose, where you can chain a .exec() call to execute the database query and get a promise in return. This gives you access to the power of JavaScript's async/await syntax. So you can then do like this:
async function main() {
// Connect to database
// ...
// Other db transactions
// ...
let result = await db.collection("data").insertOne({ key: 'some-data'}).exec();
console.log("Item added successfully");
// Any other database actions can follow here using the same pattern
// ...
}
main().catch(err => {
console.log(err);
});
The idea here is that await will throw an exception if the promise returned by .exec() eventually rejects. You can either put a standard try...catch construct around it to deal with that error, or you can just let it happen. In the latter case the promise returned by the wrapping async function will reject. So you can deal with the error at a higher level (like done above).
This way of working also removes the need for numerous nested callbacks. Often you can keep the nesting to just one of two levels by using promises.

What is the meaning of using done as a parameter in a javascript function?

i am learning Javascript, and it seems that the done as a parameter in a function is a difficult concept to understand.
I want to know why it does behaves like that (done as a parameter (completed process signal i guess), and if there is some good book or resource Online to study further this concept.
Example, i am following along with a Tutorial and it uses done as a parameter, the thing is that when i run the code on node via gulp (gulpfile.js) the process never stops when using done, if i choose to skip done in the code it runs smoothly. I am trying to track down the problem, and i know that the problem is the done as a parameter, ( it has been checked by me multiple times).
gulp.task('clean-styles', function(done) {
var files = config.temp + '**/*.css';
clean(files, done);
});
function clean(path, done) {
log('Cleaning: ' + $.util.colors.blue(path));
del(path, done).then(function(path) {
console.log("path=",util.inspect(path,false,null))
console.log('Deleted Files\/Folders:\n', path.join('\n'));
console.log('Finishing clean')
});
}
node version: 0.12.4
npm version: 2.10.1
gulp version: 3.9.0
Thanks a lot for any help, it will be really appreciated.
Salutations.

can only explain the concept. what you are trying to achieve is not clear enough.
done is just a non-official standard name for a function (a.k.a callback) that informs the calling function (parent in stacktrace) that a task is completed.
recall that javascript is asynchronous and functions can be passed around as variables.
now, imagine a function startPrinting that has to call printText1, printText2 and printText3 and then output message that process is completed. We have:
function startPrinting() {
printText1();
printText2();
printText3();
console.log("completed");
}
function printText1() {
$.get('http://ps-web.cloudapp.net/proxy.php?url=http://pastebin.com/raw.php?i=3uPCavLN', function(response){
console.log(response)
});
}
function printText2() {
$.get('http://ps-web.cloudapp.net/proxy.php?url=http://pastebin.com/raw.php?i=jZjqKgNN', function(response){
console.log(response)
});
}
function printText3() {
$.get('http://ps-web.cloudapp.net/proxy.php?url=http://pastebin.com/raw.php?i=SreCbunb', function(response){
console.log(response)
});
}
here, there is no assurance that completed will ALWAYS be printed after all three functions have been executed. since they execute asynchronously.
in order to sort this, javascript ninjas will introduce a done function so that startPrinting will only print completed when all three functions have been executed. Notice how a function is passed to printText1 ... 2 below:
function startPrinting() {
/* START OF DONE ROUTINE */
var count = 0;
var printCompleted = function() {
count+=1;
if(count == 3)
console.log("completed");
}
/* END */
printText1(printCompleted);
printText2(printCompleted);
printText3(printCompleted);
}
function printText1(done) {
$.get('http://ps-web.cloudapp.net/proxy.php?url=http://pastebin.com/raw.php?i=3uPCavLN', function(response){
console.log(response)
done();
});
}
function printText2(done) {
$.get('http://ps-web.cloudapp.net/proxy.php?url=http://pastebin.com/raw.php?i=jZjqKgNN', function(response){
console.log(response)
done();
});
}
function printText3(done) {
$.get('http://ps-web.cloudapp.net/proxy.php?url=http://pastebin.com/raw.php?i=SreCbunb', function(response){
console.log(response)
done();
});
}
I hope you are able to apply this principle to better understanding your context.

Functions are first class objects in JavaScript. You can pass them around like any other value. Once they have been passed to another function as an argument, then you can call them using the argument name (or call another function and pass them as an argument to that, or assign them properties, or convert them to strings, or whatever else you'd like to do with them).
function this_sets_the_body() {
document.body.innerHTML = "Hello, world";
}
function this_calls_a_callback(im_a_callback) {
im_a_callback();
}
this_calls_a_callback(this_sets_the_body);
In your code, you've written a function using an anonymous function expression:
function(done) {
// ...
}
… and you've told it to expect to be called with an argument which you are calling done.
Whatever value is passed to it, you are ignoring (your function doesn't mention done after the argument name).
The library you are using (presumably) is passing a function in there and expects you to call it once your function as done whatever it is that it is going to do. This lets it wait until anything asynchronous that you are doing is finished.
So call done() when your code is done.

It looks like your example is fully messed up with respect to callbacks. In some places in your example done is used as a callback -- a function given from outside to be called when everything is finished in an asynchronous process and signal the end of the operation. In other cases it looks to be used as an argument provided by the execution method itself. In yet another you use it in a promise. Anyway, as I am not familiar with gulp I can only guess, but I hope the following example would work for you to explain the concepts of callback and partially promise. I would, however, recommend to avoid situations of missing callbacks and promises in the same code as it leads to confusion.
gulp.task('clean-styles', function(done) {
console.log(1);
/* we are in the callback of gulp.task: we give the
* latter this anonymous function to call when the
* setup is ready and it gives us function done to
* call when we are done and signal the engine any errors
*/
var files = config.temp + '**/*.css';
/* this defines the action to take when files are actually deleted */
var callback = function(err, message) {
console.log(6);
console.log(message); // expect: looks good
// this is provided apparently by gulp and calling it signals the engine that everything is completed
done(err);
};
/* we call this function, but some bits (like deletion
* run asynchronously. The function will return quickly, but
* callback (function) will only be called when files are deleted */
clean(files, callback);
/* the execution of gulp.task callback is finished,
* but files are not yet deleted */
console.log(4);
});
/* done here is not the same done as above, it is actually
* the function we supply into the call above, i.e. `callback` */
function clean(path, done) {
/* the cleanup is starting */
console.log(2);
/* del is scheduled. it returns a `promise` and if
* we call `then`, then the given anonymous function
* will be executed when files are deleted. This is
* where we call the provided function `done` to
* signal that the job is complete and execute some action */
del(path).then(function() {
/* files are deleted and this callback is called */
console.log(5);
/* we let the outer caller know by calling `done` which
* was given to us from outside */
done(null, "looks good"); // null means no error
}).catch(function(err) {
done(err, "looks bad"); // err is given back
});
/* the clean method is through, but files not yet deleted */
console.log(3);
}

Node.js async, but only handle first positive/defined result

What is the best way to create parallel asynchronous HTTP requests and take the first result that comes back positive? I am familiar with the async library for JavaScript and would happy to use that but am not sure if it has exactly what I want.
Background - I have a Redis store that serves as state for a server. There is an API we can call to get some data that takes much longer than reaching the Redis store.
In most cases the data will already be in the Redis store, but in some cases it won't be there yet and we need to retrieve it from the API.
The simple thing to do would be to query Redis, and if the value is not in Redis then go to the API afterwards. However, we'll needlessly lose 20-50ms if the data is not yet in our Redis cache and we have to go to the API after failing to find the data with Redis. Since this particular API server is not under great load, it won't really hurt to go to the API simultaneously/in parallel, even if we don't absolutely need the returned value.
//pseudocode below
async.minimum([
function apiRequest(cb){
request(opts,function(err,response,body){
cb(err,body.result.hit);
}
},
function redisRequest(cb){
client.get("some_key", function(err, reply) {
cb(err,reply.result.hit);
});
}],
function minimumCompleted(err,result){
// this mimimumCompleted final callback function will be only fired once,
// and would be fired by one of the above functions -
// whichever one *first* returned a defined value for result.hit
});
is there a way to get what I am looking for with the async library or perhaps promises, or should I implement something myself?

Use Promise.any([ap, bp]).
The following is a possible way to do it without promises. It is untested but should meet the requirements.
To meet requirement of returning the first success and not just the first completion, I keep a count of the number of completions expected so that if an error occurs it can be ignored it unless it is the last error.
function asyncMinimum(a, cb) {
var triggered = false;
var completions = a.length;
function callback(err, data) {
completions--;
if (err && completions !== 0) return;
if (triggered) return;
triggered = true;
return cb(err, data);
}
a.map(function (f) { return f(callback); });
}
asyncMinimum([
function apiRequest(cb){
request(opts,function(err,response,body){
cb(err,body.result.hit);
}
},
function redisRequest(cb){
client.get("some_key", function(err, reply) {
cb(err,reply.result.hit);
});
}],
function minimumCompleted(err,result){
// this mimimumCompleted final callback function will be only fired once,
// and would be fired by one of the above functions -
// whichever one had a value for body.result.hit that was defined
});

The async.js library (and even promises) keep track of the number of asynchronous operations pending by using a counter. You can see a simple implementation of the idea in an answer to this related question: Coordinating parallel execution in node.js
We can use the same concept to implement the minimum function you want. Only, instead of waiting for the counter to count all responses before triggering a final callback, we deliberately trigger the final callback on the first response and ignore all other responses:
// IMHO, "first" is a better name than "minimum":
function first (async_functions, callback) {
var called_back = false;
var cb = function () {
if (!called_back) {
called_back = true; // block all other responses
callback.apply(null,arguments)
}
}
for (var i=0;i<async_functions.length;i++) {
async_functions[i](cb);
}
}
Using it would be as simple as:
first([apiRequest,redisRequest],function(err,result){
// ...
});

Here's an approach using promises. It takes a little extra custom code because of the non-standard result you're looking for. You aren't just looking for the first one to not return an error, but you're looking for the first one that has a specific type of result so that takes a custom result checker function. And, if none get a result, then we need to communicate that back to the caller by rejecting the promise too. Here's the code:
function firstHit() {
return new Promise(function(resolve, reject) {
var missCntr = 0, missQty = 2;
function checkResult(err, val) {
if (err || !val) {
// see if all requests failed
++missCntr;
if (missCntr === missQty) {
reject();
}
} else {
resolve(val);
}
}
request(opts,function(err, response, body){
checkResult(err, body.result.hit);
}
client.get("some_key", function(err, reply) {
checkResult(err, reply.result.hit);
});
});
}
firstHit().then(function(hit) {
// one of them succeeded here
}, function() {
// neither succeeded here
});
The first promise to call resolve() will trigger the .then() handler. If both fail to get a hit, then it will reject the promise.

How do async tests work in intern?

How do asynchronous tests work in Intern testing framework? I have tried to get them run exactly as in the example, but the async test passes immediately without waiting for the callback to be run.
it('should connect in 5 seconds', function () {
var dfd = this.async(5000);
conn.connect(credentials, dfd.callback(function(result) {
expect(result).to.... something
}));
}
The test passes immediately. What am I doing wrong?

dfd.callback doesn’t execute anything until it itself is executed. Keep in mind that it is designed for promise callbacks (i.e. the function passed to promise.then), not Node.js-style callbacks where the argument might be an error (i.e. function (error, result) {}). It will not check to see if an error is passed as an argument.
Without knowing what conn is, but seeing how you are passing dfd.callback as an argument to something that is not a promise, my suspicion is you are trying to use a Node.js-style callback and the call is erroring immediately. We may provide a convenience wrapper for these types of callbacks in the future to convert them to a promise interface, but until then, you probably just need to do something like this:
it('should connect in 5 seconds', function () {
var dfd = this.async(5000);
conn.connect(credentials, dfd.callback(function(error, result) {
if (error) {
throw error;
}
expect(result).to.... something
}));
});
Otherwise, without knowing what conn is and seeing what your actual assertion is, it’s too hard to say what the issue is here. So long as nothing inside the callback throws an error, the test will be considered successful.
Edit: So based on your comments above it sounds like your callback is an event listener called multiple times with different information. In this case, what you could do is something like this:
it('should connect in 5 seconds', function () {
var dfd = this.async(5000);
conn.connect(credentials, dfd.rejectOnError(function (result) {
if (result !== 'what I want') {
return;
}
expect(result).to.... something
// all other tests…
// nothing threw an error, so it is a successful test
dfd.resolve();
}));
});
dfd.rejectOnError works just like dfd.callback except it does not automatically resolve the promise; you do that yourself at the end.

Your structure is okay. dfd sets a timeout of 5 seconds for the test to succeed, then immediately tries conn.connect(), which is not part of the intern framework. If you are trying a simple XHR request, try the getUrlCallback function instead.
They have a pretty cool list of tests at this url: https://github.com/theintern/intern/wiki/Writing-Tests . Look for the two async examples.