Develop Reference

JavaScript Why is some code getting executed before the rest?

I've mostly learned coding with OOPs like Java.
I have a personal project where I want to import a bunch of plaintext into a mongodb. I thought I'd try to expand my horizons and do this with using node.js powered JavaScript.
I got the code working fine but I'm trying to figure out why it is executing the way it is.
The output from the console is:
1. done reading file
2. closing db
3. record inserted (n times)
var fs = require('fs'),
readline = require('readline'),
instream = fs.createReadStream(config.file),
outstream = new (require('stream'))(),
rl = readline.createInterface(instream, outstream);
rl.on('line', function (line) {
var split = line.split(" ");
_user = "#" + split[0];
_text = "'" + split[1] + "'";
_addedBy = config._addedBy;
_dateAdded = new Date().toISOString();
quoteObj = { user : _user , text : _text , addedby : _addedBy, dateadded : _dateAdded};
db.collection("quotes").insertOne(quoteObj, function(err, res) {
if (err) throw err;
console.log("record inserted.");
});
});
rl.on('close', function (line) {
console.log('done reading file.');
console.log('closing db.')
db.close();
});
(full code is here: https://github.com/HansHovanitz/Import-Stuff/blob/master/importStuff.js)
When I run it I get the message 'done reading file' and 'closing db' and then all of the 'record inserted' messages. Why is that happening? Is it because of the delay in inserting a record in the db? The fact that I see 'closing db' first makes me think that the db would be getting closed and then how are the records being inserted still?
Just curious to know why the program is executing in this order for my own peace of mind. Thanks for any insight!

In short, it's because of asynchronous nature of I/O operations in the used functions - which is quite common for Node.js.
Here's what happens. First, the script reads all the lines of the file, and for each line initiates db.insertOne() operation, supplying a callback for each of them. Note that the callback will be called when the corresponding operation is finished, not in the middle of this process.
Eventually the script reaches the end of the input file, logs two messages, then invokes db.close() line. Note that even though 'insert' callbacks (that log 'inserted' message) are not called yet, the database interface has already received all the 'insert' commands.
Now the tricky part: whether or not DB interface succeeds to store all the DB records (in other words, whether or not it'll wait until all the insert operations are completed before closing the connection) is up both to DB interface and its speed. If write op is fast enough (faster than reading the file line), you'll probably end up with all the records been inserted; if not, you can miss some of them. That's why it's a safest bet to close the connection to database not in the file close (when the reading is complete), but in insert callbacks (when the writing is complete):
let linesCount = 0;
let eofReached = false;
rl.on('line', function (line) {
++linesCount;
// parsing skipped for brevity
db.collection("quotes").insertOne(quoteObj, function(err, res) {
--linesCount;
if (linesCount === 0 && eofReached) {
db.close();
console.log('database close');
}
// the rest skipped
});
});
rl.on('close', function() {
console.log('reading complete');
eofReached = true;
});
This question describes the similar problem - and several different approaches to solve it.

Welcome to the world of asynchronicity. Inserting into the DB happens asynchronously. This means that the rest of your (synchronous) code will execute completely before this task is complete. Consider the simplest asynchronous JS function setTimeout. It takes two arguments, a function and a time (in ms) after which to execute the function. In the example below "hello!" will log before "set timeout executed" is logged, even though the time is set to 0. Crazy right? That's because setTimeout is asynchronous.
This is one of the fundamental concepts of JS and it's going to come up all the time, so watch out!
setTimeout(() => {
console.log("set timeout executed")
}, 0)
console.log("hello!")

When you call db.collection("quotes").insertOne you're actually creating an asynchronous request to the database, a good way to determine if a code will be asynchronous or not is if one (or more) of its parameters is a callback.
So the order you're running it is actually expected:
You instantiate rl
You bind your event handlers to rl
Your stream starts processing & calling your 'line' handler
Your 'line' handler opens asynchronous requests
Your stream ends and rl closes
...
4.5. Your asynchronous requests return and execute their callbacks
I labelled the callback execution as 4.5 because technically your requests can return at anytime after step 4.
I hope this is a useful explanation, most modern javascript relies heavily on asynchronous events and it can be a little tricky to figure out how to work with them!

You're on the right track. The key is that the database calls are asychronous. As the file is being read, it starts a bunch of async calls to the database. Since they are asynchronous, the program doesn't wait for them to complete at the time they are called. The file then closes. As the async calls complete, your callbacks runs and the console.logs execute.

Your code reads lines and immediately after that makes a call to the db - both asynchronous processes. When the last line is read the last request to the db is made and it takes some time for this request to be processed and the callback of the insertOne to be executed. Meanwhile the r1 has done it's job and triggers the close event.

Azure DocumentDB Stored Procedure in Node.js excute existing

I have been going around in circles with this. I'm trying to execute an existing stored procedure using the node js documentdb library.
var sproc = self.client.queryStoredProcedures(collection._self, "select * from root r WHERE r.id = 'helloWorld'");
self.client.executeStoredProcedure(sproc._self, function (err, res) {
if(err){
console.log(err);
}else{
console.log(res);`
}
});
Not entirely sure queryStoredProcedures (Seems to be no async version of this) is the correct way of retrieving the uri for the store procedure, I haven't managed to get this to work. I'm also trying to avoid too many round trips to the database, but from what I gather I either hard code the store procedure's uri or have to make at least two requests just to execute the stored procedure.

queryStoredProcedures (along with all query and read functions) return a QueryIterator rather than an actual result. The methods you call on the returned QueryIterator are async. So, following the approach of your example (minus error handling), you would do this:
var queryIterator = self.client.queryStoredProcedures(collection._self, "select * from root r WHERE r.id = 'helloWorld'");
queryIterator.toArray(function(err, result) {
var sproc = result[0];
self.client.executeStoredProcedure(sproc._self, function (err, res) {
console.log(res);`
});
});
However, since the introduction of id-based routing, you can short hand the above like this:
var sprocLink = "dbs/myDatabase/colls/myCollection/sprocs/helloWorld";
self.client.executeStoredProcedure(sprocLink, function (err, res) {
console.log(res);`
});

How do I return the results of .query in a class

I'm writing an application in NodeJS using coffee script and I need to store the results of a sql query made by a class into a variable that that class can return.
I have something like
class dataBase
mySQL = require('mysql');
connection = null;
queryResults = null;
constructor:(host,user,password,database)->
connection = mySQL.createConnection({
host : "#{host}",
user : "#{user}",
password : "#{password}",
database : "#{database}",
});
connection.connect;
then later in the class I have a function called query that queries the database and returns the results.
query:(input) ->
connection.query("#{input}",(err,result)->
queryResults = result;
);
return queryResults;
The problem is because the function in connection.query always runs asynchronously so return queryResults always returns null.
Is there anyway I can fix this so that query() will return the value of the result if I do something outside of the class like:
myDatabase = new dataBase("fee","fi","foo","fum");
users = myDatabase.query("SELECT * FROM users");

In the "asynchronous" programming model (AP), one provides a callback in which the result of the query is used. Something like:
myDatabase = new dataBase("fee","fi","foo","fum");
myDatabase.query("SELECT * FROM users",function(err,result){
// check for error and do something with result
});
The whole idea behind AP is that, rather than waiting, things can and do occur outside of your normal program flow--for example, as data become available--and so, you must use whatever result they produce when it becomes available.
This is why AP callbacks do not return values. There is no context into which they can return.
Think about it like sending a certified letter through the Post Office. You hand off your letter to the postal person and rather than wait until the letter is delivered and the receipt is available, you go do other things, leaving the instruction that once the receipt is in-hand, it should be delivered to your house or you should be called to pick it up.

How do I return the results of a query using Sequelize and Javascript?

I'm new at javascript and I've hit a wall hard here. I don't even think this is a Sequelize question and probably more so about javascript behavior.
I have this code:
sequelize.query(query).success( function(row){
console.log(row);
}
)
The var row returns the value(s) that I want, but I have no idea how to access them other than printing to the console. I've tried returning the value, but it isn't returned to where I expect it and I'm not sure where it goes. I want my row, but I don't know how to obtain it :(

Using Javascript on the server side like that requires that you use callbacks. You cannot "return" them like you want, you can however write a function to perform actions on the results.
sequelize.query(query).success(function(row) {
// Here is where you do your stuff on row
// End the process
process.exit();
}
A more practical example, in an express route handler:
// Create a session
app.post("/login", function(req, res) {
var username = req.body.username,
password = req.body.password;
// Obviously, do not inject this directly into the query in the real
// world ---- VERY BAD.
return sequelize
.query("SELECT * FROM users WHERE username = '" + username + "'")
.success(function(row) {
// Also - never store passwords in plain text
if (row.password === password) {
req.session.user = row;
return res.json({success: true});
}
else {
return res.json({success: false, incorrect: true});
}
});
});
Ignore injection and plain text password example - for brevity.
Functions act as "closures" by storing references to any variable in the scope the function is defined in. In my above example, the correct res value is stored for reference per request by the callback I've supplied to sequelize. The direct benefit of this is that more requests can be handled while the query is running and once it's finished more code will be executed. If this wasn't the case, then your process (assuming Node.js) would wait for that one query to finish block all other requests. This is not desired. The callback style is such that your code can do what it needs and move on, waiting for important or processer heavy pieces to finish up and call a function once complete.
EDIT
The API for handling callbacks has changed since answering this question. Sequelize now returns a Promise from .query so changing .success to .then should be all you need to do.

According to the changelog
Backwards compatibility changes:
Events support have been removed so using .on('success') or .success()
is no longer supported. Try using .then() instead.
According this Raw queries documentation you will use something like this now:
sequelize.query("SELECT * FROM `users`", { type: sequelize.QueryTypes.SELECT})
.then(function(users) {
console.log(users);
});

node.js mysql failing to insert

I'm messing around with node.js, using the Faker, underscore, mysql and randy libraries to add some test data to a web app.
So far so good-- but in one of my loops, the mysql call fails every time, but it also fails to generate any errors. I'm a bit stumped:
var sql = 'SELECT id FROM mytable WHERE 1';
client.query(sql, function(err, rows, fields){
// loop through results
_.each(rows, function (v, i){
// get the insert id
var id = v.id;
console.log (id); // prints valid ID
// generate other data
var stuff_count = randy.randInt(0,12);
_(stuff_count).times(function(n){
var sql = 'INSERT INTO other_table (linked_id, item_id) VALUES ('+id+','+randy.randInt(1,50)+')';
console.log(sql); // prints sql
client.query(sql, function(err, rows, fields) {
console.log(sql); // prints nothing
if (err){
console.log(sql); // prints nothing
throw err; // prints nothing
}
});
});
});
The looping logic is working fine and it gets all the way to where the sql should execute the INSERT against the other_table-- but tailing the mysql log shows nothing hitting the DB, and none of the debug statements inside the client.query block are printing anything, on success or failure.

assuming you are just running this standalone, your code probably finishes before it has a chance of doing the inserts.
place a setTimeout(..., 10000) after the _(stuff_count).times(...)
By using _().times(), you queue all the inserts, but are not waiting for completion. you should use a control flow library to ensure you wait for all inserts to complete.