Develop Reference

This code doesn't seem to fire in order?

My problem is that the code does not seem to be running in order, as seen below.
This code is for my discord.js bot that I am creating.
var Discord = require("discord.js");
var bot = new Discord.Client();
var yt = require("C:/Users/username/Documents/Coding/Discord/youtubetest.js");
var youtubetest = new yt();
var fs = require('fs');
var youtubedl = require('youtube-dl');
var prefix = "!";
var vidid;
var commands = {
play: {
name: "!play ",
fnc: "Gets a Youtube video matching given tags.",
process: function(msg, query) {
youtubetest.respond(query, msg);
var vidid = youtubetest.vidid;
console.log(typeof(vidid) + " + " + vidid);
console.log("3");
}
}
};
bot.on('ready', () => {
console.log('I am ready!');
});
bot.on("message", msg => {
if(!msg.content.startsWith(prefix) || msg.author.bot || (msg.author.id === bot.user.id)) return;
var cmdraw = msg.content.split(" ")[0].substring(1).toLowerCase();
var query = msg.content.split("!")[1];
var cmd = commands[cmdraw];
if (cmd) {
var res = cmd.process(msg, query, bot);
if (res) {
msg.channel.sendMessage(res);
}
} else {
let msgs = [];
msgs.push(msg.content + " is not a valid command.");
msgs.push(" ");
msgs.push("Available commands:");
msgs.push(" ");
msg.channel.sendMessage(msgs);
msg.channel.sendMessage(commands.help.process(msg));
}
});
bot.on('error', e => { console.error(e); });
bot.login("mytoken");
The youtubetest.js file:
var youtube_node = require('youtube-node');
var ConfigFile = require("C:/Users/username/Documents/Coding/Discord/json_config.json");
var mybot = require("C:/Users/username/Documents/Coding/Discord/mybot.js");
function myyt () {
this.youtube = new youtube_node();
this.youtube.setKey(ConfigFile.youtube_api_key);
this.vidid = "";
}
myyt.prototype.respond = function(query, msg) {
this.youtube.search(query, 1, function(error, result) {
if (error) {
msg.channel.sendMessage("There was an error finding requested video.");
} else {
vidid = 'http://www.youtube.com/watch?v=' + result.items[0].id.videoId;
myyt.vidid = vidid;
console.log("1");
}
});
console.log("2");
};
module.exports = myyt;
As the code shows, i have an object for the commands that the bot will be able to process, and I have a function to run said commands when a message is received.
Throughout the code you can see that I have put three console.logs with 1, 2 and 3 showing in which order I expect the parts of the code to run. When the code is run and a query is found the output is this:
I am ready!
string +
2
3
1
This shows that the code is running in the wrong order that I expect it to.
All help is very highly appreciated :)
*Update! Thank you all very much to understand why it isn't working. I found a solution where in the main file at vidid = youtubetest.respond(query, msg) when it does that the variable is not assigned until the function is done so it goes onto the rest of my code without the variable. To fix I simply put an if statement checking if the variable if undefined and waiting until it is defined.*

Like is mentioned before, a lot of stuff in javascript runs in async, hence the callback handlers. The reason it runs in async, is to avoid the rest of your code being "blocked" by remote calls. To avoid ending up in callback hell, most of us Javascript developers are moving more and more over to Promises. So your code could then look more like this:
myyt.prototype.respond = function(query, msg) {
return new Promise(function(resolve, reject) {
this.youtube.search(query, 1, function(error, result) {
if (error) {
reject("There was an error finding requested video."); // passed down to the ".catch" statement below
} else {
vidid = 'http://www.youtube.com/watch?v=' + result.items[0].id.videoId;
myyt.vidid = vidid;
console.log("1");
resolve(2); // Resolve marks the promises as successfully completed, and passes along to the ".then" method
}
});
}).then(function(two) {
// video is now the same as myyt.vidid as above.
console.log(two);
}).catch(function(err) {
// err contains the error object from above
msg.channel.sendMessage(err);
})
};
This would naturally require a change in anything that uses this process, but creating your own prototypes seems.. odd.
This promise returns the vidid, so you'd then set vidid = youtubetest.response(query, msg);, and whenever that function gets called, you do:
vidid.then(function(id) {
// id is now the vidid.
});
Javascript runs async by design, and trying to hack your way around that leads you to dark places fast. As far as I can tell, you're also targetting nodeJS, which means that once you start running something synchronously, you'll kill off performance for other users, as everyone has to wait for that sync call to finish.
Some suggested reading:
http://callbackhell.com/
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise
https://stackoverflow.com/a/11233849/3646975
I'd also suggest looking up ES6 syntax, as it shortens your code and makes life a hellofalot easier (native promises were only introduced in ES6, which NodeJS 4 and above supports (more or less))

In javascript, please remember that any callback function you pass to some other function is called asynchronously. I.e. the calls to callback function may not happen "in order". "In order" in this case means the order they appear on the source file.
The callback function is simply called on certain event:
When there is data to be processed
on error
in your case for example when the youtube search results are ready,
'ready' event is received or 'message' is received.
etc.

Is it possible to stop a function from executing while waiting for an event listener?

Using the gmail.js library I wrote this function that receives a string back from an event listener that I want to inject into the body of my email before it is sent. How can I somehow get the function to halt execution while waiting for the Event listener to get the reply from the external script? I have a feeling it can be done with synchronous callbacks but I dont know those well enough to know how to. I'd appreciate any help on the matter.
gmail.observe.before('send_message', function(url, body, data, xhr) {
console.log('In the before send message method');
var event = new CustomEvent('Marketing', {
detail: {
body: xhr.xhrParams.body_params.body,
email: gmail.get.user_email()
}
});
window.addEventListener('message', function(event) {
if (event.data.type && (event.data.type == "FROM_PAGE")) {
console.log("received a reply");
console.log(event.data.text);
newstring = (event.data.candidate);
console.log(newstring);
console.log(event.data.token);
gotNewString = true;
}
});
window.dispatchEvent(event);
console.log('sent the message to extension.js');
function listen(data){
console.log("123123",JSON.stringify(data));
console.log(data);
}
newstring = "Why must you torture me so?";
var oldCmml = xhr.xhrParams.url.cmml;
var body_params = xhr.xhrParams.body_params;
if (newstring.length > oldCmml) {
xhr.xhrParams.url.cmml = newstring.length;
} else {
while (newstring.length < oldCmml) {
newstring += ' ';
}
xhr.xhrParams.url.cmml = newstring.length;
}
console.log(newstring);
body_params.body = newstring;
console.log("sent");
});

You can't stop the execution of a function inside a callback. You would have to change the code of the calling function, to wait for your function to be finished. This is normally done by supplying a callback function, that you than have to call, once you are done processing, for instance:
gmail.observe.before('send_message', function(url, body, data, xhr, done) {
// you can do processing here
// and have to call done, to let the caller continue his work
done();
});
For listeners, this is obviously normally not supplied, but since gmail.js is open-source you could change it?

Maximum parallel pipes to fs [duplicate]

I am using the following node.js code to download documents from some url and save it in the disk.
I want to be informed about when the document is downloaded. i have not seen any callback with pipe.Or, Is there any 'end' event that can be captured on completion of download ?
request(some_url_doc).pipe(fs.createWriteStream('xyz.doc'));

Streams are EventEmitters so you can listen to certain events. As you said there is a finish event for request (previously end).
var stream = request(...).pipe(...);
stream.on('finish', function () { ... });
For more information about which events are available you can check the stream documentation page.

Based nodejs document, http://nodejs.org/api/stream.html#stream_event_finish,
it should handle writableStream's finish event.
var writable = getWriteable();
var readable = getReadable();
readable.pipe(writable);
writable.on('finish', function(){ ... });

Code snippet for piping content from web via http(s) to filesystem. As #starbeamrainbowlabs noticed event finish does job
var tmpFile = "/tmp/somefilename.doc";
var ws = fs.createWriteStream(tmpFile);
ws.on('finish', function() {
// pipe done here, do something with file
});
var client = url.slice(0, 5) === 'https' ? https : http;
client.get(url, function(response) {
return response.pipe(ws);
});

I found an a bit different solution of my problem regarding this context. Thought worth sharing.
Most of the example create readStreams from file. But in my case readStream has to be created from JSON string coming from a message pool.
var jsonStream = through2.obj(function(chunk, encoding, callback) {
this.push(JSON.stringify(chunk, null, 4) + '\n');
callback();
});
// message.value --> value/text to write in write.txt
jsonStream.write(JSON.parse(message.value));
var writeStream = sftp.createWriteStream("/path/to/write/write.txt");
//"close" event didn't work for me!
writeStream.on( 'close', function () {
console.log( "- done!" );
sftp.end();
}
);
//"finish" event didn't work for me either!
writeStream.on( 'close', function () {
console.log( "- done!"
sftp.end();
}
);
// finally this worked for me!
jsonStream.on('data', function(data) {
var toString = Object.prototype.toString.call(data);
console.log('type of data:', toString);
console.log( "- file transferred" );
});
jsonStream.pipe( writeStream );

Here's a solution that handles errors in requests and calls a callback after the file is written:
request(opts)
.on('error', function(err){ return callback(err)})
.pipe(fs.createWriteStream(filename))
.on('finish', function (err) {
return callback(err);
});

How to pass additional information to a callback of a 3rd party module in node.js

It is a simple node.js application. I'm using Browser.visit(url, callback) provided by Zombie module (http://zombie.labnotes.org), Browser.visit() will call the callback as (error, browser, status) in uncertain future with different context. And my callback need some extra information to finish its job.
So far I can make it "working" as:
function test(title, url) {
var browser = new Browser();
browser.visit(url, function (e, browser, status, thetitle)
{
// generate info using the title if browser visited url successfully
}
.bind(undefined, browser, undefined, title);
}
test("foo", fooUrl);
test("bar", barUrl);
But there is a problem, the bind() just bypassed the error and status parameters. It seems not the right way. How could I pass the additional info to this callback?
FYI: Just got confirmed, the API doc of Zombie.js is for 1.4.x, 2.0.0-Alpha API is not finalized yet. https://groups.google.com/forum/?hl=en#!topic/zombie-js/qOS0W_cMCgQ

It's a little difficult to understand your question, I'm not sure why you are using bind, but perhaps this is what you are trying to achieve?
Javascript
function Browser(url) {
this.url = url;
}
Browser.prototype.visit = function (url, fn) {
var self = this,
error = 'error',
status = 'status';
setTimeout(function () {
fn(error, self, status);
}, 1000);
};
function test(title, url) {
var browser = new Browser(url);
browser.visit(url, (function (theTitle) {
return function (theError, theBrowser, theStatus) {
// generate info using the title if browser visited url successfully
console.log('theError: ', theError);
console.log('theBrowser: ', theBrowser);
console.log('theStatus: ', theStatus);
console.log('theTitle: ', theTitle);
};
}(title)));
}
test('hello', 'world');
test('goodbye', 'mum');
Output
theError: error
theBrowser: Browser {url: "world", visit: function}
theStatus: status
theTitle: hello
theError: error
theBrowser: Browser {url: "mum", visit: function}
theStatus: status
theTitle: goodbye
On jsFiddle

There is no need to use bind just to make the anonymous function passed to browser.visit access title. This would work:
function test(title, url) {
var browser = new Browser();
browser.visit(url, function (e, browser, status) {
console.log(title);
});
}
The value of title in console.log(title) is the same as the value of the title parameter of the test function.
Note that this works whether the callback passed to browser.visit is called synchronously or asynchronously.

Javascript rendering. How to write Javascript so it doesn't continue executing code before a function call has ended

Not sure if my question is subjective/objective but as a JavaScript newbie i'm encountering this problem quite a lot. So here I go.
I'm used to write C#, so my JavaScript structure looks like C#. And just that, that gives problems I think ;-)
Let's give a simple example where I met my problem again today:
MyLibrary.fn.InitAddEntityForm = function () {
$('a#btnAddEntity').click(function () {
//post data and receive object with guid and isPersisted boolean
var persistedObject = MyLibrary.fn.CheckAndSendAddEntityForm("name", "avatarurl.png");
console.log("test");
//check if persisted and go to next step
if (persistedObject.isPersisted) {
MyLibrary.fn.InitAddAnotherEntityForm(persistedObject.gdEntityId);
} else {
alert("Oops, something went wrong. Please call 911");
}
});
};
//////*****/////
//SOME FUNCTION THAT SENDS MY FORM AND RETURNS AN OBJECT WITH TRUE VALUE AND POSTED ENTITY ID
/////*****//////
MyLibrary.fn.CheckAndSendAddForm = function (txtName, ImageUrl) {
var postUrl = "/admin/add";
var persistedObject = new Object();
$.post(
postUrl,
{ Name: txtName, ImageUrl: txtImageUrl},
function (data) {
if (data.Status == 200) {
console.log("Post status:" + data.Message);
persistedObject.isPersisted = true;
persistedObject.gdEntityId = data.Data;
} else if (data.Status == 500) {
console.log("Failed to post entitiy");
} else {
console.log("Fault with Javascript");
}
}, "json"
);
return persistedObject;
};
Okay, thats it. Everything looks okay right? Browser says no.
I tried to debug it using firebug, looping over my code line by line, and that way the browser does what I want: Execute a new function to show the next panel in my wizard.
After placing a lot of Console.logs() in my code I figured out that this must be something about timing in JavaScript. In C# the code executes line by line, but apparently JavaScript doesn't.
By placing that Console.log("test") I noticed that "test" appeared in my console before "Post status: Success!".
So here's my question, how should I write my JavaScript code so I have control over the way the browser executes my code?
Should I really replace the code below to the end of my CheckAndSendAddEntityForm()?
//check if persisted and go to next step
if (persistedObject.isPersisted) {
MyLibrary.fn.InitAddAnotherEntityForm(persistedObject.gdEntityId);
} else {
alert("fout");
}
Is this how I have to write JavaScript: One big domino effect or am I just doing something wrong?

$.post is a shortcut for an AJAX call, AJAX is by definition asynchronous, which means it won't wait on a response before continuing processing. If you switch it to a regular AJAX() method, there is an async option you can set to false, which will make it behave as you are expecting.
Alternatively you can also define a function to execute on successful return of the AJAX request, in which you can call the next step in your process chain.

The AJAX call is asychronous; that means that the callback method exposes by $.post will be executed when the request completes, but your javascript will continue executing as soon as the invoke to $.post finishes. If you want to do something after the ajax call is done, you need to provide a callback method and do something else, ex:
MyLibrary.fn.CheckAndSendAddForm = function (txtName, ImageUrl, callback) {
var postUrl = "/admin/add";
var persistedObject = new Object();
$.post(
postUrl,
{ Name: txtName, ImageUrl: txtImageUrl},
function (data) {
if (data.Status == 200) {
console.log("Post status:" + data.Message);
persistedObject.isPersisted = true;
persistedObject.gdEntityId = data.Data;
} else if (data.Status == 500) {
console.log("Failed to post entitiy");
} else {
console.log("Fault with Javascript");
}
callback(); // This is where you return flow to your caller
}, "json"
);
};
Then you invoke like so:
var persistedObject = MyLibrary.fn.CheckAndSendAddEntityForm("name", "avatarurl.png", function()
{
console.log("test");
//check if persisted and go to next step
if (persistedObject.isPersisted) {
MyLibrary.fn.InitAddAnotherEntityForm(persistedObject .gdPronoId);
} else {
alert("Oops, something went wrong. Please call 911");
}
});

JavaScript is single-threaded. If you have asynchronous functionality, a simple boolean semaphore variable will help not to allow invocations of a function while some processes are running.
If you want to execute asynchronous tasks one by one (like a domino line), you will need to use callback functions.

What you're encountering is the "asynchronous" bit of AJAX. If you want to physically (as in the line line by line in the Javascript file) you can use the .success,.pipe or .done jQuery methods to add a callback to process the data further. Don't embed your callbacks if you can help it, or you will get a "domino effect" as you call it.