Develop Reference

fs.readFileSync doesn't wait

fs.readFileSync(process.env.id_to_name, 'utf-8', function(err, data) {
if (err) throw err;
/*
a lot of stuff
*/
fs.mkdirSync(`clips`);
fs.writeFileSync(`clips/recap.json`, '{"players":[]}', 'utf8');
});
fs.readFileSync(`clips/recap.json`, 'utf-8', function(err, data) {
var info = JSON.parse(data);
info.players.push(/* stuff */);
fs.writeFileSync(`clips/recap.json`, JSON.stringify(info), 'utf8', function (err) { });
});
I don't know what I'm doing wrong here.
The second fs.readFileSync just doesn't wait for the first one to end so it doesn't find the file he should read.

You're using fs.readFileSync() incorrectly. It does not accept a callback as an argument and does not call a callback. See doc here.
I don't know if you meant to show us fs.readFile() that does accept a callback or not.
fs.readFileSync() returns its result directly (in a synchronous fashion) from the function call as in:
let data = fs.readFileSync(someFileName, someEncoding);
It does not use a callback and it throws an exception if there's an error reading the file.
If you meant to show us an example using fs.readFile(), then it's a non-blocking, asynchronous call. If you want your second file read to wait until the first one is done, you would have to put the second file read INSIDE the completion callback of the first.
Also, please never write code like if (err) throw err; inside an asynchronous callback. That's a pointless piece of code that does nothing useful as nothing can catch that throw and you have no way to communicate back errors. It is most unfortunate that nodejs documentation shows that regularly in its examples as real world code should never do that. You will need to write real error handling where you either handle the error in some way and continue or you abort the process or you communicate back the error (probably with a callback) so the calling code can handle and see the error. Exceptions throw in asynchronous callbacks do NOT propagate back to the caller. They end up going back to the bowels of the file system code where the callback was triggered from where you cannot catch or handle them.
If you really mean to be using all synchronous calls, then you would write your code like this:
try {
let data1 = fs.readFileSync(process.env.id_to_name, 'utf-8');
// other code here
fs.mkdirSync(`clips`);
fs.writeFileSync(`clips/recap.json`, '{"players":[]}', 'utf8');
let data2 = fs.readFileSync(`clips/recap.json`, 'utf-8');
var info = JSON.parse(data2);
info.players.push(/* stuff */);
fs.writeFileSync(`clips/recap.json`, JSON.stringify(info));
} catch(err) {
// handle errors here
console.log(err);
}
Note that this code can likely only be run once without error because fs.mkdirSync('clips') will fail unless you set the recursive flag.
Hint, you can use require() to load and parse a JSON file in one step so you don't have to read it and then parse it into a Javascript object.

Node.js Asynchronous return from HTTP request

I'm very new to JS and node. I am trying to return a variable from an asynchronous http request, and I recognize that the problem is the async nature of the call, but cannot figure out how to make a very simple code snippet function despite reading multiple tutorials.
Can someone please mark this up such that it works and I can digest it and modify for my own use? Should I be using the async library? I've taken a couple of attempts and haven't been able to make that work.
var request = require('request');
a = 0;
a = foo();
function foo() {
var num = 1;
request('http://www.google.com', function (error, response, body) {
// console.log('error:', error); // Print the error if one occurred
//console.log('statusCode:', response && response.statusCode); // Print the response status code if a response was received
//console.log('body:', body); // Print the HTML for the Google homepage.
num = 3;
return callback(num);
})
return num;
}
console.log(a);

So I am going to assume that you are making your request correctly, and instead focus on the idea of the callback. If you try this and still need a little more help let me know.
I will try to keep this code close to what you already have:
function foo(callback){
request('http://www.google.com', function (error, response, body) {
if(error)
console.log(error);
//console.log('statusCode:', response && response.statusCode);
// Print the response status code if a response was received
//console.log('body:', body); // Print the HTML for the Google homepage.
num = 3;
callback(num);
})
}
then in order to print a 3 out you would do something like this.
foo(function(value){
console.log(value);
})
You would see the value 3 printed out in this case.
If you wanted to return more then just 3 you would need to pass them as parameters to the callback function that has been passed in. If you were to try to follow callback style more you would probably get something closer to this.
function foo(callback){
request('http://www.google.com', function (error, response, body) {
if(error)
callback(error,null);
//do what ever you need to with the response and body
num = 3;
callback(null,num);
})
}
foo(function(err, value){
if(err)
console.log(err);
console.log(value);
})
You just need to remember that with callbacks, like Rob mentions, you typically don't return. Instead, you have your function call another function that you pass in, and provide that function with parameters that your function determined.

You must/can only operate on the results of an async call within its callback because
callbacks do not return values,
callbacks do not have access to your surrounding context, so in your example num is not defined when the callback is triggered, and
you cannot know when the callback completes and thus its result is available.
In your example code, foo will always immediately return 1 irrespective of the response/body returned from request.

Why does node.js look for error as the first parameter in a callback

I have the following simple code.. however the "data" variable doesn't return the contents of input.txt.
var fs = require("fs");
fs.readFile('input.txt', function (data) {
console.log(data.toString());
});
console.log("Program Ended");
The code below works because node.js reads the first parameter, err, and the input.txt contents come from the 2nd parameter
var fs = require("fs");
fs.readFile('input.txt', function (err, data) {
if (err) return console.error(err);
console.log(data.toString());
});
console.log("Program Ended");
Is this just a node.js thing to look for the error in the first parameter? What if I did not want to check for an error in the callback function?

It's convention to pass error as the first parameter to a callback function. It's first to prevent it from being ignored. You don't have to check it, of course, but if there is an error it's likely that your data is bad or meaningless anyway.
The reason that fs.readFile('input.txt', function (data) { doesn't work is that the error is passed into your data variable, since it is the first parameter. What you actually name the parameters doesn't matter, the parameter order is decided by fs.readFile.

If/Else hangs when trying to respond with different values

I have a method which searches a file for a term and then returns the entire line as a JSON object. For some reason, when I add the else portion of the logic, I get the following error:
_http_outgoing.js:335
throw new Error('Can\'t set headers after they are sent.');
^
Error: Can't set headers after they are sent.
at ServerResponse.OutgoingMessage.setHeader (_http_outgoing.js:335:11)
at ServerResponse.header (/home/vagrant/node_modules/express /lib/response.js:718:10)
at ServerResponse.contentType (/home/vagrant/node_modules/express/lib/response.js:551:15)
at ServerResponse.send (/home/vagrant/node_modules/express/lib/response.js:138:14)
at /home/vagrant/routes/service.js:18:13
at /home/vagrant/model/instance_map.js:22:17
at fs.js:334:14
at FSReqWrap.oncomplete (fs.js:95:15)
If I remove the else statement from below it works, unless the route isn't matched and then the app just hangs unless I add a timeout:
exports.mapData = function(instance, instance_map) {
//read the file and return the matched array
fs.readFile(path.join(__dirname,'../instance-map'), 'utf8', function (err, data) {
if (err) {
return console.log(err);
}
for (line of splitFile(data))
if(line.match(instance)) {
// this is the asynchronous call
instance_map(convertToJSON(line));
} else {
instance_map(instance);
}
})
}
It looks like this error pops up a lot when trying to call res() twice but I am not doing anything like that. Not entirely sure where this error would occur. The actual route which calls the method is as follows:
router.get('/mapinfo/:instance_id', function ( req, res, error) {
file_map.mapData(req.params.instance_id, function (instance_map) {
res.send(instance_map);
});
});
The idea is to match the last parameter to the contents of a file and return the json. I am new to nodejs so there is probably something simple I am missing.

Your if...else is inside a for loop. So res.send() may be called several times.

When you get into the else statement you send to your callback function the "req.params.instance_id".
If your instance_id in the req is Number it will not work:
"The body parameter can be a Buffer object, a String, an object, or an Array."
Express res.send
Also as Igal says, you have for loop, and your for loop looks pretty weird.

Read buffer object in node.js

I'm trying to get html page through this node module called Wreck
It should be very easy to get data but I'm unable to get them
'use strict';
var Wreck = require('wreck');
var url = 'http://www.google.it';
var callback = function(err, response, payload){
Wreck.read(response, null, function(err, body){
//here print out the html page
});
};
Wreck.get(url, callback);
Here above a simple script just a copy from the readme of the developer. according to the documentation body should return a buffer object but how can I read inside a body object? I have read to use toJSON or toString() but I don't get any result

...but I don't get any result
You ARE getting a result, an empty Buffer, but it's not want you want, probably.
The fact is: you are using the read method wrong, passing it inside a callback to the get method. The methods get, post, put and delete already call read internaly and return the readable Buffer for you, in a callback. Take a look at the get doc:
get(uri, [options], callback)
Convenience method for GET operations.
uri - The URI of the requested resource.
options - Optional config object containing settings for both request and read operations.
callback - The callback function using the signature function (err, response, payload) where:
err - Any error that may have occurred during handling of the request.
response - The HTTP Incoming Message object, which is also a readable stream.
payload - The payload in the form of a Buffer or (optionally) parsed JavaScript object (JSON).
So, the use of the get method is pretty straightforward (using your own example):
var callback = function(err, response, payload){
console.log(payload.toString()); // converting the buffer to a string and logging
};
Wreck.get(url, callback);