I am using "pdf-text" module for Node.js to convert a pdf into a string array and then get specific elements out of it. But the problem is, I can only access the data, "chunks", only when I am inside the callback. I want to store it in some global variable so that I can use it in different files. I have tried storing the elements of the array inside variables while inside the function, but no luck. Here's the code:
var pdfText = require('pdf-text');
var pathToPdf = "PDF FILE NAME";
var fs = require('fs');
var buffer = fs.readFileSync(pathToPdf);
var output;
pdfText(buffer, function(err, chunks){
if (err){
console.dir(err);
return;
}
console.dir(chunks);
output = chunks;
}
console.dir(output);
P.S. I am fairly new to Node.js and JavaScript and help would be appreciated greatly.
The output variable will only be set with "chunks" contents when the callback is called.
Btw, you need to add ");" after the callback function declaration on the pdfText function call.
var pdfText = require('pdf-text');
var pathToPdf = "PDF FILE NAME";
var fs = require('fs');
var buffer = fs.readFileSync(pathToPdf);
var output;
pdfText(buffer, function(err, chunks){
if (err){
console.log(err);
return;
}
otherFunction(); // undefined
output = chunks;
otherFunction(); // chunks content
});
function otherFunction() {
console.log(output);
}
console.log(output); // undefined
About js callbacks: https://www.tutorialspoint.com/nodejs/nodejs_callbacks_concept.htm
But the problem is, I can only access the data, "chunks", only when I am inside the callback.
Yes, that is correct. You can't access the data before it is available, and when it becomes available, your callback gets called with the data.
I want to store it in some global variable so that I can use it in different files.
Suppose you did this. Now you have a problem. Your code in those different files: how will it know when the data is ready? It won't.
You need some way to tell that code the data is ready. The way you tell that code is by calling a function. And at that point you don't need global variables: when you call the function in that other file, you pass the data to it as a function parameter.
In other words, don't just have global code in some file that expects to be able to use your chunks data by referencing a global variable. Instead, write a function that you can call from your callback, and pass chunks into that function.
If you are using node 8, I believe you can use the async-await feature. So you can refactor your code so that it looks like the following:
var pdfText = require('pdf-text');
var pathToPdf = "PDF FILE NAME";
var fs = require('fs');
var buffer = fs.readFileSync(pathToPdf);
var output;
async function getPDF(buffer) {
pdfText(buffer, function(err, chunks){
if (err){
console.dir(err);
return;
}
return await chunks;
}
}
// you can get the chunks given the buffer here!
console.dir(getPDF(buffer));
I want to store it in some global variable so that I can use it in different files. I have tried storing the elements of the array inside variables while inside the function, but no luck.
I don't think you can store the chunks as a global variable though as you would have to export the chunk (e.g module.exports = getPDF(buffer);), which is synchronous, while the function getPDF is asynchronous. So you have to use it within the same file. What I would do, is import the function instead and then pass it a different buffer in different js files where different pdf is required. Hope this helps.
Related
I have the below function, which reads text from a file and outputs the text into the data variable.
fs.readFile('a1.txt', 'utf8', function(err, data) {
if (err) throw err;
console.log(data);
});
I want to assign data to a global variable so i can use it in other parts of my program. at the moment I am unable to use the information taken from data. What can I do to store data into another variable which i can use freely in other functions?
What you're asking for is not really global, just a higher scope than the function.
Seeing as you're in node.js, you can just put a variable at the top of your file (or wherever):
var a1Data;
then just use a1Data = data in the callback.
If you truly want something global, you can use global.whatever = data, but that's typically not what you want.
I need to load untrusted modules, written by third parties. I'm using vm for the sandbox and I was thinking to use threads (from npm: here) in order to load the module asynchronously and avoid blocking code.
I have stripped down the code to the minimum, because I'm stuck and I dont' understand if what I'm trying to achieve is impossible or it's just me messing with scopes.
Here is a dummy module:
exports.dummy = function () {
console.log('Dummy');
};
exports.val = 5;
And here is a module where I try to load this dummy module using threads:
var spawn = require('threads').spawn;
var mod;
var other;
var t = spawn(function (input, done) {
var m = require(input.dir + '/dummyMod');
done({m: m, other: 'hey'})
});
t.send({dir: __dirname}).on('message', function (result) {
mod = result.m;
other = result.other;
console.log(mod);
console.log(other);
t.kill();
});
The logged output is:
{ val: 5 }
hey
As you can see, the function in the dummy module has been skipped. If I try to load the module in the main process and log it, then the function is, of course, part of the object.
You need to properly serialize and deserialize the function. JSON.stringify ignores functions, probably because json is a format for storing data, not scripts.
Serialize the function by calling toString() on it. Then you can send it along as a string.
done({m: m.toString(), other: 'hey'})
Converting m to a string will give you something like this:
"function m(){console.log(\'called m()\')}"
On the receiving end, you will need to deserialize the function.
var m = new Function("return " + result.m)()
I am working on a bot and my goal is to essentially create a counter for my bot every time it is called. My first thought is to have a variable stored in another file and when the main js file is called it calls a method to update the variable in the other file. Do I have to use require() or is there another way for me to do this?
You should either store data in database or a JSON file. In the case of JSON file, the code can be written like this.
let fs = require('fs');
let updateCounter = () => {
fs.readFile('/path/to/counter.json', 'utf8', (er, data) => {
if(er){
throw er;
}
data = JSON.parse(data);
data.calledTimes++;
fs.writeFile('/path/to/counter.json', JSON.stringify(data), er => {
if(er){
throw er;
}
});
});
}
counter.json
{
"calledTimes": 0
}
I can think of 2 ways so far:
in another.js, you module.exports = {counter:0}; and in main.js you require(./another).counter++; (every time the return of require is the original object (nodejs's cache mechanism)
put the counter into global scope: global.counter = 0;
Hope this helps you. Good luck.
The following are the contents of the file tags.js.
//the following initializes the MongoDB library for Javascript.
var MongoClient = require('mongodb').MongoClient,
assert = require('assert');
//connection URL
abc = 'mongodb://localhost:27017/abcbase/
/* the following is a global variable
where I want retrieved content */
var arrayToReturn = [];
/* toFind is a dictionary, toSearch is a
MongoDB collection (input is its name as a string) */
function findElemHelper(toSearch, toFind) {
/* establishes connection to the database abcbase
and runs the contents of the function that is
the second input */
MongoClient.connect(abc, function(err, db) {
/* coll is the collection being searched.
tempCurs is the results of the search returned
as a cursor
*/
coll = db.collection(toSearch);
tempCurs = coll.find(toFind);
/* the three lines below this comment
form the crux of my problem. I expect
arrayToReturn to be assigned to docs
(the contents of the cursor received by
the find function). Instead it seems like
it is declaring a new variable in local scope.*/
tempCurs.toArray(function(err, docs) {
arrayToReturn = docs;
});
});
}
function findElem(toSearch, toFind) {
findElemHelper(toSearch, toFind);
return arrayToReturn;
}
function returnF() {
return arrayToReturn;
}
var ln = findElem("userCollection", {});
var lm = returnF();
console.log(ln);
console.log(lm);
When I run the file using the node interpreter with the command node tags.js, it prints
[]
[]
And when I run the same code on the node interpreter (which I enter with the command node from Terminal and copy-paste the same code into the shell), console.log(ln) prints [] and console.log(lm) prints the contents of the document I want to retrieve from MongoDB.
Could someone explain this behavior?
As some commenters pointed at, the problem is the async nature of the findElemHelper method. This long, but detailed answer posted on one of the comments explains the basics behind async in javascript and how to approach this style of coding in general.
In a shorter answer, with asynchronous code you cannot assume the order of operations is the same as statements in your code. You are correct in identifying the location of the crux of your problem, but the issue is not scope, but rather that the function you passed into tempCurs.toArray is called whenever the database returns data, which could be after the rest of the file has finished executing. (The same is true of the function passed into MongoClient.connect, where you could end up calling console.log before the db even connects!)
Here's how we solve the problem with callbacks, the goal is to structure our code such that we are certain that the database has returned the data before calling console.log:
var MongoClient = require('mongodb').MongoClient;
var abc = 'mongodb://localhost:27017/abcbase/';
/**
* Take a callback function as the last parameter which will
* be called when the array is retrieved.
*/
function findElem(toSearch, toFind, callback) {
MongoClient.connect(abc, function(err, db) {
var coll = db.collection(toSearch);
var tempCurs = coll.find(toFind);
tempCurs.toArray(callback);
});
}
findElem("userCollection", {}, function(err, docs) {
console.log(docs);
});
Your function findElemHelper makes an asynchronous call to MongoDB with a callback. Thus, you don't really know when arrayToReturn has its contents populated. Chances are, printing lm in the console means you gave it enough time to actually populate.
You should try to restructure your code so that you can use the response from the asynchronous call in your callback, rather than outside in a global variable.
Just started coding last thursday, bear with me here:
my code for this question of the tutorial is returning a list of just the extension names from the directory and not a list of the files with the said extension, e.g. if i used a directory with 3 .js files and used js as my extension argument in the command line, then i would get
1. js
2. js
3. js
as the output, here is the question from the tutorial and my code. THANK YOU!
the question from learnyounode tutorial number 6:
LEARN YOU THE NODE.JS FOR MUCH WIN!
─────────────────────────────────────
MAKE IT MODULAR
Exercise 6 of 13
This problem is the same as the previous but introduces the concept of modules. You will need to create two files to solve this.
Create a program that prints a list of files in a given directory, filtered by the extension of the files. The first argument is the directory name and the second argument is the extension filter. Pr
int the list of files (one file per line) to the console. You must use asynchronous I/O.
You must write a module file to do most of the work. The module must export a single function that takes three arguments: the directory name, the filename extension string and a callback function, in
that order. The filename extension argument must be the same as was passed to your program. i.e. don't turn it into a RegExp or prefix with "." or do anything else but pass it to your module where y
ou can do what you need to make your filter work.
The callback function must be called using the idiomatic node(err, data) convention. This convention stipulates that unless there's an error, the first argument passed to the callback will be null, a
nd the second will be your data. In this case, the data will be your filtered list of files, as an Array. If you receive an error, e.g. from your call to fs.readdir(), the callback must be called wi
th the error, and only the error, as the first argument.
You must not print directly to the console from your module file, only from your original program.
In the case of an error bubbling up to your original program file, simply check for it and print an informative message to the console.
These four things are the contract that your module must follow.
Export a single function that takes exactly the arguments described.
Call the callback exactly once with an error or some data as described.
Don't change anything else, like global variables or stdout.
Handle all the errors that may occur and pass them to the callback.
The benefit of having a contract is that your module can be used by anyone who expects this contract. So your module could be used by anyone else who does learnyounode, or the verifier, and just work. *
and my code is:
module (p6m.js):
var fs=require('fs'), ph=require('path'), exports =module.exports={}
exports.f=function(path,ext,callbk){
fs.readdir(path,function(err,files){
if(err){
return callbk(err,null)
}
files=files.filter(
function(file){
return ph.extname(file)==="."+ext
}
)
return callbk(null,files)}
)}
and my program (p6.js):
var p6m=require('./p6m'), path=process.argv[2], ext=process.argv[3]
p6m.f(path, ext, function(err,files){
if(err){return console.log.error('Error occured:', err)};
files.forEach(function(file){
console.log(file)})})
I got the same problem with my code as of need to use a single function export . So instead of exporting a module function like this :
exports =module.exports={}
exports.f=function(path,ext,callbk){...};
try it doing this way :
module.exports = function (path, ext, callbk) {...};
because its a single function so you don't need to specify that function with a name " f " as if you are doing it in this statement :
exports.f = function(path,ext,callbk){...};
whenever you will import the module,it will automatically call this function only, since the module contains this single function.
You can try this piece of code, it works well for me.
module code: mymodule.js
var fs = require('fs');
var ph= require('path');
module.exports = function (path, ext, callbk) {
var pathio = "." + ext;
fs.readdir(path, function (err, files) {
if (err)
return callbk(err);
else {
var listf = []; //listf is the resultant list
for (var i = 0; i < files.length; i++) {
if (ph.extname(files[i]) === pathio) {
listf.push(files[i]);
}
}
callbk(null, listf);
}
});
}
program code : moduletest.js
var mod = require('./mymodule');
mod(process.argv[2], process.argv[3], function (err, listf) {
if (err) {
console.log('Error!')
} else {
for (var i = 0; i < listf.length; i++) {
console.log(listf[i]);
}
}
});
and do remember, learnyounode series is very specific about its way of coding and syntax, so even if you are doing the logic right way still you won't get pass,you need your code to be the best and optimized. I'll suggest you to refer to discussions on nodeschool itself for various issues you might get in learnyounode series.
That will work and output the right results, but what they are looking for is something like this:
module.exports = function() {};
Because they only want one function total in the exports.
You could also do something like this:
module.exports = FindFilesByExtension;
function FindFilesByExtension(path, ext, callback) {
//your code
}
Here is my solution,
Thsi is my module file filteredls.js
var fs = require('fs');
var path = require('path');
module.exports = function filterFiles(folder, extension, callback) {
fs.readdir(folder, function(err, files) {
if(err) return callback(err);
var filesArray = [];
files.forEach(function(file) {
if(path.extname(file) === "."+extension) {
filesArray.push(file);
}
});
return callback(null, filesArray);
});
}
And here is my test file for reading module modular.js
var ff = require('./filteredls.js');
ff(process.argv[2], process.argv[3], function(err, data) {
if(err)
return console.error(err);
data.forEach(function(file) {
console.log(file);
});
});
And this is my result screenshot,