Develop Reference

Mocha.js - How to save a global variable?

I'm working with Mocha.js for testing in a Node.js - Express.js - Firebase
I need a token from Firebase to access the API endpoints, I have a before hook in all my files, but after about 250 tests, probably calling the authentication endpoint multiple times, I'm getting rate limited by firebase.
I want to get the token once and use it in all my tests.
The tests are spread in different files, I have an index.js that requires them all.
I'm aware of Root Level Hooks, but how can I save the token and use it in all my separate files?
Thanks!

you can create a function that gets the token. then call it. then create your test suite only after that
function getToken(callback) {
//
}
// define tests
function allTests(token) {
describe(xxxxxx, function () {
it(xxxxxxxxx, function() {
//
})
});
}
// start all
getToken(function(token) {
allTests(token);
});

I managed to solve it myself, if anyone needs an answer on how to approach it, take a look at this.
I have multiple files where we write our unit testing, we unite them in an index.spec.js that we execute for testing ($ mocha index.spec.js)
I created a utility file that looks like this:
let token;
(() => { token = getYourToken() })()
module.exports = {
getToken: () => {
return new Promise((resolve) => {
const interval = setInterval(() => {
if (token) {
clearInterval(interval);
resolve(token);
}
}, 100);
});
}
};
Basically, it's a singleton, in the index.spec.js I require this file, executing the 'getYourToken()' once (add your logic to get token here). Then I store it in a variable that then I export.
In the export, I use an interval because my current code is not using promises, use your best practice method, interval + Promise worked for me.
This way I require this file in my tests and get the token I got at the beginning once, avoiding rate-limiting and any issue with firebase.

Create a JSON file in your test root directory.
Import the file.
Append a token property with the token value.
Then import it anywhere to access the token property .

Require , how to do a fallback when module does not exist

I have an application where I need to require a file that may or may not be available. If the file is not available, I need to check for another file. and the third option will be default. So far I have this
const file = require('./locales/${test1}') || require('./locales/${test2}') || require('./locales/default')
But it gives me error saying cannot find module. How do I do it optimally?
I did try https://www.npmjs.com/package/node-require-fallback but it does not seem to work in spite of my node version being OK
const messages = require('./locales/${test1}') works well but
const messages = requireIfExists('./locales/${test1}', './locales/${test2}') FAILS

In a vanilla node app, you can use a try-catch block:
var module;
try {
module = require(`./locales/${test1}`);
} catch(error) {
// Do something as a fallback. For example:
module = require('./locales/default');
}
}

Using try/except, you can implement a function that replicates requireIfExists on your own.
function requireIfExists(...modules) {
for (let module of modules) {
try {
return require(module);
} catch (error) {
// pass and try next file
}
}
throw('None of the provided modules exist.')
}
Also, make sure you are using the ` character instead of quotes when using template strings.

Trading RAM by CPU (performance issue)

I'm working with a program that deals with files, I can do several things like rename them, read the contents of them, etc.
Today I'm initializing it as follows:
return new Promise((resolve, reject) => {
glob("path/for/files/**/*", {
nodir: true
}, (error, files) => {
files = files.map((file) => {
// properties like full name, basename, extension, etc.
});
resolve(files);
});
});
So, i read the content of a specific directory, return all files within an array, and then use the Array.map to iterate over the array and change the paths for a object with properties.
Sometimes i work with 200.000 text files, so, this is becoming a problem because it is consuming too much RAM.
So, i want replace by a construction function with lazy loading.. but i never did that before... so i'm looking for a help hand.
That's my code:
class File {
constructor(path) {
this.path = path;
}
extension() {
return path.extname(this.path);
}
// etc
}
So, my main question is: should i only return the evaluation of the property, or should i replace it? Like this:
extension() {
this.extension = path.extname(this.path);
}
I understand this is a trade off.. i'm going to trade the memory by cpu usage.
Thank you.

If you want to reduce RAM usage, I suggest you store an extra meta-data file for each path, as follows:
Keep the paths in memory, or some of them, as necessary.
Save files properties to hard drive
files.forEach( (file) => {
// collect the properties you want for the file
// ...
var json = { path: file, extension: extension, .. }
// mark the metadata file so you can access it later, for example: put it in the same path with a suffix
var metaFile = path + '_meta.json';
fs.writeFile(metaFile, JSON.stringify(json), (err) => {
if (err) throw err;
});
});
Now all the meta data is on hard drive. This way, I believe, you trade memory for disk space and CPU calls.
If you wish to get properties for a file, just read and JSON.parse its corresponding meta data file.

There's no reason to trade CPU for space. Just walk the tree and process files as they're found. The space needed for walking the tree is proportional to the tree depth if it's done depth first. This is almost certainly has same overhead as just creating the list of paths in your existing code.
For directory walking, the node.js FAQ recommends node-findit. The documentation there is pretty clear. Your code will look something like:
var finder = require('findit')(root_directory);
var path = require('path');
var basenames = [];
finder.on('file', function (file, stat) {
basenames.push(path.basename(file));
// etc
}
Or you can wrap the captured values in an object if you like.

If you store only path property NodeJS class instance take for your example 200k * (path.length * 2 + 6) bytes memory.
If you want to use lazy loading for basenames, extenstions etc use lazy getters
class File {
constructor(path) {
this.path = path;
this._basename = null;
this._extname = null;
}
get extname() {
return this._extname || (this._extname = path.extname(this.path));
}
get basename() {
return this._basename || (this._basename = path.basename(this.path));
}
}

Node.js listen for module load

With RequireJS on the front-end, we can listen to see when modules get loaded into the runtime module cache using:
requirejs.onResourceLoad = function (context, map, depArray) {
console.log('onResourceLoad>>>', 'map.id:', map.id, 'context:', context);
};
Can we do this with Node.js somehow? Will be useful for debugging. Especially when servers are loading different files (or in different order) based on configuration.
I assume this might be documented in
https://nodejs.org/api/modules.html
but I am not seeing anything

If you look at the source code for require(), you will find this:
Module._load = function(request, parent, isMain) {
if (parent) {
debug('Module._load REQUEST %s parent: %s', request, parent.id);
}
This shows that you can leverage the debug() call to get the information you need. In order to do this, you will notice that module is setup using util.debuglog('module'). This means that you need to run your application with with the NODE_DEBUG variable set to module. You can do it the following way from the console:
NODE_DEBUG=module node main.js
This will log what you are looking for.

I'm not aware of a documented callback API for the purpose of module load callbacks (although a logging mechanism for module loading appears to exist).
Here's a quick workaround to the apparent lack of such a callback, by monkeypatching Module._load:
const Module = require('module');
const originalModuleLoad = Module._load;
Module._load = function() {
originalModuleLoad.apply(this, arguments);
console.log("Loaded with arguments " + JSON.stringify(arguments));
}
I executed the above code in a REPL and then did require('assert'). Lo and behold:
> require('assert')
Loading with arguments {"0":"assert","1":{"id":"<repl>","exports":{},"filename":null,"loaded":false,"children":[],"paths":["/Users/mz2/Projects/manuscripts-endnote-promo/repl/node_modules","/Users/mz2/Projects/manuscripts-endnote-promo/node_modules","/Users/mz2/Projects/node_modules","/Users/mz2/node_modules","/Users/node_modules","/Users/mz2/.nvm-fish/v6.1.0/lib/node_modules","/Users/mz2/.node_modules","/Users/mz2/.node_libraries","/Users/mz2/.nvm-fish/v6.1.0/lib/node"]},"2":false}
Please don't think about using code like above for anything but debug only purposes.

Because node.js modules are imported (required) synchronously, simply having the require statement means the module is imported.
While RequireJS can import modules asynchronously, the even listening is an important feature, but native require in Node.js leaves this necessity out. This way, as you probably know:
const module = require('module')
// You can use the module here, async or sync.
To add to that, not only require is sync, but also in order to use a module it has to be explicitly required in the same file where it's used. This can be bypassed in several ways, but best practice is to require in every module where you use a module.
For specific modules which require async initialization, either the module should provide an event, or you can wrap the init function using a promise or a callback. For example, using a promise:
const module = require('module')
// Create a promise to initialize the module inside it:
const initialized = new Promise((resolve, reject) => {
// Init module inside the promise:
module.init((error) => {
if(error){
return reject(error)
}
// Resolve will indicate successful init:
resolve()
})
})
// Now with wrapped init, proceed when done:
initialized
.then(() => {
// Module is initialized, do what you need.
})
.catch(error => {
// Handle init error.
})

nodejs fs.exists()

I'm trying to call fs.exists in a node script but I get the error:
TypeError: Object # has no method 'exists'
I've tried replacing fs.exists() with require('fs').exists and even require('path').exists (just in case), but neither of these even list the method exists() with my IDE. fs is declared at the top of my script as fs = require('fs'); and I've used it previously to read files.
How can I call exists()?

Your require statement may be incorrect, make sure you have the following
var fs = require("fs");
fs.exists("/path/to/file",function(exists){
// handle result
});
Read the documentation here
http://nodejs.org/api/fs.html#fs_fs_exists_path_callback

You should be using fs.stats or fs.access instead. From the node documentation, exists is deprecated (possibly removed.)
If you are trying to do more than check existence, the documentation says to use fs.open. To example
fs.access('myfile', (err) => {
if (!err) {
console.log('myfile exists');
return;
}
console.log('myfile does not exist');
});

Do NOT use fs.exists please read its API doc for alternative
this is the suggested alternative : go ahead and open file then handle error if any :
var fs = require('fs');
var cb_done_open_file = function(interesting_file, fd) {
console.log("Done opening file : " + interesting_file);
// we know the file exists and is readable
// now do something interesting with given file handle
};
// ------------ open file -------------------- //
// var interesting_file = "/tmp/aaa"; // does not exist
var interesting_file = "/some/cool_file";
var open_flags = "r";
fs.open(interesting_file, open_flags, function(error, fd) {
if (error) {
// either file does not exist or simply is not readable
throw new Error("ERROR - failed to open file : " + interesting_file);
}
cb_done_open_file(interesting_file, fd);
});

As others have pointed out, fs.exists is deprecated, in part because it uses a single (success: boolean) parameter instead of the much more common (error, result) parameters present nearly everywhere else.
However, fs.existsSync is not deprecated (because it doesn't use a callback, it just returns a value), and if the whole rest of your script depends on checking the existence of a single file, it can make things easier than having to deal with callbacks or surrounding the call with try/catch (in the case of accessSync):
const fs = require('fs');
if (fs.existsSync(path)) {
// It exists
} else {
// It doesn't exist
}
Of course, existsSync is synchronous and blocking. While this can sometimes be handy, if you need to do other operations in parallel (such as checking for the existence of multiple files at once), you should use one one of the other callback-based methods.
Modern versions of Node also support promise-based versions of fs methods, which one might prefer over callbacks:
fs.promises.access(path)
.then(() => {
// It exists
})
.catch(() => {
// It doesn't exist
});