Develop Reference

How to mock fs module together with unionfs?

I have written a test case that successfully load files into virtual FS, and at the same time mounted a virtual volume as below
describe("should work", () => {
const { vol } = require("memfs");
afterEach(() => vol.reset());
beforeEach(() => {
vol.mkdirSync(process.cwd(), { recursive: true });
jest.resetModules();
jest.resetAllMocks();
});
it("should be able to mock fs that being called in actual code", async () => {
jest.mock("fs", () => {
return ufs //
.use(jest.requireActual("fs"))
.use(createFsFromVolume(vol) as any);
});
jest.mock("fs/promises", () => {
return ufs //
.use(jest.requireActual("fs/promises"))
.use(createFsFromVolume(vol) as any);
});
const { createFsFromVolume } = require("memfs");
const { ufs } = require("unionfs");
const { countFile } = require("../src/ops/fs");
vol.fromJSON(
{
"./some/README.md": "1",
"./some/index.js": "2",
"./destination": null,
},
"/app"
);
const result = ufs.readdirSync(process.cwd());
const result2 = ufs.readdirSync("/app");
const result3 = await countFile("/app");
console.log({ result, result2, result3 });
});
});
By using ufs.readdirSync, I can access to virtual FS and indeed result giving me files that loaded from disc into virtual FS, result2 representing /app which is a new volume created from vol.fromJSON.
Now my problem is I am unable to get the result for result3, which is calling countFile method as below
import fsPromises from "fs/promises";
export const countFile = async (path: string) => {
const result = await fsPromises.readdir(path);
return result.length;
};
I'm getting error
Error: ENOENT: no such file or directory, scandir '/app'
which I think it's because countFile is accessing the actual FS instead of the virtual despite I've had jest.mock('fs/promises')?
Please if anyone can provide some lead?

This is the function you want to unit test.
//CommonJS version
const fsPromises = require('fs/promises');
const countFile = async (path) => {
const result = await fsPromises.readdir(path);
return result.length;
};
module.exports = {
countFile
}
Now, how you would normally go about this, is to mock fsPromises. In this example specifically readdir() since that is the function being used in countFile.
This is what we call: a stub.
A skeletal or special-purpose implementation of a software component, used to develop or test a component that calls or is otherwise dependent on it. It replaces a called component.
const {countFile} = require('./index');
const {readdir} = require("fs/promises");
jest.mock('fs/promises');
beforeEach(() => {
readdir.mockReset();
});
it("When testing countFile, given string, then return files", async () => {
const path = "/path/to/dir";
// vvvvvvv STUB HERE
readdir.mockResolvedValueOnce(["src", "node_modules", "package-lock.json" ,"package.json"]);
const res = await countFile(path);
expect(res).toBe(4);
})
You do this because you're unit testing. You don't want to be dependent on other functions because that fails to be a unit test and more integration test. Secondly, it's a third-party library, which is maintained/tested by someone else.
Here is where your scenario applies. From my perspective, your objective isn't to test countFile() rather, to test fsPromises and maybe test functionality to read virtual file-systems: unionfs. If so then, fsPromises doesn't need to really be mocked.

loop in a module export javascript/sequelize

I have an index.js script that contains sequelize models
here is the tree structure of my folder
models/
files/
xml_files/
csv_files/
txt_files/
index.js
server.js
this my index.js code :
const generate_files = require('./files')
const File = generate_files(sequelize, DataTypes)
const generate_xml_files = require('./xml_files')
const Xml_File = generate_xml_files(sequelize, DataTypes)
const generate_csv_files = require('./csv_files')
const Csv_File = generate_csv_files(sequelize, DataTypes)
const generate_txt_files = require('./txt_files')
const Txt_File = generate_txt_files(sequelize, DataTypes)
module.exports = {
Files, Xml_File, Csv_File, Txt_File
}
in `` server.js ``` I imported each model like this :
const {Files, Xml_File, Csv_File, Txt_File} = require('./models')
Now in server.js I want to get the name of the module sequelize then I check if the name of the module matches the name of the table like this :
const {Files, Xml_File, Csv_File, Txt_File} = require('./models')
if (Files.name == TableName){
Files.findAll()
}
if (Xml_File.name == TableName){
Xml_File.findAll()
}
if (Csv_File.name == TableName){
Csv_File.findAll()
}
....
how can I avoid doing tests every time ?

You can do:
const Example = require('./index')
Example.Files
As for making a loop, I'm not sure you can do that, but you can do:
export const File = generate_files(sequelize, DataTypes)
Which is the same thing as:
module.exports = {File}

You can create an object and loop through the entries, like so.
var exports = {
'File': './files',
'Xml_File': './xml_files',
'Csv_File': './csv_files',
'Txt_File': './txt_files'
};
var e = {};
Object.entries(exports).forEach(ex=>{
e[ex[0]] = require(ex[1])(sequelize, DataTypes);
});
module.exports = e;
Then you'll want to import them from this index file...
const {Files, Xml_File, Csv_File, Txt_File} = require('./models/index.js')
Per your update..
const Models = require('./models/index.js');
Object.entries(Models).forEach(model=>{
if (model.name == TableName){
model.findAll();
}
}

To simplify your index.js you could also do it like this:
module.exports = {
Files: require('./files')(sequelize, DataTypes),
Xml_File: require('./xml_files')(sequelize, DataTypes),
Csv_File: require('./csv_files')(sequelize, DataTypes),
Txt_File: require('./txt_files')(sequelize, DataTypes)
}
it's easier to read, at least for me.
Another way is to loop over the dir so you never need to change the code in index.js even if you add another folder like /xxx_files
const Fs = require('fs');
var dir = Fs.readdirSync(__dirname, { withFileTypes: true });
var exports = {};
dir.forEach(d => {
if (d.isDirectory())
exports[d.name] = require('./' + d)(sequelize, DataTypes);
});
// !!!!!!! the names of the exports will be names of the directories
module.exports = exports;
And to avoid the checks if (Files.name == TableName)
all you need is to not import like this:
const {Files, Xml_File, Csv_File, Txt_File} = require('./models')
but rather like this:
const models = require('./models')
// models = { files, xml_files, csv_files, txt_files }
then you can do this:
var db = models[TableName] // assuming TableName is files, xml_files etc.
if (db)
doSomething();

Node JS can't find module

I have a files.js file and am requiring it at the start of my code.
When I use:
const files = require('./lib/files');
I get this error:
Error: Cannot find module './lib/files'
However if I test it with another one of the files in the same folder like:
const files = require('./lib/repo');
It runs.
files.js:
const fs = require('fs');
const path = require('path');
module.exports = {
getCurrentDirectoryBase: () => {
return path.basename(process.cwd());
},
directoryExists: (filePath) => {
return fs.existsSync(filePath);
}
};
I would use tree command however I have too many node modules installed to show it correctly so:

const { getCurrentDirectoryBase, directoryExists } = require('./lib/files')

Fail to navigate to a proper folder in nodejs

I'm trying to create a function that pull a remote repository and then navigate through this repo and install it's dependencies but somehow it fail to install the dependencies inside the cloned repo and it install them outside:
const spinner = clui.Spinner;
const git = require("simple-git/promise");
const path = require("path");
const { install } = require("pkg-install");
async function pullRepo() {
const pulling = new spinner("Initializing project...");
const installing = new spinner("Installing dependencies...");
const rep = await inquirer.DirectoryName();
const package = path.join(rep.project, "package.json");
pulling.start();
await git()
.silent(true)
.clone("git#github.com:blacklane/create-blacklane-app.git", rep.project)
.then(async () => {
pulling.stop();
console.log(`working directory:`, process.cwd());
// check file exist asynchronously
fs.access(package, fs.constants.F_OK, err => {
console.log(`${package} ${err ? "does not exist" : "exists"}`);
});
installing.start();
const obj = JSON.parse(fs.readFileSync(package, "utf8"));
const dependencies = { ...obj.dependencies, ...obj.devDependencies };
process.chdir(rep.project); // navigate to directory to install dpendencies
console.log(`new working directory from git:`, process.cwd());
const { stdout } = await install(dependencies, {
dev: true,
prefer: "npm"
});
console.log(stdout);
installing.stop();
})
.catch(error => console.error("failed: ", error));
// progress.finish();
}

The reason this can happen because install is somehow not able to get to the cloned repo. May be because underlying shell which is running the Node process is still the same.
following is the working code:
const clui = require('clui');
const spinner = clui.Spinner;
const git = require("simple-git/promise");
const path = require("path");
const fs = require('fs');
const fsPromises = fs.promises;
const { spawn } = require('child_process');
async function pullRepo(repo, dirName) {
const pulling = new spinner("Initializing project...");
const installing = new spinner("Installing dependencies...");
pulling.start();
await git().silent(true).clone(repo, dirName);
pulling.stop();
const package = path.join(dirName, "package.json");
// check file exist asynchronously
await fsPromises.access(package, fs.constants.F_OK);
installing.start();
const npmInstall = spawn('npm', ['i'], { cwd: dirName });
npmInstall.stdout.on('data', (data) => {
console.log(`stdout: ${data}`);
});
npmInstall.stderr.on('data', (data) => {
console.error(`stderr: ${data}`);
installing.stop();
});
npmInstall.on('close', (data) => {
installing.stop();
});
}
const repoToPull = "https://github.com/facebook/create-react-app.git"; // can be any repo
const dirToPullTo = path.join(__dirname,'gitpull'); // directory you want to pull it to.
pullRepo(
repoToPull,
dirToPullTo
).then(res => console.log(res));
This code needs better error handling.
You don't need to add .then when you are awaiting a promise

Does having the same `require` in multiple files increase runtime

So I'm planning to separate my functions into separate files and then import them into a single index.js which then becomes the main exporter. So I'm wondering if having something like var bcrypt = require('bcrypt') in several of my files be slower than just having it in one file.
Here's how I'm planning to group and export in index.js
const fs = require('fs');
const path = require('path')
const modules = {}
const files = fs.readdirSync(__dirname)
files.forEach(file => {
if (file === 'index.js') return
let temp = require(path.join(__dirname, file))
for (let key in temp) {
modules[key] = temp[key]
}
});
module.exports = modules
As an example of what I mean:
file1.js
var bcrypt = require("bcrypt");
module.exports.file1test = "hi"
file2.js
var bcrypt = require("bcrypt");
module.exports.file2test = "bye"

No, it does not. Whenever a module is required for the first time, the module's code runs, assigns something to its exports, and those exports are returned. Further requires of that module simply reference those exports again. The logic is similar to this:
const importModule = (() => {
const exports = {};
return (name) => {
if (!exports[name]) exports[name] = runModule(name);
return exports[name];
};
})();
So, multiple imports of the same module is no more expensive than referencing an object multiple times.