This question already has answers here:
Call An Asynchronous Javascript Function Synchronously
(13 answers)
Closed last month.
I am currently developing a plugin for eslint. I have encountered the following problem.
I want to make hot changes to the configuration file, so we are required to make an Http request to get the JSON file with the configuration.
As for the context execution when reading the package.json, and traversing the scripts, when adding async/await, it executes in node the code, but does not return the context response from the context object.
If I do it with callbacks exactly the same thing happens. I leave you the code so that you have a visual idea of the problem.
With Promises using then(), catch(), doesnt works too.
I think that what I am asking for in the JavaScript world is not possible, but by asking I don't lose anything. I understand that the functions like execSync, are native inside the C core written in JavaScript, and that's why they can be executed synchronously.
Thanks and greetings.
'use strict';
const { readFileSync } = require("fs");
//------------------------------------------------------------------------------
// Meta Definition
//------------------------------------------------------------------------------
const meta = {
type: 'problem',
docs: {
description: `Rule to don't allow changes in scope scripts of package.json`,
category: 'Possible Errors'
},
schema: [
{
type: 'object',
properties: {
alternatives: {
type: 'object',
patternProperties: {
'^.+$': { type: 'string' }
}
}
},
additionalProperties: false
}
]
};
//------------------------------------------------------------------------------
// Rule Definition
//------------------------------------------------------------------------------
const TEST_CONTENT = 'jest';
const isPackageJsonFile = filePath => filePath.includes('package.json');
module.exports = {
meta,
create: (context) => {
return {
'Program:exit': async node => {
if (!isPackageJsonFile(context.getFilename())) {
return;
}
const packageJsonRaw = readFileSync(context.getFilename(), 'utf8');
const packageJson = JSON.parse(packageJsonRaw );
// Peticion async / await
const testScript = await httpRequest.getJSONFromURL().test || TEST_CONTENT;
if (!packageJson.scripts || packageJson.scripts.test !== TEST_CONTENT) {
context.report({
message: `Dont allow changes on script "scripts.test" of package.json`,
node
});
}
}
}
}
};
A way to be able to return the result of the http request, like fileReadSync but without callbacks because it doesn't work, is it possible, I can't think how to do it.
The implementation that I have made at the moment, I make use of child_process with curl, with an execSync, and with that I have managed to make it work, but the truth is that I am not convinced at all by this implementation.
try this simple way of promising
async function doB(){
try{
const res = await CertainPromise()
return res.json();//something
}catch(e){}
}
This should actually work;
Let's say we are creating a module called app by constructing a new vm.SourceTextModule object:
const context = {
exports: {},
console, // custom console object
};
const sandbox = vm.createContext(context);
const app = new vm.SourceTextModule(
`import path from 'path';
console.log(path.resolve('./src'));`,
{
context: sandbox,
}
);
According to the Node.js documentation to obtain the default export from path module we should "link" the imported dependencies of app module to it.
To achieve this we should pass linker callback to app.link method:
async function linker(specifier, referencingModule) {
// the desired logic...
}
await app.link(linker);
How to implement linker function properly so that we could import path module in newly created app module and use it:
await app.evaluate(); // => /home/user/Documents/project/src
P.S. We are using TypeScript, so I checked if we have installed types for path package.
package.json:
"#types/node": "^17.0.31",
I found https://github.com/nodejs/node/issues/35848 where someone posted a code snippet.
From there I've adapted the following linker callback:
const imports = new Map();
async function linker(specifier, referencingModule) {
if (imports.has(specifier))
return imports.get(specifier);
const mod = await import(specifier);
const exportNames = Object.keys(mod);
const imported = new vm.SyntheticModule(
exportNames,
() => {
// somehow called with this === undefined?
exportNames.forEach(key => imported.setExport(key, mod[key]));
},
{ identifier: specifier, context: referencingModule.context }
);
imports.set(specifier, imported);
return imported;
}
The code snippet from the GitHub issue didn't work for me on Node 18.7.0 as is, because the evaluator callback passed to the constructor of SyntheticModule is somehow called with this set to undefined. This may be a Node bug.
I also cached the imported SyntheticModules in a Map because if they have internal state, creating a new SyntheticModule every time will reset that state.
I'm trying to work around the fact that I need to import a pure ESM package into a non-module. I can't change that fact about the script.
The workaround I'm trying to use is the import() function (a "dynamic import"). That returns a Promise instead of the actual module. I can't use await since I'm not in a module, so instead I'm using .then().
The pure ESM package (unist-util-visit) is used in a function exported by my script, which is then used in another script. So the import chain goes:
importer.js imports imported.js imports unist-util-visit
So the issue is that anything I export from within the .then() function in imported.js does not show up in importer.js.
And it's not even a timing issue. I used an EventEmitter to make importer.js wait until imported.js's .then() is done executing:
imported.js:
const EventEmitter = require('events');
module.exports.emitter = new EventEmitter();
module.exports.outsideFxn = function () {
console.log('hello');
}
import('unist-util-visit').then((unistUtilVisit) => {
module.exports.fxn = function() {
console.log(`unistUtilVisit: ${typeof unistUtilVisit}`);
}
module.exports.emitter.emit('ready');
});
importer.js:
import('./imported.js').then((imported) => {
console.log("In importer.js's .then():");
console.log(' fxn:', imported.fxn);
console.log(' outsideFxn:', imported.outsideFxn);
imported.emitter.on('ready', () => {
console.log("After imported.js is done:")
console.log(' fxn:', imported.fxn);
});
});
When I execute it, this is the output:
$ node importer.js
In importer.js's .then():
fxn: undefined
outsideFxn: [Function (anonymous)]
After imported.js is done:
fxn: undefined
What am I missing? Why are no exports being defined in the .then() function? How can I get my function exported?
Instead of
import('unist-util-visit').then((unistUtilVisit) => {
module.exports.fxn = function() {
console.log(`unistUtilVisit: ${typeof unistUtilVisit}`);
}
module.exports.emitter.emit('ready');
});
where you attempt to modify your module's exports after it has probably been consumed by dependents, why not export a promise that yields the function when it completes?
module.exports.fxnP =
import('unist-util-visit')
.then((unistUtilVisit) => () => {
console.log(`unistUtilVisit: ${typeof unistUtilVisit}`);
});
Now you consume it:
import('./imported.js').then((imported) => {
imported.fxnP.then((fxn) => {
fxn();
});
});
or, more neatly::
import('./imported.js')
.then(({fxnP}) => fxnP)
.then((fxn) => fxn());
You could use a custom require hook like jiti which can do exactly what you want synchronously.
Without hooking require:
$ node test.cjs
// test.cjs
const jiti = require('jiti')();
const unistUtilVisit = jiti("unist-util-visit");
Hooking require programmatically:
$ node test.cjs
// test.cjs
require('jiti')().register();
const unistUtilVisit = require("unist-util-visit");
Hooking require through a command line option:
$ node -r jiti/register test.cjs
// test.cjs
const unistUtilVisit = require("unist-util-visit");
I'm trying to leverage nuxtjs SSG capabilities by creating a static web site where the pages content and navigation are fetched from an API.
I already found my way around on how to dynamically generate the routes by defining a module where I use the generate:before hook to fetch the pages content and routes. When creating the routes I store the page content as the route payload. The following code does just that and works as intended.
modules/dynamicRoutesGenerator.js
const generator = function () {
//Before hook to generate our custom routes
this.nuxt.hook('generate:before', async (generator, generatorOptions) => {
generator.generateRoutes(await generateDynamicRoutes())
})
}
let generateDynamicRoutes = async function() {
//...
return routes
}
export default generator
Now the problem I'm facing is that I have some navigation components that need the generated routes and I was thinking to store them into the vuex store.
I tried the generate:done hook but I don't know how to get the vuex store context from there. What I ended up using was the nuxtServerInit() action because as stated in the docs:
If nuxt generate is ran, nuxtServerInit will be executed for every dynamic route generated.
This is exactly what I need so I'm trying to use it with the following code:
store/index.js
export const actions = {
nuxtServerInit (context, nuxtContext) {
context.commit("dynamicRoutes/addRoute", nuxtContext)
}
}
store/dynamicRoutes.js
export const state = () => ({
navMenuNivel0: {}
})
export const mutations = {
addRoute (state, { ssrContext }) {
//Ignore static generated routes
if (!ssrContext.payload || !ssrContext.payload.entrada) return
//If we match this condition then it's a nivel0 route
if (!ssrContext.payload.navMenuNivel0) {
console.log(JSON.stringify(state.navMenuNivel0, null, 2));
//Store nivel0 route, we could use url only but only _id is guaranteed to be unique
state.navMenuNivel0[ssrContext.payload._id] = {
url: ssrContext.url,
entrada: ssrContext.payload.entrada,
navMenuNivel1: []
}
console.log(JSON.stringify(state.navMenuNivel0, null, 2));
//Nivel1 route
} else {
//...
}
}
}
export const getters = {
navMenuNivel0: state => state.navMenuNivel0
}
The action is indeed called and I get all the expected values, however it seems like that with each call of nuxtServerInit() the store state gets reset. I printed the values in the console (because I'm not sure even if it's possible to debug this) and this is what they look like:
{}
{
"5fc2f4f15a691a0fe8d6d7e5": {
"url": "/A",
"entrada": "A",
"navMenuNivel1": []
}
}
{}
{
"5fc2f5115a691a0fe8d6d7e6": {
"url": "/B",
"entrada": "B",
"navMenuNivel1": []
}
}
I have searched all that I could on this subject and altough I didn't find an example similar to mine, I put all the pieces I could together and this was what I came up with.
My idea was to make only one request to the API (during build time), store everything in vuex then use that data in the components and pages.
Either there is a way of doing it better or I don't fully grasp the nuxtServerInit() action. I'm stuck and don't know how to solve this problem and can't see another solution.
If you made it this far thanks for your time!
I came up a with solution but I don't find it very elegant.
The idea is to store the the API requests data in a static file. Then create a plugin to have a $staticAPI object that expose the API data and some functions.
I used the build:before hook because it runs before generate:before and builder:extendPlugins which means that by the time the route generation or plugin creation happen, we already have the API data stored.
dynamicRoutesGenerator.js
const generator = function () {
//Add hook before build to create our static API files
this.nuxt.hook('build:before', async (plugins) => {
//Fetch the routes and pages from API
let navMenuRoutes = await APIService.fetchQuery(QueryService.navMenuRoutesQuery())
let pages = await APIService.fetchQuery(QueryService.paginasQuery())
//Cache the queries results into staticAPI file
APIService.saveStaticAPIData("navMenuRoutes", navMenuRoutes)
APIService.saveStaticAPIData("pages", pages)
})
//Before hook to generate our custom routes
this.nuxt.hook('generate:before', async (generator, generatorOptions) => {
console.log('generate:before')
generator.generateRoutes(await generateDynamicRoutes())
})
}
//Here I can't find a way to access via $staticAPI
let generateDynamicRoutes = async function() {
let navMenuRoutes = APIService.getStaticAPIData("navMenuRoutes")
//...
}
The plugin staticAPI.js:
import APIService from '../services/APIService'
let fetchPage = function(fetchUrl) {
return this.pages.find(p => { return p.url === fetchUrl})
}
export default async (context, inject) => {
//Get routes and files from the files
let navMenuRoutes = APIService.getStaticAPIData("navMenuRoutes")
let pages = APIService.getStaticAPIData("pages")
//Put the objects and functions in the $staticAPI property
inject ('staticAPI', { navMenuRoutes, pages, fetchPage })
}
The APIService helper to save/load data to the file:
//...
let fs = require('fs');
let saveStaticAPIData = function (fileName = 'test', fileContent = '{}') {
fs.writeFileSync("./static-api-data/" + fileName + ".json", JSON.stringify(fileContent, null, 2));
}
let getStaticAPIData = function (fileName = '{}') {
let staticData = {};
try {
staticData = require("../static-api-data/" + fileName + ".json");
} catch (ex) {}
return staticData;
}
module.exports = { fetchQuery, apiUrl, saveStaticAPIData, getStaticAPIData }
nuxt.config.js
build: {
//Enable 'fs' module
extend (config, { isDev, isClient }) {
config.node = { fs: 'empty' }
}
},
plugins: [
{ src: '~/plugins/staticAPI.js', mode: 'server' }
],
buildModules: [
'#nuxtjs/style-resources',
'#/modules/staticAPIGenerator',
'#/modules/dynamicRoutesGenerator'
]
When running unit tests with Jest in react the window.crypto API is causing problems. I haven't found a way to incorporate crypto in Jest without installing other packages which is something I can't do. So without using another npm package is there a way to test functions that use: crypto.getRandomValues() in them that doesn't crash Jest? Any links, advice, or tips are appreciated
Use the following code to set up the crypto property globally. It will allow Jest to access
window.crypto in the browser environment
global.crypto in non-browsers environments. (Node/Typescript scripts).
It uses the globalThis which is now available on most of the latest browsers as well as Node.js 12+
const crypto = require('crypto');
Object.defineProperty(globalThis, 'crypto', {
value: {
getRandomValues: arr => crypto.randomBytes(arr.length)
}
});
Like #RwwL, the accepted answer did not work for me. I found that the polyfill used in this library did work: commit with polyfill
//setupTests.tsx
const nodeCrypto = require('crypto');
window.crypto = {
getRandomValues: function (buffer) {
return nodeCrypto.randomFillSync(buffer);
}
};
//jest.config.js
module.exports = {
//...
setupFilesAfterEnv: ["<rootDir>/src/setupTests.tsx"],
};
For nodeJS + typescript, just use global instead of global.self
import crypto from 'crypto'
Object.defineProperty(global, 'crypto', {
value: {
getRandomValues: (arr:any) => crypto.randomBytes(arr.length)
}
});
Since node 15.x you can use crypto.webcrypto
eg.
import crypto from "crypto";
Object.defineProperty(global.self, "crypto", {
value: {
subtle: crypto.webcrypto.subtle,
},
});
The polyfills in the current answers are incomplete, since Crypto.getRandomValues() modifies its argument in-place as well as returning it. You can verify this by running something like const foo = new Int8Array(8); console.log(foo === crypto.getRandomValues(foo)) in your browser console, which will print true.
getRandomValues() also does not accept an Array as its argument, it only accepts integer TypedArrays. Node.js' crypto.randomBytes() function is not appropriate for this polyfill, as it outputs raw bytes, whereas getRandomValues() can accept signed integer arrays with elements up to 32 bits. If you try crypto.getRandomValues(new Int32Array(8)) in your browser, you might see something like [ 304988465, -2059294531, 229644318, 2114525000, -1735257198, -1757724709, -52939542, 486981698 ]. But if you try node -e 'console.log([...require("crypto").randomBytes(8)])' on the command line, you might see [ 155, 124, 189, 86, 25, 44, 167, 159 ]. Clearly these are not equivalent, and your component under test might not behave as expected if tested with the latter.
The latest versions of Node.js solve this problem with the webcrypto module (should be a matter of setting globalThis.crypto = require('crypto').webcrypto). If you're using an older version of Node (v14 or below) you might have better luck using crypto.randomFillSync(), which should be useable as a drop-in replacement for getRandomValues() as it modifies a passed buffer/TypedArray in-place.
In your Jest setup file (can't be set via the globals configuration as it only allows JSON-compatible values):
const { randomFillSync } = require('crypto')
Object.defineProperty(globalThis, 'crypto', {
value: { getRandomValues: randomFillSync },
})
Deriving from AIVeligs answer:
Since I use "node" environment in Jest I had to use
module.exports = {
preset: "ts-jest",
testEnvironment: "node",
globals: {
crypto: {
getRandomValues: (arr) => require("crypto").randomBytes(arr.length),
},
},
};
I'm using vue-jest, and what worked for me is the following configuration in jest.config.js file:
module.exports = {
...
setupFiles: [
'<rootDir>/tests/settings/jest.crypto-setup.js',
],
};
and in jest.crypto-setup.js:
global.crypto = {
getRandomValues: (arr) => require('crypto').randomBytes(arr.length)
};
Adding the getRandomValues function definition directly in module.exports didn't work since the globals object must be json-serializable (as it is specified here: https://jestjs.io/docs/configuration#globals-object).
Add crypto global for your jest environment as if it were in browser.
Your jest.config.js should look like:
const {defaults} = require('jest-config');
module.exports = {
globals: {
...defaults.globals,
crypto: require('crypto')
}
};
Ref: https://jestjs.io/docs/en/configuration#globals-object
I have this problem in Angular 8 with Jest tests for lib that are using uuid generator. In jest test setup i mock this:
Object.defineProperty(global.self, 'crypto', {
value: {
getRandomValues: arr => arr
},
});
The default crypto dependency didn't work for me during testing with Jest.
Instead I used the #peculiar/webcrypto library:
yarn add -D #peculiar/webcrypto
Then in your Jest setup file, just add this:
import { Crypto } from "#peculiar/webcrypto";
window.crypto = new Crypto();
For the ones using jsdom (jest-environment-jsdom) environment with Jest >=28 you should define replacement module as a getter.
//jest.config.js
module.exports = {
testEnvironment: "jsdom",
rootDir: './',
moduleFileExtensions: ['ts', 'js'],
setupFilesAfterEnv: ["<rootDir>/test/setup-env.tsx"],
preset: 'ts-jest',
};
// setup-env.tsx
const { Crypto } = require("#peculiar/webcrypto");
const cryptoModule = new Crypto();
Object.defineProperty(window, 'crypto', {
get(){
return cryptoModule
}
})
I am using #peculiar/webcrypto but other implementations should work also.
dspacejs's answer almost worked for me, except I had the same problem as Mozgor. I got an error saying that window.crypto is readonly. You can use Object.assign instead of directly trying to overwrite it.
Install #peculiar/webcrypto with yarn add -D #peculiar/webcrypto or npm i --save-dev #peculiar/webcrypto
Then add the following to your Jest setup file:
import { Crypto } from "#peculiar/webcrypto";
Object.assign(window, {
crypto: new Crypto(),
})
const crypto = require('crypto');
global.crypto = crypto;
In the default configuration, Jest assumes you are testing a Node.js environment. But when you are getting errors using methods of the window object, you are probably making a web app.
So if you are making a web app, you should use "jsdom" as your "testEnvironment". To do this, insert "testEnvironment": "jsdom", into your Jest configurations.
If you maintain a "jest.config.js" file, then add it like:
module.exports = {
...
"testEnvironment": "jsdom",
...
};
Or if, like me, you keep the Jest configs in "package.json":
{
...,
"jest": {
...,
"testEnvironment": "jsdom",
...
},
...
}
Building upon what others suggested here, I resolved the issue with window.crypto.subtle.digest with the following:
Object.defineProperty(global.self, "crypto", {
value: {
getRandomValues: (arr: any) => crypto.randomBytes(arr.length),
subtle: {
digest: (algorithm: string, data: Uint8Array) => {
return new Promise((resolve, reject) =>
resolve(
createHash(algorithm.toLowerCase().replace("-", ""))
.update(data)
.digest()
)
);
},
},
},
});
Or, if not using Typescript:
Object.defineProperty(global.self, "crypto", {
value: {
getRandomValues: (arr) => crypto.randomBytes(arr.length),
subtle: {
digest: (algorithm, data) => {
return new Promise((resolve, reject) =>
resolve(
createHash(algorithm.toLowerCase().replace("-", ""))
.update(data)
.digest()
)
);
},
},
},
});
The reformating of the string is optional. It is also possible to hardcode the algorithm, e.g. by stating 'sha256' or 'sha512' or alike.
Depency injection is one way to solve this.
Node.js provides an implementation of the standard Web Crypto API. Use require('node:crypto').webcrypto to access this module.
So you pass the crypto object to the code that depends on it.
Notice how we "inject" the correct crypto object when invoking the method utils.aesGcmEncrypt
test("Encrypt and decrypt text using password", async () => {
const password = "Elvis is alive";
const secret =
"surprise action festival assume omit copper title fit tower will chalk bird";
const crypto = require("crypto").webcrypto;
const encrypted = await utils.aesGcmEncrypt(crypto, secret, password);
const decrypted = await utils.aesGcmDecrypt(crypto, encrypted, password);
expect(decrypted).toBe(secret);
});
late to the party, but I usually do something like:
// module imports here
// important: the following mock should be placed in the global scope
jest.mock('crypto', function () {
return {
randomBytes: jest
.fn()
.mockImplementation(
() =>
'bla bla bla'
),
}
});
describe('My API endpoint', () => {
it('should work', async () => {
const spy = jest.spyOn(DB.prototype, 'method_name_here');
// prepare app and call endpoint here
expect(spy).toBeCalledWith({ password: 'bla bla bla' });
});
});