I am building some custom plugins for a product that uses proprietary npm module to setup connection and perform an operation(read, write and submit) on mainframe emulator.
I am setting up connection in index.js file and want to pass terminal variable to separate module while calling it
index.js snippet
var terminal;
const mainframeTerminal =require(‘private_module’);
const accountDetailsModule =require('./src/accountDetails');
terminal = private_module.connect('11.11.11.1:789');
let screen = await terminal.status();
// expose module from index.js file so that it can be consumes in product
export.getAccountDetails = accountDetailsModule.getAccountDetails(terminal)
accountDetails.js
module.exports.getAccountDetails = async function(terminal){
//perform some operation with termianl var - passed from index file
return data;
}
I am getting below error
exports.getAccountDetails = accountDetailsModule.getAccountDetails is not a function.
I also need to pass data input but for the time being it's not required,
Would like to know how node.js function will understand mapping if I just need to pass anyone of input only.
Please throw some inputs, I am new to coding.
like this
index.js
const mainframeTerminal =require('private_module');
const accountDetailsModule = require('./src/accountDetails');
const terminal = private_module.connect('11.11.11.1:789');
let screen = await terminal.status();
// expose module from index.js
exports = accountDetailsModule(terminal);
accountDetails.js
const getAccountDetails = async (terminal) => {
// perform some operation
return data;
}
exports = getAccountDetails;
In CommonJS, modules are just a way to assign variables. So what you get in index.js when you require('./src/accountDetails') is exactly what you export in accountDetails.js (the value of getAccountDetails, which is an async function taking one argument).
runMyCode.js
Here's how you might call your code...
const mainframeTerminal =require('private_module');
const accountDetailsModule = require('./src/accountDetails');
const terminal = private_module.connect('11.11.11.1:789');
let screen = await terminal.status();
// run the code a print the result to console
accountDetailsModule(terminal).then(console.log);
Related
I'm looking for a way to change a text string or a value in one command by typing the new value in a different command. For example I have Discord js v12 and I'm using module commands with each command being in its own .js file.
module.exports = {
name: 'calc',
cooldown: 1000,
run: async(client, message, args) => {
if (!message.member.hasPermission("ADMINISTRATOR")) return await message.delete();
await message.delete();
var multiply = args[0] * (100 - percalc) / 100;
var calculation = multiply.toFixed(2);
if(!args[0]) return await message.channel.send('Specify a Value');
await message.channel.send(changableValue);
await message.channel.send(calculation < 5 ? 5 : calculation);
}
and I have the consts in the config file like
const percalc = 50;
const changableValue = 'Text example';
Right now the command _calc {number} puts out a calculation in percentage based on the percalc const and a text that comes with it in the changableValue part.
I'd like to make a command let's say _calcset {Value} that will save the provided value and will send it in place of the changableValue const.
First of all, note that the only reason the keyword const exists is because it stands for constant and constant variables can never be changed once initialized. So make sure you change the variable declaration keyword to just var or let.
Method 1 - If you don't need data to persist
Now, if you only want the variable to be changed per session, and you're fine with it going back to what you defined it as when you shut down the bot, you can just update the variable using functions exported from the js file. But to get the dynamic variable you'll also need to use a getter function that you export as well. Example:
config.js
var changeableValue = "foo";
function getValue() {
return changeableValue;
}
function updateValue(newValue) {
changeableValue = newValue;
}
module.exports = {
getValue,
updateValue
}
command.js
const { getValue, updateValue } = require("config.js");
console.log(getValue()); // logs "foo"
updateValue("bar");
console.log(getValue()); // logs "bar"
Unfortunately, as I mentioned, the changeableValue var will be reset back to "foo" every time you shut off the bot. If that's okay with you, then the above works fine.
Method 2 - If you need data to persist through sessions
If you want to persist the changeableValue variable through sessions, then it gets a little more complicated. Your two most likely options are either to write the value to a JSON file using the fs module (so it will be saved to your disk), or save the value in some other database, like MongoDB. I would recommend using another database provider because there are more problems you can run into when writing to your own disk, for example, if you make two write requests at the same time (like if two users use the command at the same time), you can corrupt the file when the requests try to write at the same time. However setting up an external database is outside of the scope of this question, so here's how you would set up writing to a JSON file:
config.json
{
"changeableValue": "foo"
}
command.js
const fs = require("fs");
var { changeableValue } = require("config.json");
console.log(changeableValue) // logs "foo"
var updatedValueJSON = JSON.stringify({ changeableValue: "bar" }); // necessary because the JSON object must be converted to a string
fs.writeFile("config.json", updatedValueJSON, "utf8", () => {
// this is the callback function called when the file write completes
let { changeableValue } = require("config.json");
console.log(changeableValue); // logs "bar", and now if you restart the bot and import changeableValue, it will still be bar
});
I'm adapting Jest to an old JS which consists of several js files and is built via concatenation (and is supposed to be run in browser). Some files use "globals" defined in others. To make Jest work, I've tested the Rewire package and got it working for no-dependency case like this:
// experiment.js
function sum(a, b) {
return a + b
}
const myVar = 'some'
// experiment.test.js
const rewire = require('rewire')
const experimentRewired = rewire('./experiment.js')
const $sum = experimentRewired.__get__('sum')
const $myVar = experimentRewired.__get__('myVar')
test('sum gives 1 + 2 = 3', () => {
expect($sum(1, 2)).toBe(3)
})
// not a real test, just shows that variables are loaded too
test('myVar is equal to string "some"', () => {
expect($myVar).toBe("some")
})
This is a simple case when everything to be tested is inside a single file. What I'm trying to test now is issentially injecting a dependency from another file:
// globals.js – config should be injected into experiment.js
const config = {}
// experiment.js
config.name = 'value'
// experiment.test.js
const rewire = require('rewire')
const globalsRewired = rewire('./globals.js')
const experimentRewired = rewire('./experiment.js')
const $config = globalsRewired.__get__('config')
experimentRewired.__set__('config', $config) // trying to inject
const $$config = experimentRewired.__get__('config')
test('config has .name', () => {
expect($$config.name).toBeTruthy()
})
but this results in ReferenceError: config is not defined. I'm guessing that what happens is – experiment.js is called first and "injecting" would happen after it (if it were not the error before it).
Is there a way to "inject" the dependency or may be virtually concatenate the files without actually saving a new js file (which will be the last resort and is not desirable due to performance considerations)? Suggestions both with and without Rewire are welcome.
I am trying to open a puppeteer browser upon startup and then exporting the wsEndpoint so that I may use the link to connect to the browser rather than opening a new browser every time I call the function.
Here is the code snippet in the file app.js that is the entry point for node.
const mainFunction = async () => {
const browser = await puppeteer.launch()
const wsEndpoint = browser.wsEndpoint()
return wsEndpoint
}
mainFunction().then(async endpoint => {
console.log(endpoint)
module.exports = endpoint
})
upon startup, the console log above returns a link that I then export
And here is the code snippet in the utility file equities.js
const puppeteer = require("puppeteer")
const endpoint = require("../../app.js")
module.exports = async(symbol)=>{
console.log(endpoint)
const browser = await puppeteer.connect({connectWSEndpoint: endpoint})
}
Every time I call the function, the console log only returns an empty object meaning that the export in app.js failed for some reason. I tried to google a few things and tried different ways of exporting but none seem to work. Can someone help guide me? Thank you so much in advance.
A few things here seem amiss to me -- this code feels like it wasn't tested along the way, leading to multiple points of failure. Try to take smaller steps so you can isolate problems instead of accumulating them.
For starters, the mainFunction code abandons the browser object, creating a leaked subprocess resource can't be closed.
I'd return or store the browser variable along with the endpoint so someone can clean it up through a function. Or just return the browser and let the client code pull the endpoint out of it if they want, as well as manage and close the resource.
Next, the export code:
mainFunction().then(async endpoint => {
console.log(endpoint)
module.exports = endpoint
})
I don't understand the motivation for this extra then wrapper that receives an async resolution function that never uses await. You may think Node awaits all of this code, then sets the module.exports value before the client file's require runs synchronously. That's not the case, which can be determined with a simpler piece of code:
app.js (in the same folder throughout this post for convenience):
const mainFunction = async () => 42;
mainFunction().then(async endpoint => {
console.log("endpoint":, endpoint)
module.exports = endpoint
})
index.js:
const endpoint = require("./app");
console.log("imported:", endpoint);
node index gives me:
imported: {}
endpoint: 42
The promise resolved after the require, which synchronously brought in the default blank object module.exports -- probably not what you expected.
If you have async code, it has to stay async forever, including exports and imports. Try exporting the promise directly, then awaiting it in the client:
app.js:
const mainFunction = async () => 42;
module.exports = mainFunction;
index.js:
const getEndpoint = require("./app");
getEndpoint().then(endpoint => console.log("imported:", endpoint));
Running node index gives me: imported: 42.
The client code in equities.js looks more reasonable because it exports a promise synchronously, but it's going to have to await the endpoint promise it imported anywhere it uses it.
Also, Puppeteer throws on puppeteer.connect({connectWSEndpoint: endpoint}), Error: Exactly one of browserWSEndpoint, browserURL or transport must be passed to puppeteer.connect. I'll leave that up to you to work out based on your goals.
Here's a rewrite sketch that fixes the promise problems, but is only a proof of concept which will need tweaks to do whatever you're trying to do:
app.js:
const puppeteer = require("puppeteer");
const browserPromise = puppeteer.launch();
const endpointPromise = browserPromise
.then(browser => browser.wsEndpoint())
;
module.exports = {browserPromise, endpointPromise};
equities.js:
const puppeteer = require("puppeteer");
const {browserPromise, endpointPromise} = require("./app");
module.exports = async symbol => {
const endpoint = await endpointPromise;
console.log(endpoint);
//const browser = await puppeteer.connect({connectWSEndpoint: endpoint}) // FIXME
const browser = await browserPromise;
await browser.close();
};
index.js:
const equitiesFn = require("./equities");
(async () => {
await equitiesFn();
})();
Run node index and you should see the ws printed.
Note that you can wrap the exported promises in functions or as part of an object which is a layer of abstraction more typical for the interface of a library if you want. But this doesn't change the fundamental asynchrony. The client will call the exported functions and await the endpoint and/or browser promises through that extra layer of indirection,
require("./app").getBrowser().then(browser => /* */);
versus
require("./app").browserPromise.then(browser => /* */);
If you don't want to expose the browser object, that's fine, but I'd suggest exposing a function that closes the underlying browser so you can get a clean exit, e.g.
app.js:
const puppeteer = require("puppeteer");
const browserPromise = puppeteer.launch();
const closeBrowser = () =>
browserPromise.then(browser => browser.close())
;
module.exports = {closeBrowser};
index.js:
require("./app")
.closeBrowser()
.then(() => console.log("closed"))
;
I have a node.js controller that create a csv file and it is stored in a specific folder, there is also a value from a var called endpoint.
The thing is that in the end of the lines of the controller i have this start function to start another process in another javascript file called bot.js
const scrapeWebsite = require('../bot')
const start = new scrapeWebsite();
var endpoint ="192.186.1.1"
try {
start.init();
} catch (e) {
res.send(JSON.stringify(e, null, 5))
}
In the bot.js file i want to access the value of the endpoint variable of my controller.js to use it in the bot function process and i dont know how to do this.
like declare a new var in the bot.js with the same value as the endpoint in the controller.js
class scrapeWebsite {
init() {
this.startScrape()
}
async startScrape() {...}
Export that function from bot.js
module.exports.function_name = function_name
And when you want to run it in node.js controller, import it using
const {function_name} = require("../bot")
And when you want to execute that function, call that function
function_name(<with, arguments, from, bot.js, file>)
I need to create an API that web scraps GitHub's repos getting the following data:
File name;
File extension;
File size (bytes, kbytes, mbytes, etc);
File number of lines;
I'm using Node with TypeScript so, to get the most out of it, I decided to create an interface called FileInterface, that has the four attributes mentioned above.
And of course, the variable is an array of that interface:
let files: FileInterface[] = [];
Let's take my own repo to use as an example: https://github.com/raphaelalvarenga/git-hub-web-scraping
So far so good.
I'm already pointing to the HTML's files section with request-promise dependency and storing them in a Cheerio variable so I can traverse through the "tr" tags to create a loop. As you might think, those "tr" tags represent each files/folders inside of a "table" tag (if you inspect the page, it can easily be found). The loop will fill a temp variable called:
let tempFile: FileInterface;
And at the end of every cycle of the loop, the array will be populated:
files.push(tempFile);
In GitHub repo's initial page, we can find the file names and their extension. But the size and total of lines, we can't. They are found when clicking on them to redirect to the file page. Let's say we clicked in README.md:
Ok, now we can see README.md has 2 lines and 91 Bytes.
My problem is, since this will take a long time, it needs to be an async function. But I can't handle the loop in Cheerio content inside the async function.
Things that I've tried:
Using map and each methods to loop through it and push in the array files;
Using await before the loop. I knew this one wouldn't actually work since it's just a loop that doesn't return anything;
The last thing I tried and believed that would work is Promise. But TypeScript accuses Promises return the "Promise unknown" type and I'm not allowed to populate the result in files arrays, since the types "unknown" and "FilesInterface[]" are not equal.
Below I'll put the code I created so far. I'll upload the repo in case you want to download and test (the link is at the beginning of this post), but I need to warn that this code is in the branch "repo-request-bad-loop". It's not in the master. Don't forget because the master branch doesn't have any of this that I mentioned =)
I'm making a request in Insomnia to the route "/" and passing this object:
{
"action": "getRepoData",
"url": "https://github.com/raphaelalvarenga/git-hub-web-scraping"
}
index-controller.ts file:
As you can see, it calls the getRowData file, the problematic one. And here it is.
getRowData.ts file:
I will try to help you, although I do not know typescript. I redid the getRowData function a bit and now it works for me:
import cheerio from "cheerio";
import FileInterface from "../interfaces/file-interface";
import getFileRemainingData from "../routines/getFileRemaningData";
const getRowData = async (html: string): Promise<FileInterface[]> => {
const $ = cheerio.load(html);
const promises: any[] = $('.files .js-navigation-item').map(async (i: number, item: CheerioElement) => {
const tempFile: FileInterface = {name: "", extension: "", size: "", totalLines: ""};
const svgClasses = $(item).find(".icon > svg").attr("class");
const isFile = svgClasses?.split(" ")[1] === "octicon-file";
if (isFile) {
// Get the file name
const content: Cheerio = $(item).find("td.content a");
tempFile.name = content.text();
// Get the extension. In case the name is such as ".gitignore", the whole name will be considered
const [filename, extension] = tempFile.name.split(".");
tempFile.extension = filename === "" ? tempFile.name : extension;
// Get the total lines and the size. A new request to the file screen will be needed
const relativeLink = content.attr("href")
const FILEURL = `https://github.com${relativeLink}`;
const fileRemainingData: {totalLines: string, size: string} = await getFileRemainingData(FILEURL, tempFile);
tempFile.totalLines = fileRemainingData.totalLines;
tempFile.size = fileRemainingData.size;
} else {
// is not file
}
return tempFile;
}).get();
const files: FileInterface[] = await Promise.all(promises);
return files;
}
export default getRowData;