I am new to Nodejs
I have written a function that reads contents from a file and stores into a variable as an array. And finally I am mutating the variable and writing it to a file. See below:
function(file, item) {
return fs.readLine(file).then(function(contents) {
var data = contents.split(/\n/);
data.splice(data.indexOf(item), 1);
return fs.writeFile(file, data.join(/\n/));
});
}
Is there a way to do the same without mutating the variable or even having had to store the contents of a file into a variable and delete it in node js?
Thanks
If you don't want to mutate the variable with data.splice() then you can use data.slice() that doesn't mutate the original array. See the docs:
https://developer.mozilla.org/en/docs/Web/JavaScript/Reference/Global_Objects/Array/slice
Also instead of reading the contents of the file into a variable, you can create a readable stream and filter out the line that you don't want.
See how I created my filt module that filters lines of standard input:
https://www.npmjs.com/package/filt
The source code and a lot of examples are on GitHub:
https://github.com/rsp/node-filt
Basically what you can do here is - using the handy split module - something like this:
fs.createReadStream(file).pipe(split()).on('data', (line) => {
// if line is something ... etc.
});
Related
Working in an AI/ML project, I need a way to run the pickle file inside nodejs so I can use it to run the algorithm on the data they submitted.
Try to use the node-pickle library to convert the pickle file to the JSON object. Here documentation of node-pickle
const nodePickle = require('node-pickle');
// Convert pickled object to JSON object
nodePickle.load(pickledData)
.then(data => ({
// data is a JSON object here
})
Then you can use tensorflow.js to run that JSON object as a model.
Tenrsorflow.js Documentation
The only one I could find from a quick google search is node-jpickle. According to it's docs, it seems to be able to handle most of the situations pickle can, even some of the more advanced ones such as classes:
function MyClass() {}
var jpickle = require('jpickle');
jpickle.emulated['__main__.MyClass'] = MyClass;
var unpickled = jpickle.loads(pickled);
If you're just trying to pickle/unpickle, you can just do it like you would in Python:
var unpickled = jpickle.loads(pickled);
The docs don't state anything about a normal load function however.
Basically what I need to do is to get a config file from a csv file and load it as an array into a global variable each time I am running tests in before function.
Lets say mynightwatch.conf.js contains following, example global variable myGlobalArr
"globals": {
myGlobalArr : []
}
And in globals.js I am using readFileSync to read a file and get it's text as an array to a variable
before: function(done) {
var file = fs.readFileSync('path').toString().split('\n')
}
How can I pass an array which is stored in file to myGlobalArr and then call it from inside a Page Object to use it?
Thanks
I have 3 JSON files. Lets call them file1.json , file2.json and file3.json.
They all look very similar and are in this structure:
"orgItems":[...]
There is a top level orgItems property. Now what I'm struggling with is that these are 3 large files. Almost 20-30mb each. I want to concatenate all of these into 1 file.
Whats the best way to do this ? To be able to grab the orgItems object out of each of the 3 files and then have it all in one object in one file. So I just want one file combinedResponses.json and that should have
combinedResponses.json
"orgItems":[...file1.orgItems,...file2.orgItems,...file3.orgItems]
Take a look at JSONStream. It provides an easy way to stream json files (so you don't have to load it all into memory at once) and receive events for matching paths. You could pipe the input stream to an output stream that writes the concatenated file.
This is a bit outside my wheelhouse and I'm sure it could be improved, but I think something like this does what you want. You could tweak the JSONStream.parse argument to get only the objects you care about.
const JSONStream = require('JSONStream');
const fs = require('fs');
function appendData (sourceFile, outputStream) {
const file = fs.createReadStream(sourceFile);
const jsonStream = JSONStream.parse('*');
jsonStream.on('data', data => outputStream.write(JSON.stringify(data)));
file.pipe(jsonStream);
file.on('end', () => {
file.close();
});
}
const out = fs.createWriteStream('data/concatted.json', {autoClose: true});
appendData('data/data-1.json', out);
appendData('data/data-2.json', out);
appendData('data/data-3.json', out);
I use a JSON file for common phrases so I don't have to type them and maybe in the future they can be translated. So for example in my main code I want to say You don't have the permission to use ${command_name}. This works perfectly fine hardcoded into my .js file but ultimately I want this to be in a JSON file, which does not allow any variables to be inserted.
Does anyone know a solution to my problem?
EDIT: Thanks for the suggestions. I guess string.replace would be my best option here. Wish there was some built in feature that'd convert variables in a JSON string to variables declared in that JS file.
You cannot treat template string literals in JSON files like in Javascript "code". You said it yourself. But: You could use a template engine for this - or just simple String.replace().
Example for a template engine: https://github.com/janl/mustache.js
With Mustache (as an example) your code will look like this
var trans = {
command_name: "dump"
};
var output = Mustache.render("You don't have the permission to use {{command_name}}", trans);
With simple String.replace():
var str = "You don't have the permission to use %command_name%";
console.log(str.replace('%command_name%', 'dump'));
You can simply use placeholders. The following function replaces the placeholders with user-defined values:
const messages = {
msgName: 'Foo is :foo: and bar is :bar:!'
}
function _(key, placeholders) {
return messages[key].replace(/:(\w+):/g, function(__, item) {
return placeholders[item] || item;
});
}
Usage:
_('msgName', { foo: 'one', bar: 'two' })
// "Foo is one and bar is two!"
It's just an example. You can change the placeholders style and the function behavior the way you want!
You can use config npm module and separate your JSON files according to your environment.
./name.json
{
command: "this is the output of 'command'"
}
./Node.js
cost names = require('./name.json');
console.log('name === ', name.command);
// name === this is the output of 'command'
So the main challenge is getting separated file with string constants when some of them being parametrizable, right?
JSON format itself operates on strings(numbers, booleans, lists and hashmap) and knows nothing about substitution and parameters.
You are also unable to use template strings like you don't have permission to do ${actionName} since template strings are interpolated immediately.
So what can you do?
Writing your own parser that takes config data from JSON file, parse a string, find a reference to variable and substitute it with value. Simple example:
const varPattern = /\${([^{}]+)}/g;
function replaceVarWithValue(templateStr, params) {
return templateStr.replace(varPattern, (fullMatch, varName) => params[varName] || fullMatch);
}
or you can use any npm package aimed on localization like i18n so it would handle templates for you
Basically you can implement a function parse which, given a text and a dictionary, it could replace any ocurrence of each dictionary key:
const parse = (template, textMap) => {
let output = template
for (let [id, text] of Object.entries(textMap)) {
output = output.replace(new RegExp(`\\$\{${id}}`, 'mg'), text)
}
return output
}
const textMap = {
commandName: 'grep',
foo: 'hello',
bar: 'world'
}
const parsed = parse('command "${commandName}" said "${foo} ${bar}"', textMap)
console.log(parsed)
BTW, I would suggest you that you should use some existing string templating engine like string-template to avoid reinventing the wheel.
So I have 2 files a mapgen.js and a main.js. In mapgen.js there is a function that generates a giant 2d array. I want to use this aray in main.js but don't want the function that generates the map to run everytime its 'required' in main.js. I also want to be able to edit the map array eventually.
Example: (not real code just wrote some crap to show what the issue is)
mapgen.js:
var map;
function mapGen(){
//make the map here
this function takes like 2 seconds and some decent CPU power, so
don't want it to ever run more than once per server launch
map = map contents!
}
main.js
var map = require mapgen.js;
console.log(map.map);
//start using map variable defined earlier, but want to use it without
having to the run the big funciton again, since it's already defined.
I know i have to module.exports somewhere but I dont think that will solve my problem still. I would write it to a file but is that not much slower to read and edit than keeping it in the ram? Previously I had gotten past this by keeping everything in 1 file but now I need to clean it all up.
Requiring the module won't automatically invoke the function. You can do that in the main.js file.
mapgen.js
module.exports = function mapGen() {
return [/* hundreds of items here. */];
};
main.js
// Require the module that constructs the array.
const mapGen = require('./mapgen');
// Construct the array by invoking the mapGen function and
// store a reference to it in 'map'.
const map = mapGen(); // `map` is now a reference to the returned array.
// Do whatever you want with 'map'.
console.log(map[0]); // Logs the first element.
I'm not an expert but if you put one condition in mapgen.js that don't work ?
var map;
function mapGen(){
if(!map){
//your code here
map = map contents!
}
}
Combine that with global variable and/or module.exports See
How to use global variable in node.js?