I'm trying to send data (as an object) to a child process in Node.js, however, all of my regular expressions get lost in transfer.
var arguments = {
something: {
name: 'test',
age: 28,
active; true
},
otherThing: 'some string',
regex: /test/i,
regex2: new RegExp('test')
};
var child = cp.fork(path.join(__dirname, 'child.js'));
child.on('message', function (data) {
console.log(data);
});
child.send(arguments);
In the child.js file I have this at the top:
process.on('message', function () {
console.log(arguments); // This is where the data has changed
});
When the log is output from the child process the arguments object instead looks like this:
{
something: {
name: 'test',
age: 28,
active: true
},
otherThing: 'some string',
regex: {},
regex2: {}
}
So far unable to find anything elsewhere about why this may be happening, any ideas?
Because they are completely separate JavaScript processes, you can't send objects. When you pass an object, it gets serialized to JSON and parsed by the child. (See the docs.)
JSON does not support serializing regex objects. (Try putting JSON.stringify(/abc/) through your console -- you get back "{}".)
To include regexes in a JSON object, you can use the json-fn module. It supports serializing functions, dates, and regexes. (It was actually thanks to an issue i raised that they added regex support. :))
You could then do something like:
var JSONfn = require('json-fn');
var arguments = {
something: {
name: 'test',
age: 28,
active; true
},
otherThing: 'some string',
regex: /test/i,
regex2: new RegExp('test')
};
var child = cp.fork(path.join(__dirname, 'child.js'));
});
child.send(JSONfn.stringify(arguments));
and:
var JSONfn = require('json-fn');
process.on('message', function (data) {
console.log(JSONfn.parse(data))); // This is where the data has changed
});
You can store the regex as a string like
myRegex.string = "/test/";
myRegex.modif = "i";
Send it to child and then use it like
new RegExp(myRegex.string, myRegex.modif);
I tried json-fn but Date objects stringified are not reverted back to Date. This module JSON4Process stringifies the objects' properties of type Date, RegExp, Function, Set and Map while maintaining the object as a javascript object. You don't need to stringify the whole object if you're using fork, you can directly send it.
const { fork } = require('child_process');
const JSON4Process = require('json4process');
let obj = {
date: new Date(),
regex: new RegExp(/regex/g),
func: () => console.log('func')
}
obj = JSON4Process.stringifyProps(obj);
const child = fork('child.js');
child.send(obj);
And then parse the properties back in the other file:
process.on('message', data => {
let obj = JSON4Process.parseProps(data);
});
In case you need to use spawn or exec you can just use the default JSON.stringify over the modified object with json4process:
let newObj = JSON.stringify(JSON4Process.stringifyProps(obj));
let originalObj = JSON4Process.parseProps(JSON.parse(newObj));
Related
I receive JSON data from the service, but the keys change in the data with each request, below I will give an example in three versions.
Exmaple 1:
{
"trackingPayloads": {
"Rltyn4gLRIWRKj9kS0YpWXytG81GZwcPWjEE7f31ALlq": "{"title":"Red Shoes","index":3,"id":"17777","type":"category"}',
"ywtA6OyM0hzVZZvnUjxoxJDI1Er9ArfNr8XKyi1D5Zzk": "{"title":"White Shoes","index":3,"id":"17777","type":"category"}',
}
}
Example 2:
{
"trackingPayloads": {
"36tW7DqZ3H9KKBEAumZmowmUwmDRmVCjQgv5zi9GM3Kz": "{"title":"Red Shoes","index":3,"id":"17777","type":"category"}',
"OgtE51n3YtvrVXWLFjPmpnRt2k5DExF7ovxmBTZrZ6wV": "{"title":"White Shoes","index":3,"id":"17777","type":"category"}',
}
}
Example 3:
{
"trackingPayloads": {
"k2toY29glt2JEp9Wi1X5M7ocno0E0mS4JQVyDuGyQ2rM": "{"title":"Red Shoes","index":3,"id":"17777","type":"category"}'",
"5ef2ec3c3573eebecc9690b69619ec7b9c93b609": "{"title":"White Shoes","index":3,"id":"17777","type":"category"}',
}
}
As you can see, the data included in the keys does not change since I am requesting the same information, but the key will change with each request.
Please help, what are the options to get the data Title, Index and any other content in these keys using node js?
Only one option came to my mind - to rename the keys upon receipt in 1,2,3 ... and then get data from them, but this needs to be done dynamically, since about 120 requests per minute are made, you need to get this data quickly, there are no options to save it to a file (I didn’t understand how)
UPDATE, added my code.
I am attaching an example of my code, the idea is to eventually get the data I need from the right keys from trackingPayloads, please help with the code <3
const AwaitAPIResponse = await ProductAPI(product_sku);
const $ = cheerio.load(AwaitAPIResponse);
const JSONDATA = [];
$('pre').each(function() {
JSONDATA.push($(this).text());
});
const ProductJson = JSON.parse(JSONDATA[0]) // this is where I get all the data
const MainJson = ProductJson["trackingPayloads"] // here I go to the trackingPayloads you saw above
How can I get the data I need?
You can use Object.keys() to get all the different keys of an object and use a loop to go through them.
Therefore, you can rework this code in such a way that each of the values is stored as an element in an array, maybe makes the data easier to work with:
const convert = object => {
const ret = []
for (const key of Object.keys(object)) {
ret.push(object[key])
}
return ret
}
This will give you following result for your use case:
[{"title":"Red Shoes","index":3,"id":"17777","type":"category"},
{"title":"Red Shoes","index":3,"id":"17777","type":"category"}]
The way you'd call this is as follows:
const some_parsed_json = {
"k2toY29glt2JEp9Wi1X5M7ocno0E0mS4JQVyDuGyQ2rM": {
title:"Red Shoes",
index:3,
id:"17777",
type:"category"
},
"5ef2ec3c3573eebecc9690b69619ec7b9c93b609": {
title:"Red Shoes",
index:3,
id:"17777",
type:"category"
}
}
const json_object_values = convertor(some_parsed_json)
If you don't car about the key you could use Object.values on the received object to get the values
Object.values(payload)
// With your example it will return:
// [{"title":"Red Shoes","index":3,"id":"17777","type":"category"},
// {"title":"Red Shoes","index":3,"id":"17777","type":"category"}]
or in a more complete example
async function getParsedValues() {
const responseString = await service.getData(); // '{"trackingPayloads":{"Rltyn4gLRIWRKj9kS0YpWXytG81GZwcPWjEE7f31ALlq":{"title":"Red Shoes","index":3,"id":"17777","type":"category"},"ywtA6OyM0hzVZZvnUjxoxJDI1Er9ArfNr8XKyi1D5Zzk":{"title":"White Shoes","index":3,"id":"17777","type":"category"}}}'
const parsedResponse = JSON.parse(responseString); // { trackingPayloads: { Rltyn4gLRIWRKj9kS0YpWXytG81GZwcPWjEE7f31ALlq: { title:'RedShoes', index: 3, id: '17777', type: 'category' }, ywtA6OyM0hzVZZvnUjxoxJDI1Er9ArfNr8XKyi1D5Zzk:{title:'WhiteShoes', index: 3, id: '17777', type: 'category' } }}
const values = Object.values(parsedResponse); // [{"title":"Red Shoes","index":3,"id":"17777","type":"category"}, {title:'WhiteShoes', index: 3, id: '17777', type: 'category' }]
return values;
}
I am building a dictionary but I would like some of the values to contain variables. is there a way to pass a variable to the dictionary so I can assign a dot notation variable? the variables object will always have the same structure and the dictionary will be static and structured the same for each key value pair. essentially I want to pass the value from the dictionary to another function to handle the data.
main.js
import myDictionary from "myDictionary.js"
const variables ={
item:"Hello"
}
const data = myDictionary[key](variables)
console.log(data)
myDictionary.js
const myDictionary = {
key: variables.item
}
so the log should display hello. I know it willl be something straightforward but cant seem to figure it out.
as always any help is greatly appreciated
You should modify the dictionary so that it keeps actual callback functions instead. Only then it will be able to accept arguments.
const myDictionary = {
key: (variables) => variables.item
}
const variables = {
item: "Hello"
}
const key = "key";
const data = myDictionary[key](variables)
console.log(data)
What you are trying to do is not possible. The myDictionary.js file has no idea whats inside you main file. The only thing you could do would be:
myDictionary.js
const myDictionary = {
key: "item"
}
main.js
import myDictionary from "myDictionary.js";
const variables = {
item: "Hello"
};
const data = variables[myDictionary["key"]];
console.log(data);
Also, even though JavaScript does not enforce semi-colons, they will save you a lot of headaches of some stupid rule that breaks the automatic inserter.
I must apologise as when I asked the question I wasn't fully clear on what I needed but after some experimentation and looking at my edge cases and after looking at Krzysztof's answer I had a thought and came up with something similar to this -
const dict = {
key: (eventData) => {
return [
{
module: 'company',
entity: 'placement',
variables: {
placement_id: {
value: eventData.id,
},
},
},
{
module: 'company',
entity: 'placement',
variables: {
client_id: {
value: eventData.client.id,
},
},
},
];
},
}
Then I'm getting the data like this -
const data = dict?.[key](eventData)
console.log(data)
I can then navigate or manipulate the data however I need.
thank you everyone who spent time to help me
I have a function that is using eval to convert a string with an expression to an object based on the parameter.
let indexType = ["Mac", "User", "Line", "Mask", "Ip", "Location"]
const filterIndex = (item) => {
filteredIndexSearch = []
eval(`search${item}`).forEach((e) => filteredIndexSearch.push(searchData[e.key]))
}
filterIndex(indexType[searchTotal.indexOf(Math.max(...searchTotal))])
searchData is an array that returns values based on the user input.
searchTotal is an array with the length of each search{item} array.
The filterIndex function takes the highest value from the searchData array and corresponds it to the indexType array, then use eval to convert the string to an object to pass the value to the filteredIndexSearch array.
What would be a better alternative to eval?
EDIT
To add more information on what this does:
searchData = [
[
{
key: 1,
data: "0123456789101"
},
{
key: 1,
data: "John Smith"
}
],
[
{
key: 2,
data: "0123456789102"
},
{
key: 2,
data: "Jane Smith"
},
]
]
const search = (data, key, container) => {
if (!data) data = "";
if (data.toLowerCase().includes(string)) {
container = container[container.length] = {
key: key,
data: data
}
}
}
const returnSearch = () => {
for (let i = 0; i < searchData.length; i++) {
search(searchData[i][0].data, searchData[i][0].key, searchMac)
search(searchData[i][1].data, searchData[i][1].key, searchUser)
}
}
returnSearch()
The data is incomplete, but hopefully conveys what I'm trying to do.
search will take the user input, and store the information in the corresponding array. If I input "Jo", it will return the searchUser array with only the "John Smith" value and all the other values with the same key. Inputting "102" returns the searchMac with the "0123456789102" value and all other values with the same key.
At the end of the day. I just want to convert search${parameter} to an object without using eval.
Move your global arrays into an object.
Somewhere it appears that you're defining the arrays, something like:
searchMac = [...];
searchUser = [...];
...
Instead of defining them as individual arrays, I'd define them as properties in an object:
searchIndices.Mac = [...];
searchIndices.User = [...];
...
Then, instead of using eval, your can replace your eval().forEach with searchIndices[item].forEach.
If the order of your search isn't important, your can instead loop through the keys of searchIndices:
Object.keys(searchIndices).forEach(item => {
searchIndices[item].forEach(...);
});
This ensures that if you ever add or drop an entry in searchIndices, you won't miss it or accidentally error out on an undefined search index.
Any time you have a situation with variables named x0, x1 etc, that should be a red flag to tell you you should be using an array instead. Variable names should never be semantically meaningful - that is code should never rely on the name of a variable to determine how the code behaves. Convert search0 etc into an array of search terms. Then use:
const filterIndex = (item) => search[item].map(i => searchData[i.key]);
filteredIndexSearch = filterIndex(indexType[searchTotal.indexOf(Math.max(...searchTotal))]);
(simplifying your code). Note that in your code, filteredIndexSearch is modified inside the arrow function. Better to have it return the result as above.
const Data = {
teachers: {
size: 15,
num: 4,
color: 'blue'
}
}
const person = {
name: 'John',
group: Data.teachers, // This is Default. If I do this, the entire teachers tree gets cached. Later, teachers tree changes, this is unusable.
group: () => Data.teachers // I want to do this, so that it fetches fresh data. But, see below for the LJSON problem...
}
I understand that JSON.stringify does not support functions.
I've looked into LJSON (https://github.com/MaiaVictor/LJSON). LJSON does not fix the problem. They also cache Data.teachers, as is. When using LJSON, if Data.teachers changes, then calling group() will not get the latest. It will return the old tree.
(Current stack is React-Native, with AsyncStorage, using JSON.stringify/parse.)
Have you tried the toString() method of a Function & eval?
const fn = (arg) => {
console.log('function:', arg)
}
fn('yes, it works')
const jsonFn = JSON.stringify(fn.toString()) // now its a JSON.stringified string
const newJsonFn = JSON.parse(jsonFn) // parsing it back from JSON to string
const newFn = eval(newJsonFn) // eval!!!
newFn('yes, it works again')
IMPORTANT
Yes, this snippet solves your problem. But don't forget that eval is a dangerous tool: about eval security issues.
What I'd like to do
describe('my object', function() {
it('has these properties', function() {
expect(Object.keys(myObject)).toEqual([
'property1',
'property2',
...
]);
});
});
but of course Object.keys returns an array, which by definition is ordered...I'd prefer to have this test pass regardless of property ordering (which makes sense to me since there is no spec for object key ordering anyway...(at least up to ES5)).
How can I verify my object has all the properties it is supposed to have, while also making sure it isn't missing any properties, without having to worry about listing those properties in the right order?
It's built in now!
describe("jasmine.objectContaining", function() {
var foo;
beforeEach(function() {
foo = {
a: 1,
b: 2,
bar: "baz"
};
});
it("matches objects with the expect key/value pairs", function() {
expect(foo).toEqual(jasmine.objectContaining({
bar: "baz"
}));
expect(foo).not.toEqual(jasmine.objectContaining({
c: 37
}));
});
});
Alternatively, you could use external checks like _.has (which wraps myObject.hasOwnProperty(prop)):
var _ = require('underscore');
describe('my object', function() {
it('has these properties', function() {
var props = [
'property1',
'property2',
...
];
props.forEach(function(prop){
expect(_.has(myObject, prop)).toBeTruthy();
})
});
});
The simplest solution? Sort.
var actual = Object.keys(myObject).sort();
var expected = [
'property1',
'property2',
...
].sort();
expect(actual).toEqual(expected);
it('should contain object keys', () => {
expect(Object.keys(myObject)).toContain('property1');
expect(Object.keys(myObject)).toContain('property2');
expect(Object.keys(myObject)).toContain('...');
});
I ended up here because I was looking for a way to check that an object had a particular subset of properties.
I started with _.has or Object.hasOwnProperties but the output of Expected false to be truthy when it failed wasn't very useful.
Using underscore's intersection gave me a better expected/actual output
var actualProps = Object.keys(myObj); // ["foo", "baz"]
var expectedProps =["foo","bar"];
expect(_.intersection(actualProps, expectedProps)).toEqual(expectedProps);
In which case a failure might look more like
Expected [ 'foo' ] to equal [ 'foo', 'bar' ]
Here are some new possible solutions too:
There's a module for that: https://www.npmjs.com/package/jasmine-object-matchers
With ES2016, you can use a map and convert the object to a map too
I prefer use this; becouse, you have more possibilities to be execute indivual test.
import AuthRoutes from '#/router/auth/Auth.ts';
describe('AuthRoutes', () => {
it('Verify that AuthRoutes be an object', () => {
expect(AuthRoutes instanceof Object).toBe(true);
});
it("Verify that authroutes in key 'comecios' contains expected key", () => {
expect(Object.keys(AuthRoutes.comercios)).toContain("path");
expect(Object.keys(AuthRoutes.comercios)).toContain("component");
expect(Object.keys(AuthRoutes.comercios)).toContain("children");
expect(AuthRoutes.comercios.children instanceof Array).toBe(true);
// Convert the children Array to Object for verify if this contains the spected key
let childrenCommerce = Object.assign({}, AuthRoutes.comercios.children);
expect(Object.keys(childrenCommerce[0])).toContain("path");
expect(Object.keys(childrenCommerce[0])).toContain("name");
expect(Object.keys(childrenCommerce[0])).toContain("component");
expect(Object.keys(childrenCommerce[0])).toContain("meta");
expect(childrenCommerce[0].meta instanceof Object).toBe(true);
expect(Object.keys(childrenCommerce[0].meta)).toContain("Auth");
expect(Object.keys(childrenCommerce[0].meta)).toContain("title");
})
});
I am late to this topic but there is a a method that allows you to check if an object has a property or key/value pair:
expect(myObject).toHaveProperty(key);
expect({"a": 1, "b":2}).toHaveProperty("a");
or
expect(myObject).toHaveProperty(key,value);
expect({"a": 1, "b":2}).toHaveProperty("a", "1");