Develop Reference

Use child array to get count for table

I have some json data. I need to count the number of strings in one of the children then create a table that is grouped by part of the child strings and used the count of specific child strings in the table. Confusing and impossible as it seems, this is what we need.
Honestly, I am barely sure where to even start here. Just displaying the strings correctly was a nightmare.
Here's some example json, I'm using for in loops to get through the levels of json above this, that part works fine:
"DataValues": [
{
"Key": "Stuff Type",
"Id": "95492",
"ComboBoxPairs": [
{
"Value": {
"Key": "3 Gallon",
"Value": "3 Gallon",
"ExtraValues": []
},
"Children": [
{
"Key": "Scan",
"Id": "93478",
"Strings": [
"DogType:Lab,Age:3,Name:Bowser",
"DogType:Lab,Age:5,Name:Dingo",
"DogType:Mutt,Age:1,Name:Muttman",
"DogType:Weiner,Age:1,Name:Goof",
"DogType:Mutt,Age:5,Name:Muttman",
"DogType:Puppy,Age:1,Name:Silly",
"DogType:Puppy,Age:1,Name:Sammy",
"DogType:Puppy,Age:1,Name:Shooter",
"DogType:Puppy,Age:1,Name:Doc",
]
}
]
},
{
"Value": {
"Key": "1 Gallon",
"Value": "1 Gallon",
"ExtraValues": []
},
"Children": [
{
"Key": "Scan",
"Id": "93478",
"Strings": [
"DogType:Puppy,Age:1,Name:Wingo",
"DogType:Puppy,Age:1,Name:Scrappy",
"DogType:Mutt,Age:4,Name:Goober"
]
}
]
}
]
}
]
Here's what I need to build:
DogType ContainerType Quantity Volume
Lab
3 Gallon 2 6 Gallon
Mutt
1 Gallon 1 1 Gallon
3 Gallon 2 6 Gallon
Weiner
3 Gallon 1 3 Gallon
Puppy
1 Gallon 2 6 Gallon
3 Gallon 4 12 Gallon
I am honestly not even sure where to get started
Honestly, I'm not even sure if this is possible? You can see that the table needs to be grouped by part of the string, the DogType. I then need to be able to count how many strings with a certain dog type there are in each ContainerType object, and pass that into the Quantity and ContainerType columns in the table. Then volume is multiplying the Gallon value, which is just text, by the quantity.
I'm sure I need to put more code but I can't even think of how to manage this. There could also be multiple ContainerTypes, this data isn't static.
The data isn't my design but I can't change it, just getting to this point has been a disaster. Can anyone think of a way to do this?

Okay so what you need to do is transform some data. Basically change the way a certain data is stored / represented. Now the Strings are in a consistent format, so we can use a combination of string operations to get a key => value store out of it (I am going to use ES6 syntax to simplify boilerplate)
let parseString = str => {
let pairs = str.split(',');
let obj = {};
pairs.forEach(pair => {
pair = pair.split(':');
obj[pair[0]] = pair[1];
});
return obj;
};
This function simply takes one of your strings, "DogType:Lab,Age:3,Name:Bowser" for example, and splits out an object such as {'DogType': 'Lab', 'Age': 3...}. Now that we have this, we can start to manipulate and group the data itself (if you're not sure what array.map does, it basically gives a new array after running a function on each of the array's values).
let comboBoxPairs = data.ComboBoxPairs.map(comboBoxPair => {
comboBoxPair.Children = comboBoxPair.Children.map(children => parseString(children));
return comboBoxPair;
});
Now we have replaced the incomprehensible array of strings into an array of objects we can start to group. First we need to figure out a data structure. You're grouping by dog type first, so our object's keys are the Dog's Type. Then we simply need to add each quantity into the dog's type and increment a count each time the quantity is encountered.
let dogType = {};
comboBoxPairs.forEach(comboBoxPair => {
comboBoxPair.Children.forEach(children => {
if (typeof dogType[children.DogType] === 'undefined') {
dogType[children.DogType] = {};
}
if (typeof dogType[children.DogType][comboBoxPair.Value.Key] === 'undefined') {
dogType[children.DogType][comboBoxPair.Value.Key] = 0;
}
dogType[children.DogType][comboBoxPair.Value.Key]++;
});
});
And you're done! This gives you a Object such as :
{
'Lab': {
'1 Gallon': 2,
....
},
};
You can then loop through this object to display the values and totals.

Is your table example illustrative or is it meant to represent your sample data?
Your table does not seem to tally with the sample data so I am in doubt as to whether my understanding is correct?
If I can start with a generalised answer - which may not be enough but it may head you in the right direction~
1. If you can parse the data in full before seeking to output your results then you can build a summary array which makes your 'un-ordered' input data trivial - then presentation becomes a separate step.
2. If you have to do it in one pass - then you would normally think about sorting the data first - on the input - this is so you can handle mutt's appearing at non-contiguous points.
Make sense?
My personal preference would be to build your running totals into a separate data structure - whether you have scenario 1 or 2 so that you can trap the increment no matter what level you find your relevant keys at within the for-in.
In short - is there any conceptual problem with building a new object for your summary?

How can I reformat this simple JSON so it doesn't catch "Circular structure to JSON" exception?

Introduction
I'm learning JavaScript on my own and JSON its something along the path. I'm working on a JavaScript WebScraper and I want, for now, load my results in JSON format.
I know I can use data base, server-client stuff, etc to work with data. But I want to take this approach as learning JSON and how to parse/create/format it's my main goal for today.
Explaining variables
As you may have guessed the data stored in the fore mentioned variables comes from an html file. So an example of the content in:
users[] -> "Egypt"
GDP[] -> "<td> $2,971</td>"
Regions[] -> "<td> Egypt </td>"
Align[] -> "<td> Eastern Bloc </td>"
Code
let countries = [];
for(let i = 0; i < users.length; i++)
{
countries.push( {
'country' : [{
'name' : users[i],
'GDP' : GDP[i],
'Region' : regions[i],
'Align' : align[i]
}]})
};
let obj_data = JSON.stringify(countries, null, 2);
fs.writeFileSync('countryballs.json', obj_data);
Code explanation
I have previously loaded into arrays (users, GDP, regionsm align) those store the data (String format) I had extracted from a website.
My idea was to then "dump" it into an object with which the stringify() function format would format it into JSON.
I have tested it without the loop (static data just for testing) and it works.
Type of error
let obj_data = JSON.stringify(countries, null, 2);
^
TypeError: Converting circular structure to JSON
--> starting at object with constructor 'Node'
| property 'children' -> object with constructor 'Array'
| index 0 -> object with constructor 'Node'
--- property 'parent' closes the circle
What I want from this question
I want to know what makes this JSON format "Circular" and how to make this code work for my goals.
Notes
I am working with Node.js and Visual Studio Code
EDIT
This is further explanation for those who were interested and thought it was not a good question.
Test code that works
let countries;
console.log(users.length)
for(let i = 0; i < users.length; i++)
{
countries = {
country : [
{
"name" : 'CountryTest'
}
]
}
};
let obj_data = JSON.stringify(countries, null, 2);
fs.writeFileSync('countryballs.json', obj_data);
});
Notice in comparison to the previous code, right now I am inputing "manually" the name of the country object.
This way absolutely works as you can see below:
Now, if I change 'CountryTest' to into a users[i] where I store country names (Forget about why countries are tagged users, it is out of the scope of this question)
It shows me the previous circular error.
A "Partial Solution" for this was to add +"" which, as I said, partially solved the problem as now there is not "Circular Error"
Example:
for(let i = 0; i < users.length; i++)
{
countries = {
country : [
{
"name" : users[i]+''
}
]
}
};
Resulting in:
Another bug, which I do not know why is that only shows 1 country when there are 32 in the array users[]
This makes me think that the answers provided are not correct so far.
Desired JSON format
{
"countries": {
"country": [
{
"name": "",
"GDP" : "",
"Region" : "",
"Align" : ""
},
{
"name": "",
"GDP" : "",
"Region" : "",
"Align" : ""
},
{
"name": "",
"GDP" : "",
"Region" : "",
"Align" : ""
}
]}
}

Circular structure error occurs when you have a property of the object which is the object itself directly (a -> a) or indirectly (a -> b -> a).
To avoid the error message, tell JSON.stringify what to do when it encounters a circular reference. For example, if you have a person pointing to another person ("parent"), which may (or may not) point to the original person, do the following:
JSON.stringify( that.person, function( key, value) {
if( key == 'parent') { return value.id;}
else {return value;}
})
The second parameter to stringify is a filter function. Here it simply converts the referred object to its ID, but you are free to do whatever you like to break the circular reference.
You can test the above code with the following:
function Person( params) {
this.id = params['id'];
this.name = params['name'];
this.father = null;
this.fingers = [];
// etc.
}
var me = new Person({ id: 1, name: 'Luke'});
var him = new Person( { id:2, name: 'Darth Vader'});
me.father = him;
JSON.stringify(me); // so far so good
him.father = me; // time travel assumed :-)
JSON.stringify(me); // "TypeError: Converting circular structure to JSON"
// But this should do the job:
JSON.stringify(me, function( key, value) {
if(key == 'father') {
return value.id;
} else {
return value;
};
})
The answer is from StackOverflow question,
Stringify (convert to JSON) a JavaScript object with circular reference

From your output, it looks as though users is a list of DOM nodes. Rather than referring to these directly (where there are all sort of possible cyclical structures), if you just want their text, instead of using users directly, try something like
country : [
{
"name" : users[i].textContent // maybe also followed by `.trim()
}
]
Or you could do this up front to your whole list:
const usersText = [...users].map(node => node.textContent)
and then use usersText in place of users as you build your object.
If GDP, regions and align are also references to your HTML, then you might have to do the same with them.

EUREKA!
As some of you have mentioned above, let me tell you it is not a problem of circularity, at first..., in the JSON design. It is an error of the data itself.
When I scraped the data it came in html format i.e <td>whatever</td>, I did not care about that as I could simply take it away later. I was way too focused in having the JSON well formatted and learning.
As #VLAZ and #Scott Sauyezt mentioned above, it could be that some of the data, if it is not well formatted into string, it might be referring to itself somehow as so I started to work on that.
Lets have a look at this assumption...
To extract the data I used the cheerio.js which gives you a kind of jquery thing to parse html.
To extract the name of the country I used:
nullTest = ($('table').eq(2).find('tr').eq(i).find('td').find('a').last());
//"Partial solution" for the OutOfIndex nulls
if (nullTest != null)
{
users.push(nullTest);
}
(nullTest helps me avoid nulls, I will implement some RegEx when everything works to polish the code a bit)
This "query" would output me something like:
whatEverIsInHereIfThereIsAny
or else.
to get rid off this html thing just add .html() at the end of the "jquery" such as:
($('table').eq(2).find('tr').eq(i).find('td').find('a').last().html());
That way you are now working with String and avoiding any error and thus solves this question.

Can Repeater Model execute JAVASCRIPT?

I have an XmlListModel in QML
XmlListModel {
id: model
source: "qrc:/Config/myConfig.xml"
query: "/levels/level"
XmlRole { name: "levName"; query: "#levName/string()" }
XmlRole { name: "from"; query: "from/number()" }
XmlRole { name: "to"; query: "to/number()" }
}
that reads this XML file
<levels parId = "3">
<level levelName = "level1">
<from>0</from>
<to>1</to
</level>
<level levelName = "level2">
<from>1</from>
<to>2</to>
</level>
</levels>
I also have a text element:
Text {
id: myText
x: 0; y:0
text: ""
}
I need to iterate through the XmlListModel in order to assign to myText.text the right level on the basis of what I found in myList.get(3).value, where myList is a ListModel.
Example:
if myList.get(3).value is between 0 (included) and 1 (excluded) I have to set myText.text = "level1", if it is between 1 (included) and 2 (excluded) I have to set myText.text = "level2", and so on...
Any suggestion?

Unfortunately you can't query your XmlListModel in O(1) like give me the value, where x is between role from and role to.
Good for you, you have an ordered list, so you can perform a binary search on your XmlListModel. The algorithm basically goes like this:
You first check whether your search value is by coincidence the one in the middle. If it is smaller, you search in the middle of the lower half, if it is larger, you search in the upper half... and so on.
With this you can find your value in O(log n) where n is the number of entries in your XmlListModel.
https://en.wikipedia.org/wiki/Binary_search_algorithm
If you have this implemented, to work on your model - either in JavaScript or in C++ or Python... you can have it like this:
Text {
text: binarySearch(model, myList.get(3).value).levName
}
When you implement this algorithm, make sure to deal with the gaps.

Using jQuery and Math.random() to select nested objects properties

I'm creating a random quote machine that will present a random quote from various philosophers.
I have an object literal with nested objects containing philosophers and their quotes. Using jQuery functions and Math.random(), how can I select a random quote from my object literal structure? Is there a better way to organize the data?
I've started with a jQuery closure that will display a designated quote that I'd like to modify using Math.random().
Looking for explanations to solutions as I'm a beginner. Thanks in advance.
Example object literal:
var quotes =
{
awatts: {
name: "Alan Watts",
quote: "The only way to make sense out of change is to plunge into it, move with it, and join the dance."
},
etolle: {
name: "Eckhart Tolle",
quote: "Realize deeply that the present moment is all you ever have."
},
tmckenna: {
name: "Terrence Mckenna",
quote: "“The cost of sanity in this society, is a certain level of alienation” "
}
};
Example jQuery functions with single quote selected:
$(document).ready(function() {
$('.mybutton').click(function() {
$('#quote').html(quotes.awatts.quote);
});
});

The structure of the data seems fine. You could use an array, but an object isn't a problem.
You'd get the keys from the object, and then pick a random key
var quotes = {
awatts: {
name: "Alan Watts",
quote: "The only way to make sense out of change is to plunge into it, move with it, and join the dance."
},
etolle: {
name: "Eckhart Tolle",
quote: "Realize deeply that the present moment is all you ever have."
},
tmckenna: {
name: "Terrence Mckenna",
quote: "“The cost of sanity in this society, is a certain level of alienation” "
}
};
$('.mybutton').click(function() {
var keys = Object.keys(quotes);
var rand = keys[Math.floor(Math.random() * keys.length)];
$('#quote').html(quotes[rand].quote);
});
<script src="https://ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script>
<button class="mybutton">Quote</button>
<br><br>
<div id="quote"></div>

If you can make your quotes object an array, the following would do the trick. Change your array
var quotes = [
{
name: "Alan Watts",
quote: "The only way to make sense out of change is to plunge into it, move with it, and join the dance."
},
{
name: "Eckhart Tolle",
quote: "Realize deeply that the present moment is all you ever have."
},
{
name: "Terrence Mckenna",
quote: "“The cost of sanity in this society, is a certain level of alienation” "
}
];
Set the max and min (to set the upper and lower limits for the random number)
var max = quotes.length, min = 0;
Generate a random number
var rand = Math.random() * (max - min) + min;
On the click event use the random number to choose your random quote
$('#quote').html(quotes[rand]quote);
I have not tested the code. Hope this will get you going :-)

Optimize comparing huge amounts of data with itself in NodeJS

I have a big multidimensional object in nodejs, say 50 MB worth of JSON. It contains variations of biological data. I think I can sufficiently simplify it like so:
{
lads : {
// a lad
lad4515643 : {
brains : {
// a brain
brain1256251 : {
var01 : 'lala',
var02 : 'jaja',
var99 : 'haha',
},
// another brain
brain3567432 : {},
brain4867321 : {},
brain5145621 : {} // etc
},
var01 : 'foo',
var02 : 'bar',
var99 : 'baz'
},
// another lad
lad4555672 : {},
lad5625627 : {},
lad7457255 : {} // etc
}
}
I need to compare all combinations of lads with brains to all lads with brains to see which ones are "better", in order to make some kind of hierarchy. Some parent lads keys weigh in on the brains comparison.
I figured, using iterations over objects using references we can easily assign IDs of the better ones. Take a quick glance over the code (comments) and you see what I mean:
// Iterate over lads
for (var ladId in obj.lads) {
if (obj.lads.hasOwnProperty(ladId)) {
var lad = obj.lads[ladId];
// Iterate over brains
for (var brainId in lad.brains) {
if (lad.brains.hasOwnProperty(brainId)) {
var brain = lad.brains[brainId];
// Iterate over lads again
for (var lad2Id in obj.lads) {
if (obj.lads.hasOwnProperty(lad2Id)) {
var lad2 = obj.lads[lad2Id];
// Iterate over this lads' brains
for (var brain2Id in lad2.brains) {
if (lad2.brains.hasOwnProperty(brain2Id)) {
var brain2 = lad2.brains[brain2Id];
// One lad+brain combination
var drone1 = {
lad : lad,
brain : brain
};
// Another lad+brain combination
var drone2 = {
lad : lad2,
brain : brain2
ladId : lad2Id, // Required to store the reference if better
brainId : brain2Id // Required to store the reference if better
};
// Do the comparison unless we are comparing ourselves
if (brain != brain2) {
// Objects are passed as reference, so this is convenient:
judge(drone1, drone2);
}
}
}
}
}
}
}
}
}
// Judge who is better
function judge(drone1, drone2) {
// some magic that compares lad+brain combos
if (magic) {
// Add list of better versions
drone1.brain.better = drone1.brain.better || [];
// Everything is passed by reference - I can modify the original brain object directly
drone1.brain.better.push({
ladId : drone2.ladId,
brainId : drone2.brainId
});
}
}
Now of course, the number of iterations increases exponentially when the dataset increases. With 3000 brains in total, there are already 9 million iterations, which with the magic adds up to more than 10 seconds of execution time.
What optimizations would be (hugely) beneficial in a scenario like this, apart from using multiple threads?
Since judge() is purely math, does it really make a difference if I convert every single step of the iteration to a callback style? (In my imagination, that would only create a huge overhead of anonymous functions and memory usage.)