Develop Reference

Use child array to get count for table

I have some json data. I need to count the number of strings in one of the children then create a table that is grouped by part of the child strings and used the count of specific child strings in the table. Confusing and impossible as it seems, this is what we need.
Honestly, I am barely sure where to even start here. Just displaying the strings correctly was a nightmare.
Here's some example json, I'm using for in loops to get through the levels of json above this, that part works fine:
"DataValues": [
{
"Key": "Stuff Type",
"Id": "95492",
"ComboBoxPairs": [
{
"Value": {
"Key": "3 Gallon",
"Value": "3 Gallon",
"ExtraValues": []
},
"Children": [
{
"Key": "Scan",
"Id": "93478",
"Strings": [
"DogType:Lab,Age:3,Name:Bowser",
"DogType:Lab,Age:5,Name:Dingo",
"DogType:Mutt,Age:1,Name:Muttman",
"DogType:Weiner,Age:1,Name:Goof",
"DogType:Mutt,Age:5,Name:Muttman",
"DogType:Puppy,Age:1,Name:Silly",
"DogType:Puppy,Age:1,Name:Sammy",
"DogType:Puppy,Age:1,Name:Shooter",
"DogType:Puppy,Age:1,Name:Doc",
]
}
]
},
{
"Value": {
"Key": "1 Gallon",
"Value": "1 Gallon",
"ExtraValues": []
},
"Children": [
{
"Key": "Scan",
"Id": "93478",
"Strings": [
"DogType:Puppy,Age:1,Name:Wingo",
"DogType:Puppy,Age:1,Name:Scrappy",
"DogType:Mutt,Age:4,Name:Goober"
]
}
]
}
]
}
]
Here's what I need to build:
DogType ContainerType Quantity Volume
Lab
3 Gallon 2 6 Gallon
Mutt
1 Gallon 1 1 Gallon
3 Gallon 2 6 Gallon
Weiner
3 Gallon 1 3 Gallon
Puppy
1 Gallon 2 6 Gallon
3 Gallon 4 12 Gallon
I am honestly not even sure where to get started
Honestly, I'm not even sure if this is possible? You can see that the table needs to be grouped by part of the string, the DogType. I then need to be able to count how many strings with a certain dog type there are in each ContainerType object, and pass that into the Quantity and ContainerType columns in the table. Then volume is multiplying the Gallon value, which is just text, by the quantity.
I'm sure I need to put more code but I can't even think of how to manage this. There could also be multiple ContainerTypes, this data isn't static.
The data isn't my design but I can't change it, just getting to this point has been a disaster. Can anyone think of a way to do this?

Okay so what you need to do is transform some data. Basically change the way a certain data is stored / represented. Now the Strings are in a consistent format, so we can use a combination of string operations to get a key => value store out of it (I am going to use ES6 syntax to simplify boilerplate)
let parseString = str => {
let pairs = str.split(',');
let obj = {};
pairs.forEach(pair => {
pair = pair.split(':');
obj[pair[0]] = pair[1];
});
return obj;
};
This function simply takes one of your strings, "DogType:Lab,Age:3,Name:Bowser" for example, and splits out an object such as {'DogType': 'Lab', 'Age': 3...}. Now that we have this, we can start to manipulate and group the data itself (if you're not sure what array.map does, it basically gives a new array after running a function on each of the array's values).
let comboBoxPairs = data.ComboBoxPairs.map(comboBoxPair => {
comboBoxPair.Children = comboBoxPair.Children.map(children => parseString(children));
return comboBoxPair;
});
Now we have replaced the incomprehensible array of strings into an array of objects we can start to group. First we need to figure out a data structure. You're grouping by dog type first, so our object's keys are the Dog's Type. Then we simply need to add each quantity into the dog's type and increment a count each time the quantity is encountered.
let dogType = {};
comboBoxPairs.forEach(comboBoxPair => {
comboBoxPair.Children.forEach(children => {
if (typeof dogType[children.DogType] === 'undefined') {
dogType[children.DogType] = {};
}
if (typeof dogType[children.DogType][comboBoxPair.Value.Key] === 'undefined') {
dogType[children.DogType][comboBoxPair.Value.Key] = 0;
}
dogType[children.DogType][comboBoxPair.Value.Key]++;
});
});
And you're done! This gives you a Object such as :
{
'Lab': {
'1 Gallon': 2,
....
},
};
You can then loop through this object to display the values and totals.

Is your table example illustrative or is it meant to represent your sample data?
Your table does not seem to tally with the sample data so I am in doubt as to whether my understanding is correct?
If I can start with a generalised answer - which may not be enough but it may head you in the right direction~
1. If you can parse the data in full before seeking to output your results then you can build a summary array which makes your 'un-ordered' input data trivial - then presentation becomes a separate step.
2. If you have to do it in one pass - then you would normally think about sorting the data first - on the input - this is so you can handle mutt's appearing at non-contiguous points.
Make sense?
My personal preference would be to build your running totals into a separate data structure - whether you have scenario 1 or 2 so that you can trap the increment no matter what level you find your relevant keys at within the for-in.
In short - is there any conceptual problem with building a new object for your summary?

How can I reformat this simple JSON so it doesn't catch "Circular structure to JSON" exception?

Introduction
I'm learning JavaScript on my own and JSON its something along the path. I'm working on a JavaScript WebScraper and I want, for now, load my results in JSON format.
I know I can use data base, server-client stuff, etc to work with data. But I want to take this approach as learning JSON and how to parse/create/format it's my main goal for today.
Explaining variables
As you may have guessed the data stored in the fore mentioned variables comes from an html file. So an example of the content in:
users[] -> "Egypt"
GDP[] -> "<td> $2,971</td>"
Regions[] -> "<td> Egypt </td>"
Align[] -> "<td> Eastern Bloc </td>"
Code
let countries = [];
for(let i = 0; i < users.length; i++)
{
countries.push( {
'country' : [{
'name' : users[i],
'GDP' : GDP[i],
'Region' : regions[i],
'Align' : align[i]
}]})
};
let obj_data = JSON.stringify(countries, null, 2);
fs.writeFileSync('countryballs.json', obj_data);
Code explanation
I have previously loaded into arrays (users, GDP, regionsm align) those store the data (String format) I had extracted from a website.
My idea was to then "dump" it into an object with which the stringify() function format would format it into JSON.
I have tested it without the loop (static data just for testing) and it works.
Type of error
let obj_data = JSON.stringify(countries, null, 2);
^
TypeError: Converting circular structure to JSON
--> starting at object with constructor 'Node'
| property 'children' -> object with constructor 'Array'
| index 0 -> object with constructor 'Node'
--- property 'parent' closes the circle
What I want from this question
I want to know what makes this JSON format "Circular" and how to make this code work for my goals.
Notes
I am working with Node.js and Visual Studio Code
EDIT
This is further explanation for those who were interested and thought it was not a good question.
Test code that works
let countries;
console.log(users.length)
for(let i = 0; i < users.length; i++)
{
countries = {
country : [
{
"name" : 'CountryTest'
}
]
}
};
let obj_data = JSON.stringify(countries, null, 2);
fs.writeFileSync('countryballs.json', obj_data);
});
Notice in comparison to the previous code, right now I am inputing "manually" the name of the country object.
This way absolutely works as you can see below:
Now, if I change 'CountryTest' to into a users[i] where I store country names (Forget about why countries are tagged users, it is out of the scope of this question)
It shows me the previous circular error.
A "Partial Solution" for this was to add +"" which, as I said, partially solved the problem as now there is not "Circular Error"
Example:
for(let i = 0; i < users.length; i++)
{
countries = {
country : [
{
"name" : users[i]+''
}
]
}
};
Resulting in:
Another bug, which I do not know why is that only shows 1 country when there are 32 in the array users[]
This makes me think that the answers provided are not correct so far.
Desired JSON format
{
"countries": {
"country": [
{
"name": "",
"GDP" : "",
"Region" : "",
"Align" : ""
},
{
"name": "",
"GDP" : "",
"Region" : "",
"Align" : ""
},
{
"name": "",
"GDP" : "",
"Region" : "",
"Align" : ""
}
]}
}

Circular structure error occurs when you have a property of the object which is the object itself directly (a -> a) or indirectly (a -> b -> a).
To avoid the error message, tell JSON.stringify what to do when it encounters a circular reference. For example, if you have a person pointing to another person ("parent"), which may (or may not) point to the original person, do the following:
JSON.stringify( that.person, function( key, value) {
if( key == 'parent') { return value.id;}
else {return value;}
})
The second parameter to stringify is a filter function. Here it simply converts the referred object to its ID, but you are free to do whatever you like to break the circular reference.
You can test the above code with the following:
function Person( params) {
this.id = params['id'];
this.name = params['name'];
this.father = null;
this.fingers = [];
// etc.
}
var me = new Person({ id: 1, name: 'Luke'});
var him = new Person( { id:2, name: 'Darth Vader'});
me.father = him;
JSON.stringify(me); // so far so good
him.father = me; // time travel assumed :-)
JSON.stringify(me); // "TypeError: Converting circular structure to JSON"
// But this should do the job:
JSON.stringify(me, function( key, value) {
if(key == 'father') {
return value.id;
} else {
return value;
};
})
The answer is from StackOverflow question,
Stringify (convert to JSON) a JavaScript object with circular reference

From your output, it looks as though users is a list of DOM nodes. Rather than referring to these directly (where there are all sort of possible cyclical structures), if you just want their text, instead of using users directly, try something like
country : [
{
"name" : users[i].textContent // maybe also followed by `.trim()
}
]
Or you could do this up front to your whole list:
const usersText = [...users].map(node => node.textContent)
and then use usersText in place of users as you build your object.
If GDP, regions and align are also references to your HTML, then you might have to do the same with them.

EUREKA!
As some of you have mentioned above, let me tell you it is not a problem of circularity, at first..., in the JSON design. It is an error of the data itself.
When I scraped the data it came in html format i.e <td>whatever</td>, I did not care about that as I could simply take it away later. I was way too focused in having the JSON well formatted and learning.
As #VLAZ and #Scott Sauyezt mentioned above, it could be that some of the data, if it is not well formatted into string, it might be referring to itself somehow as so I started to work on that.
Lets have a look at this assumption...
To extract the data I used the cheerio.js which gives you a kind of jquery thing to parse html.
To extract the name of the country I used:
nullTest = ($('table').eq(2).find('tr').eq(i).find('td').find('a').last());
//"Partial solution" for the OutOfIndex nulls
if (nullTest != null)
{
users.push(nullTest);
}
(nullTest helps me avoid nulls, I will implement some RegEx when everything works to polish the code a bit)
This "query" would output me something like:
whatEverIsInHereIfThereIsAny
or else.
to get rid off this html thing just add .html() at the end of the "jquery" such as:
($('table').eq(2).find('tr').eq(i).find('td').find('a').last().html());
That way you are now working with String and avoiding any error and thus solves this question.

How do I to target a single value in firebase database?

I'm having trouble figuring out how to target a single object in the firebase database. For example, I want to have a specific word/definition shown when I click an index card. I'm using this to store the data:
wordVal = $("#word").val();
defVal = $("#def").val();
data = firebase.database().ref("indexCards");
data.push( { 'word': wordVal, 'definition': defVal } );
and this to retrieve it:
data.on("child_added", function(snapshot){
console.log(snapshot.val().word);
console.log(snapshot.val().definition);
});
This gives me the whole list of words and definitions. I want to refer to specific words and specific definitions, separately. The docs say that I can reference the specific values by doing this - firebase.database().ref("child/path").
But my question is...how can I reference the path when the parents are those random numbers and letters (see below)? I know that these are unique IDs generated by firebase, but I don't know how to access them like I'd access ordinary objects.
{
"-KQIOHLNsruyrrnAhqis" : {
"definition" : "person, place, thing",
"word" : "noun"
},
"-KQIOO7Wtp2d5v2VorqL" : {
"definition" : "device for photos",
"word" : "camera"
},
"-KQISp4WMnjABxQayToD" : {
"definition" : "circus act",
"word" : "clown"
},
"-KQITC9W1lapBkMyiL7n" : {
"definition" : "device used for cutting",
"word" : "scissors"
}
}

You can use a query like this:
data = firebase.database().ref('indexCards').orderByChild('word').equalTo('noun')
It is self-explanatory, we get reference to indexCards where value of key word is equal noun, so it will return object with key -KQHZ3xCHTQjuTvonSwj
Link to Firebase docs: Retrieve data - Sorting and Filtering data

You need to put the initial push into an object then call the object later. ie
To PUSH data:
firebase.database().ref().push( { 'word': wordVal, 'definition': defVal } )
To UPDATE data or call it later:
firebase.database().ref("indexCards").set({})
or in this case:
data.set({})

IndexedDB: Can you use an array element as a key or an index?

Consider the following object store, with the domain key set as the keyPath
var tags = [
//codes: 0 - markdown wrap tag
// 1 - HTML wrap tag
// 2 - single tag
{ domain: "youtube",
bold:["*",0],
strikethrough:["-",0],
italic:["_",0]
},
{ domain: "stackoverflow",
bold:["<strong>",1],
italic:["<em>",1],
strikethrough:["<del>",1],
superscript:["<sup>",1],
subscript:["<sub>",1],
heading1:["<h1>",1],
heading2:["<h2>",1],
heading3:["<h3>",1],
blockquote:["<blockquote>",1],
code:["<code>",1],
newline:["<br>",2],
horizontal:["<hr>",2]
}
];
The above code works fine and lets me do look-ups easily and efficiently. However, there are many cases where two objects in the store are completely identical except for their domain attribute.
For example, I want to add objects for all of the Stack Exchange sites to the store, and all of those objects would be equal to the one for StackOverflow.
So, rather than create many separate objects, I want to do something like this:
var tags = [
//codes: 0 - markdown wrap tag
// 1 - HTML wrap tag
// 2 - single tag
{ domain: ["youtube"],
bold:["*",0],
strikethrough:["-",0],
italic:["_",0]
},
{ domain: ["stackoverflow","stackexchange",...],
bold:["<strong>",1],
italic:["<em>",1],
strikethrough:["<del>",1],
superscript:["<sup>",1],
subscript:["<sub>",1],
heading1:["<h1>",1],
heading2:["<h2>",1],
heading3:["<h3>",1],
blockquote:["<blockquote>",1],
code:["<code>",1],
newline:["<br>",2],
horizontal:["<hr>",2]
}
];
Would it be possible to use a KeyGen rather than a keyPath and set up some kind of index that took a value and searched for it in the arrays pointed to by the domain key?
Or would I have to use a cursor each time I want to do a look up?
Some potentially helpful references are:
https://developer.mozilla.org/en-US/docs/Web/API/IndexedDB_API/Basic_Concepts_Behind_IndexedDB
http://www.w3.org/TR/IndexedDB/#key-path-construct
https://developer.mozilla.org/en-US/docs/Web/API/IndexedDB_API/Using_IndexedDB

The solution is to use an index with the multiEntry key property set to true
see this link (thanks #kyaw Tun)
Each index also has a multiEntry flag. This flag affects how the index behaves when the result of evaluating the index's key path yields an Array. If the multiEntry flag is false, then a single record whose key is an Array is added to the index. If the multiEntry flag is true, then the one record is added to the index for each item in the Array. The key for each record is the value of respective item in the Array.
Armed with this index, a specific keyPath is no longer necessary, so you can just use a keyGen for simplicity.
So, to create the database:
request.onupgradeneeded = function(event)
{
var db = event.target.result;
var objectStore = db.createObjectStore("domains", {autoIncrement: true });
objectStore.createIndex("domain", "domain", { unique: true, multiEntry: true });
for(var i in tags)
{
objectStore.add(tags[i]);
console.log("added " + tags[i]["domain"] + " to the DB");
}
};
and an example of using a domain to query for an object:
var objectStore = db.transaction("domains").objectStore("domains");
var query = objectStore.index("domain").get(queryURL);
query.onsuccess = function(event){...};

How can i navigate through the json?

I have some JSON which I have in a object but I can seem to return the values a sample of the json is as follows.
{
"rootLayout":"main",
"layoutDescriptions":[
{
"id":"main",
"container" : {
"type":"Tabs",
"content":[
{
"type":"Panel",
"label":"Simple Address",
"layout":"SimpleForm",
"comment":"This form is simple name value pairs",
"content":[
{ "type":"label", "constraint":"newline", "text":"Org Name" },
{ "type":"text", "property":"propOne" },
{ "type":"label", "constraint":"newline", "text":"Address" },
{ "type":"text", "property":"addrLine1" },
{ "type":"text", "property":"addrLine2" },
{ "type":"text", "property":"addrLine3" },
{ "type":"label", "constraint":"newline", "text":"Postcode" },
{ "type":"text", "property":"postcode" }
]
},
I am trying to return the rootLayout using
obj[0].rootLayout.id
This doesn't work also I am wondering how to access the content elements.
I am new to json and I have been thrown in the deep end I think. I cannot find any good reading on the internet can anyone recommend some.
Thanks.

Some explanation because you don't seem to understand JSON
It's not as complicated as one may think. It actually represents javascript objects as if they'd be written by code.
So if you have JSON written as:
{
id : 100,
name: "Yeah baby"
}
This means that your object has two properties: id and name. The first one is numeric and the second one is string.
In your example you can see that your object has two properties: rootLayout and layoutDescriptions. The first one jsonObj.rootLayout is string and will return "main" and the second one is an array:
layoutDescriptions: [ {...}, {...},... ]
Apparently an array of objects because array elements are enclosed in curly braces. This particular array element object that you provided in your example has its own properties just like I've explained for the top level object: id (string), container (another object because it's again enclosed in curlies) etc...
I hope you understand JSON notation a bit more.
So let's go to your question then
You can get to id by accessing it via:
jsonObj.layoutDescriptions[0].id
and further getting to your content objects:
var contentObjects = jsonObj.layoutDescriptions[0].container.content[0].content;
for(var i = 0; i < contentObjects.length, i++)
{
// assign this inner object to a variable for simpler property access
var contObj = contentObjects[i];
// do with this object whatever you need to and access properties as
// contObj.type
// contObj.property
// contObj.text
// contObj.constraint
}
Mind that this will only enumerate first content object's content objects... If this makes sense... Well look at your JSON object and you'll see that you have nested content array of objects.

The object is an object, not an array, and it doesn't have a property called 0.
To get rootLayout:
obj.rootLayout
However, rootLayout is a string, not an object. It doesn't have an id. The first item in the layoutDescriptions array does.
obj.layoutDescriptions[0].id

Are you trying to get one of layoutDescriptions with id equals to obj.rootLayout?
var targetLayout = {};
for(var i = 0; i < obj.layoutDescriptions.length; i++) {
if(obj.layoutDescriptions[i].id == obj.rootLayout) {
targetLayout = obj.layoutDescriptions[i]; break;
}
}
console.log(targetLayout);