Using the api I was able to retrieve the info i needed trough the following endpoints and create the following object.
var lego = {
"name" : item.name,
"followers" : item.follower_count,
"created" : item.created_at,
"updated" : item.updated_at
}
<datagrid id = "topicGrid" data='[{}]'>
<datagrid-column head="Topic" content-key="name"></datagrid-column>
<datagrid-column head="Followers" content-key="followers"></datagrid-column>
<datagrid-column-time head="Created" content-key="created" format="date"></datagrid-column-time>
<datagrid-column-time head="Updated" content-key="updated" format="date"></datagrid-column-time>
</datagrid>
I would like to somehow append the "lego" object in the new data attribute ?
Now i tried something like
setAttribute("data", lego); but my knowledge here is limited.
Any help is apreciated. Thank you.
Update:
I tried passing the object as a string but I only get the last element of the object. Kind of like it's looping trough the whole object and stopping at the last one.
$('datagrid').attr('data', JSON.stringify(lego));
My solution
$.get( "..." ).done(function( data ) {
var legoArr = []
$.each(data.topics, function(index,item) {
var lego = {
"name" : item.name,
"followers" : item.follower_count,
"created" : item.created_at,
"updated" : item.updated_at
};
legoArr.push(lego);
});
$('datagrid').attr('data', JSON.stringify(legoArr));
});
Introduction
I'm learning JavaScript on my own and JSON its something along the path. I'm working on a JavaScript WebScraper and I want, for now, load my results in JSON format.
I know I can use data base, server-client stuff, etc to work with data. But I want to take this approach as learning JSON and how to parse/create/format it's my main goal for today.
Explaining variables
As you may have guessed the data stored in the fore mentioned variables comes from an html file. So an example of the content in:
users[] -> "Egypt"
GDP[] -> "<td> $2,971</td>"
Regions[] -> "<td> Egypt </td>"
Align[] -> "<td> Eastern Bloc </td>"
Code
let countries = [];
for(let i = 0; i < users.length; i++)
{
countries.push( {
'country' : [{
'name' : users[i],
'GDP' : GDP[i],
'Region' : regions[i],
'Align' : align[i]
}]})
};
let obj_data = JSON.stringify(countries, null, 2);
fs.writeFileSync('countryballs.json', obj_data);
Code explanation
I have previously loaded into arrays (users, GDP, regionsm align) those store the data (String format) I had extracted from a website.
My idea was to then "dump" it into an object with which the stringify() function format would format it into JSON.
I have tested it without the loop (static data just for testing) and it works.
Type of error
let obj_data = JSON.stringify(countries, null, 2);
^
TypeError: Converting circular structure to JSON
--> starting at object with constructor 'Node'
| property 'children' -> object with constructor 'Array'
| index 0 -> object with constructor 'Node'
--- property 'parent' closes the circle
What I want from this question
I want to know what makes this JSON format "Circular" and how to make this code work for my goals.
Notes
I am working with Node.js and Visual Studio Code
EDIT
This is further explanation for those who were interested and thought it was not a good question.
Test code that works
let countries;
console.log(users.length)
for(let i = 0; i < users.length; i++)
{
countries = {
country : [
{
"name" : 'CountryTest'
}
]
}
};
let obj_data = JSON.stringify(countries, null, 2);
fs.writeFileSync('countryballs.json', obj_data);
});
Notice in comparison to the previous code, right now I am inputing "manually" the name of the country object.
This way absolutely works as you can see below:
Now, if I change 'CountryTest' to into a users[i] where I store country names (Forget about why countries are tagged users, it is out of the scope of this question)
It shows me the previous circular error.
A "Partial Solution" for this was to add +"" which, as I said, partially solved the problem as now there is not "Circular Error"
Example:
for(let i = 0; i < users.length; i++)
{
countries = {
country : [
{
"name" : users[i]+''
}
]
}
};
Resulting in:
Another bug, which I do not know why is that only shows 1 country when there are 32 in the array users[]
This makes me think that the answers provided are not correct so far.
Desired JSON format
{
"countries": {
"country": [
{
"name": "",
"GDP" : "",
"Region" : "",
"Align" : ""
},
{
"name": "",
"GDP" : "",
"Region" : "",
"Align" : ""
},
{
"name": "",
"GDP" : "",
"Region" : "",
"Align" : ""
}
]}
}
Circular structure error occurs when you have a property of the object which is the object itself directly (a -> a) or indirectly (a -> b -> a).
To avoid the error message, tell JSON.stringify what to do when it encounters a circular reference. For example, if you have a person pointing to another person ("parent"), which may (or may not) point to the original person, do the following:
JSON.stringify( that.person, function( key, value) {
if( key == 'parent') { return value.id;}
else {return value;}
})
The second parameter to stringify is a filter function. Here it simply converts the referred object to its ID, but you are free to do whatever you like to break the circular reference.
You can test the above code with the following:
function Person( params) {
this.id = params['id'];
this.name = params['name'];
this.father = null;
this.fingers = [];
// etc.
}
var me = new Person({ id: 1, name: 'Luke'});
var him = new Person( { id:2, name: 'Darth Vader'});
me.father = him;
JSON.stringify(me); // so far so good
him.father = me; // time travel assumed :-)
JSON.stringify(me); // "TypeError: Converting circular structure to JSON"
// But this should do the job:
JSON.stringify(me, function( key, value) {
if(key == 'father') {
return value.id;
} else {
return value;
};
})
The answer is from StackOverflow question,
Stringify (convert to JSON) a JavaScript object with circular reference
From your output, it looks as though users is a list of DOM nodes. Rather than referring to these directly (where there are all sort of possible cyclical structures), if you just want their text, instead of using users directly, try something like
country : [
{
"name" : users[i].textContent // maybe also followed by `.trim()
}
]
Or you could do this up front to your whole list:
const usersText = [...users].map(node => node.textContent)
and then use usersText in place of users as you build your object.
If GDP, regions and align are also references to your HTML, then you might have to do the same with them.
EUREKA!
As some of you have mentioned above, let me tell you it is not a problem of circularity, at first..., in the JSON design. It is an error of the data itself.
When I scraped the data it came in html format i.e <td>whatever</td>, I did not care about that as I could simply take it away later. I was way too focused in having the JSON well formatted and learning.
As #VLAZ and #Scott Sauyezt mentioned above, it could be that some of the data, if it is not well formatted into string, it might be referring to itself somehow as so I started to work on that.
Lets have a look at this assumption...
To extract the data I used the cheerio.js which gives you a kind of jquery thing to parse html.
To extract the name of the country I used:
nullTest = ($('table').eq(2).find('tr').eq(i).find('td').find('a').last());
//"Partial solution" for the OutOfIndex nulls
if (nullTest != null)
{
users.push(nullTest);
}
(nullTest helps me avoid nulls, I will implement some RegEx when everything works to polish the code a bit)
This "query" would output me something like:
whatEverIsInHereIfThereIsAny
or else.
to get rid off this html thing just add .html() at the end of the "jquery" such as:
($('table').eq(2).find('tr').eq(i).find('td').find('a').last().html());
That way you are now working with String and avoiding any error and thus solves this question.
I have a CouchDB that contains items with something called save_data (actual data I need), rel (as related account - account linked to that document) and created_at (date of creation).
When I first created my view which I called recent-items I thought that items in that view are sorted in order they were created in, but it didn't take long for me to discover just how wrong I was. I wanted to get all documents related to one user (rel, save_profile in my js which calls the db.view funcion) and then sort them based on created_at so I created map funcion:
function(doc) {
if (doc.rel) {
var p = doc.save_data || {};
var r = doc.rel || {};
var q = doc.created_at || {};
emit([doc.rel], {save_data: doc.save_data, rel: doc.rel});
}
};
and then I called it with these parameters:
db.view(design + "/recent-items", {
descending : "true",
limit : 10,
update_seq : true,
key : [save_profile],
success : function(data) {
...somecode...
}
});
and then I noticed that they didn't appear in the order I wanted, but sorted by their ID which makes sense now, but that's not what I needed. So I did this: I reworked map function so that it shows user and date in key (as fields to be sorted by)
function(doc) {
if (doc.rel) {
var p = doc.save_data || {};
var r = doc.rel || {};
var q = doc.created_at || {};
emit([doc.rel, doc.created_at], {save_data: doc.save_data, rel: doc.rel});
}
};
and then I used startkey instead of key in db.view like this:
db.view(design + "/recent-items", {
descending : "true",
limit : 10,
update_seq : true,
startkey : [save_profile, null],
success : function(data) {
...somecode...
}
});
so that I get all documents related to save_profile but sorted by date also. I did get them, but I also got documents from other users with it so that function is completely unreliable. What I did then is implement endkey also, like this:
db.view(design + "/recent-items", {
descending : "true",
limit : 10,
update_seq : true,
startkey : [save_profile, null],
endkey : [save_profile, {}],
success : function(data) {
...somecode...
}
});
but then I get empty view as result.
Dates I use are in this format:
"2013-05-13T11:59:22.281Z"
and profiles are something like this:
{"rand":"0.26928815129213035","nickname":"Nickname","email":"Label for Gravatar","url":"URL","gravatar_url":"http://www.gravatar.com/avatar/35838c85593335493a1fa916863fee0c.jpg?s=40&d=identicon","name":"testacc"}
I also tried replacing [] and {} in start/endkeys with "0000-00-00T00:00:00.000Z" and "9999-99-99T99:99:99.999Z", but it changed nothing. So... Any ideas? As far as I've seen in other similar problems people just used {} as end key and left startkey without second parameter but that doesn't work here either.
Edit: Solved!
Alright folks, believe it or not I've done it in this way:
I changed my map function to display created at and profile in this way:
function(doc) {
if (doc.rel) {
var p = doc.save_data || {};
var r = doc.rel || {};
var q = doc.created_at || {};
emit([doc.rel, doc.created_at], {save_data: doc.save_data, rel: doc.rel});
}
};
and it appears that if you set descending to true before setting start and end keys you have to reverse the search interval so that endkey contains starting value and startkey containst ending value, like this:
db.view(design + "/recent-items", {
descending : "true",
update_seq : "true",
startkey : [save_profile.name, save_profile.rand, {}],
endkey : [save_profile.name, save_profile.rand, null],
inclusive_end : "true",
success : function(data) {
...somecode...
}
});
That did the trick and it works flawlessly (but also with a complete absence of logic).
I should have caught this before. So when you do the following, it works the way you'd like:
db.view('_design/test_views/_view/view1', startkey: ["account2"])
=> {"total_rows"db.view('_design/test_views/_view/view1')
=> {"total_rows"=>4, "offset"=>0, "rows"=>[{"id"=>"test1", "key"=>["account1", "2009-05-13T11:59:22.281Z"], "value"=>{"rel"=>"account1"}}, {"id"=>"test2", "key"=>["account2", "2012-05-13T11:59:22.281Z"], "value"=>{"rel"=>"account2"}}, {"id"=>"test3", "key"=>["account3", "2011-05-13T11:59:22.281Z"], "value"=>{"rel"=>"account3"}}, {"id"=>"test4", "key"=>["account4", "2010-05-13T11:59:22.281Z"], "value"=>{"rel"=>"account4"}}]}`
db.view('_design/test_views/_view/view1',startkey: ["account2"])
=> {"total_rows"=>4, "offset"=>1, "rows"=>[{"id"=>"test2", "key"=>["account2", "2012-05-13T11:59:22.281Z"], "value"=>{"rel"=>"account2"}}, {"id"=>"test3", "key"=>["account3", "2011-05-13T11:59:22.281Z"], "value"=>{"rel"=>"account3"}}, {"id"=>"test4", "key"=>["account4", "2010-05-13T11:59:22.281Z"], "value"=>{"rel"=>"account4"}}]}`
db.view('_design/test_views/_view/view1',
startkey: ["account2"],
endkey: ["account3",{}])
=> {"total_rows"=>4, "offset"=>1, "rows"=>[{"id"=>"test2", "key"=>["account2", "2012-05-13T11:59:22.281Z"], "value"=>{"rel"=>"account2"}}, {"id"=>"test3", "key"=>["account3", "2011-05-13T11:59:22.281Z"], "value"=>{"rel"=>"account3"}}]}`.
But when you set descending=true, you reverse the order first. Previously you were setting your start and end keys the way that you should when descending=false, but you need to reverse them when you change descending. The CouchDB Definitive Guide is great and talks about these reversed results, worth the read.
I totally missed that you were setting descending, sorry for the confusion.
I think the startkey [save_profile, null] is what is messing you up (although it works via curl). Whatever library you're using (db.view is some javascript lib or ruby CouchRest maybe?) may be recoding null as something else. You can always verify this compared to curl by breaking out the packet sniffer (i.e. Wireshark), although you'll get an idea how inefficient a library can be with all the extra requests it makes. Can make the troubleshooting more difficult.
What results do you get with curl? I mocked up your database, and here's what I get:
$ curl -X GET 'http://localhost:5984/test2/_design/test_views/_view/view1'
{"total_rows":4,"offset":0,"rows":[
{"id":"test1","key":["account1","2009-05-13T11:59:22.281Z"],"value":{"rel":"account1"}},
{"id":"test2","key":["account2","2012-05-13T11:59:22.281Z"],"value":{"rel":"account2"}},
{"id":"test3","key":["account3","2011-05-13T11:59:22.281Z"],"value":{"rel":"account3"}},
{"id":"test4","key":["account4","2010-05-13T11:59:22.281Z"],"value":{"rel":"account4"}}
]}
$ curl -X GET 'http://localhost:5984/test2/_design/test_views/_view/view1?startkey=\["account2"\]&endkey=\["account3",\{\}\]'
{"total_rows":4,"offset":1,"rows":[
{"id":"test2","key":["account2","2012-05-13T11:59:22.281Z"],"value":{"rel":"account2"}},
{"id":"test3","key":["account3","2011-05-13T11:59:22.281Z"],"value":{"rel":"account3"}}
]}
Notice that I am not using null, I'm just leaving that element of the key out. Also notice in curl (and possibly you're library), you have to be really careful what you put in start/end keys. I have to escape all bash stuff that even single quotes don't escape (i.e. [ { etc), and curl escapes spaces as %20, which is then used for the keys. A good approach is to run couchdb not forked out, or just watch its logs, and see what the incoming requests look like. Has been a source of issues for me.
You're using the wildcards in the keys, which is a cool feature. You may have seen this, I've re-read this snippet a few times to try to understand them.
I've beat my head on this similar problem too, until I really understood what views do. But what you're doing should be possible. Any more complicated searches, and I'd really consider Elasticsearch or something similar. The river install for it is really easy, and the queries are pretty similar. But you don't have to work around the limitations of views and orders.
Hope that helps.
Alright folks, believe it or not I've done it in this way:
I changed my map function to display created at and profile in this way:
function(doc) {
if (doc.rel) {
var p = doc.save_data || {};
var r = doc.rel || {};
var q = doc.created_at || {};
emit([doc.rel, doc.created_at], {save_data: doc.save_data, rel: doc.rel});
}
};
and it appears that if you set descending to true before setting start and end keys you have to reverse the search interval so that endkey contains starting value and startkey containst ending value, like this:
db.view(design + "/recent-items", {
descending : "true",
update_seq : "true",
startkey : [save_profile.name, save_profile.rand, {}],
endkey : [save_profile.name, save_profile.rand, null],
inclusive_end : "true",
success : function(data) {
...somecode...
}
});
That did the trick and it works flawlessly (but also with a complete absence of logic).
I am sending some request on a server and it's reply me this:
{"COLUMNS":["REGISTRATION_DT","USERNAME","PASSWORD","FNAME","LNAME","EMAIL","MOBILE","FACEBOOK_ID"],"DATA":[["March, 17 2012 16:18:00","someuser",somepass,"somename","somesur","someemail",sometel,"someid"]]}
I tried a lot but nothing seems to working for me!
var xml2 = this.responseData;
var xml3 = xml2.getElementsByTagName("data");
Ti.API.log(xml3.FNAME);
For this code I get "null".
Any help would be appreciated!
If you're trying to use JSON format, your problem is that the data within the [...] also needs to be in pairs, and grouped in {...} like here.
For instance,
{
"sales": [
{ "firstname" : "John", "lastname" : "Brown" },
{ "firstname" : "Marc", "lastname" : "Johnson" }
] // end of sales array
}
So you might have:
{"COLUMNS": [
{"REGISTRATION_DT" : "19901212", "USERNAME" : "kudos", "PASSWORD" : "tx91!#1", ... },
{"REGISTRATION_DT" : "19940709", "USERNAME" : "jenny", "PASSWORD" : "fxuf#2", ... },
{"REGISTRATION_DT" : "20070110", "USERNAME" : "benji12", "PASSWORD" : "rabbit19", ... }
]
}
If the server is sending you something which you refer to as res, you can just do this to parse it in your Javascript:
var o=JSON.parse(res);
You can then cycle through each instance within columns like follows:
for (var i=0;i<o.COLUMNS.length;i++)
{
var date = o.COLUMNS[i].REGISTRATION_DT; ....
}
see that link. READ JSON RESPONSE
It's perfect.
JSON objects work just like any normal javascript objects or dictionaries
// You can do it this way
var data = this.responseData["DATA"]
// Or this way
var data = this.responseData.DATA
In your case, COLUMNS and data are both arrays, so it looks like you're trying to get the element from data that corresponds to the "FNAME" element in COLUMNS?
var columns = this.responseData["COLUMNS"];
var data = this.responseData["DATA"][0];
for(var i=0; i<columns.length; i++){
if(columns[i] == "FNAME"){
Ti.API.log(data[i]);
}
}
EDIT: If you can't change the data on the server end, you can make your own object client side. This also helps if you have to refer to multiple columns (which you probably do).
var columns = this.responseData["COLUMNS"];
var data = this.responseData["DATA"][0];
var realData = {};
for(var i=0; i<columns.length; i++){
realData[columns[i]] = data[i];
}
// Now you can access properties directly by name.
Ti.API.log(data.FNAME);
More edit:
My answers only consider the first row in DATA, because I misread originally. I'll leave it up to you to figure out how to process the others.
If you got here trying to find out how to read from [Response object] (as I did) -
this what can help:
- if you use fetch don't forget about res.json() before logging in console
fetch(`http://localhost:3000/data/${hour}`, {
method: 'get'
})
.then(res => {
return res.json()
})
.then((response) => {
console.log('res: ' + JSON.stringify(response))
})
Testing out your code in http://jsonlint.com/, it says that your server's response is not a valid JSON string.
Additionally, I recommend checking out jQuery.parseJSON http://api.jquery.com/jQuery.parseJSON/
Just use JSON.parse(serverResponse)