I'm trying to use JSON.strigify() on my object (see screenshot).
However I get result which I'm not expecting, object indexes are in wrong order.
Full stringified json, you can see here: http://vpaste.net/LqNlq
As you can see first index is 9:0, but not 8:0 as expected.
What's the problem here ?
The keys of Objects in javascript aren't guaranteed to be in any order.
You should make it an Array of Objects instead to preserve order.
e.g.
{
"1": [
{ "key": "8:0", ... },
{ "key": "8:30", ... },
...
],
"2": ...
}
This should also be the same structure if you expect your top level keys ("1", "2", etc.) to be iterated over in order.
Related
I am working on a project that has JSON format output. I need a clarity on the JSON array structure. So There are fields that are multiple entry like an array. If an element is an array but has only one value, does it still include an array node '[' in the structure?
Example:
This is a sample JSON element which is an array and has multiple values.
"Talents": [
{
"Items": "test"
},
{
"Items": "test"
}
]
If this element does not have multiple values, will it appear as below?
"Talents":
{
"Items": "test"
}
The '[' does not appear for an array type element with single value. Can someone Pls clarify this?
Single-item arrays will still include the array brackets in JSON format, as they are still arrays. In other words, there is no such native mechanism which converts single-item arrays to a non-array representation. So with your single-item example, it would be represented like this:
"Talents": [
{
"Items": "test"
}
]
You can easily test this out with some simple code:
let jsonSingleItem = { "Talents": [ {"Items": "item1"} ] };
let arraySingleItem = [ {"Items": "item1"} ];
console.log(JSON.stringify(jsonSingleItem));
console.log(jsonSingleItem);
console.log(arraySingleItem);
Which yields the following output:
{"Talents":[{"Items":"item1"}]}
{ Talents: [ { Items: 'item1' } ] }
[ { Items: 'item1' } ]
So in all cases (a stringified JSON object, native JSON, and a javascript array) the single item is still in an array.
Note: It is not uncommon that a consumer of an API will send data (i.e. JSON) in ways which are outside the agreed-upon contract/schema that API defines, and this (sending an object instead of a single-object array when there is just one item) is one example I have seen before. It would be up to the owner/developer of the API as to whether they build in flexibility to handle input which deviates from the API schema.
Square brackets ("[]") denotes JSONArray which in your case can access like
Talents[0]
will return
{
"Items": "test"
}
In second case, curve brackets denotes an JSON object. If you want to access value of items. Than you can by
Talents.Items
OR
Talents["Items"]
will return
"Test"
for complete reference,
JSON Syntax
There's a number of posts here about this issue, and they all contain a lot of assertions that can be summarized like this:
Object properties are never guaranteed to be ordered in any way.
JSON.parse() never sorts properties in any way.
Obviously we tend to have no doubt about #1 above, so we may reasonably expect that, for any operation, properties are processed merely in the order they appear.
[edit, following the #Bergi's comment: or at least they should appear in a random order]
Then from that we might especially infer that #2 should be true.
But look at this snippet:
(BTW note: to show the results, snippets below don't use console.log() which may itself change order of the output. Instead objects are iterated by for (key in obj) and the output displayed in the document)
var inputs = [
'{"c": "C", "a": "A", "b": "B"}',
'{"3": "C", "1": "A", "2": "B"}',
'{"c": "C", "a": "A", "3": "C", "1": "A", "b": "B", "2": "B"}'
];
for (var i in inputs) {
var json = inputs[i],
parsed = JSON.parse(json),
output = [];
for (var j in parsed) {
output.push(j + ': ' + parsed[j]);
}
document.write(`JSON: ${json}<br />Parsed: ${output.join(', ')})<hr />`);
}
It shows that, given a JSON string having unordered keys:
When the input has keys with non-numeric values, the parsed object has its properties in the same order than in the input. This is consistent with the #2 assumption above.
Conversely when the input has keys with numeric values (though they're strings, so not firing parse error), the parsed object has its properties sorted. This now contradicts the #2 assumption.
More: when there are mixed numeric and non-numeric key values, first appear the numeric properties sorted, then the non-numeric properties in their original order.
From that I was first tempted to conclude that actually there would be a (non-documented?) feature, so JSON.parse() works following the "rules" exposed above.
But I had the idea to look further, so the snippet below now shows how ordered are the properties of a merely coded object:
var objects = [
[
'{"c": "C", "a": "A", "b": "B"}',
{"c": "C", "a": "A", "b": "B"}
],
[
'{"3": "C", "1": "A", "2": "B"}',
{"3": "C", "1": "A", "2": "B"}
],
[
'{"c": "C", "a": "A", "3": "C", "1": "A", "b": "B", "2": "B"}',
{"c": "C", "a": "A", "3": "C", "1": "A", "b": "B", "2": "B"}
]
];
for (var i in objects) {
var object = objects[i],
output = [];
for (var j in object[1]) {
output.push(j + ': ' + object[1][j]);
}
document.write(`Code: ${object[0]}<br />Object: ${output.join(', ')}<hr />`);
}
It results in analogue observations, i.e. whichever order they're coded, properties are stored following the 3rd rule above:
numerically named properties are all put first, sorted
other properties are set next, ordered as coded
So it means that JSON.parse() is not involved: in fact it seems to be a fundamental process of object building.
Again this appears not documented, at least as far I could find.
Any clue for a real, authoritative, rule?
[Edit, thanks to #Oriol's answer] It actually appears that, synthetically:
This behaviour conforms to an ECMA specification rule.
This rule should apply to all methods where a specific order is guaranteed but is optional for other cases.
However it seems that modern browsers all choose to apply the rule whatever method is involved, hence the apparent contradiction.
The properties of an object have no order, so JSON.parse can't sort them. However, when you list or enumerate the properties of an object, the order may be well-defined or not.
Not necessarily for for...in loops nor Object.keys
As fully explained in Does ES6 introduce a well-defined order of enumeration for object properties?, the spec says
The mechanics and order of enumerating the properties is not specified
But yes for OrdinaryOwnPropertyKeys
Objects have an internal [[OwnPropertyKeys]] method, which is used for example by Object.getOwnPropertyNames and Object.getOwnPropertySymbols.
In the case of ordinary objects, that method uses the OrdinaryGetOwnProperty abstract operation, which returns properties in a well-defined order:
When the abstract operation OrdinaryOwnPropertyKeys is called with
Object O, the following steps are taken:
Let keys be a new empty List.
For each own property key P of O that is an integer index, in
ascending numeric index order
Add P as the last element of keys.
For each own property key P of O that is a String but is not
an integer index, in ascending chronological order of property creation
Add P as the last element of keys.
For each own property key P of O that is a Symbol, in ascending chronological order of property creation
Add P as the last element of keys.
Return keys.
Therefore, since an order is required by OrdinaryOwnPropertyKeys, implementations may decide to internally store the properties in that order, and use it too when enumerating. That's what you observed, but you can't rely on it.
Also be aware non-ordinary objects (e.g. proxy objects) may have another [[OwnPropertyKeys]] internal method, so even when using Object.getOwnPropertyNames the order could still be different.
so we may reasonably expect that, for any operation, properties are processed merely in the order they appear
That's where the flaw in the reasoning lies. Given that object properties aren't guaranteed to be ordered, we have to assume that any operation processes properties in any order that it sees fit.
And in fact engines evolved in a way that treat integer properties specially - they're like array indices, and are stored in a more efficient format than a lookup table.
I have some data which I originally stored in a generic Javascript object, with the ID as a key:
{
"7": {"id":"7","name":"Hello"},
"3": {"id":"3","name":"World"},
...
}
However, I discovered that browsers do not guarantee a particular object order when looping through them, so in the above "3" would come before "7". I switched to using an array format like this:
[
{"id":"7","name":"Hello"},
{"id":"3","name":"World"},
...
]
Now, I can loop in the correct order but cannot do fast lookups, e.g. data["3"] without having to loop through the array.
Is there a good way to combine both approaches? I would rather avoid using a separate object for each format, because the object is pretty large (hundreds of elements).
I have run across this problem as well. A solution is to keep an ordered array of keys in addition to the original object.
var objects = {
"7": {"id":"7","name":"Hello"},
"3": {"id":"3","name":"World"},
...
}
var order = [ "3", "7", ... ];
Now if you want the second element you can do this lookup:
var second_object = objects[order[1]];
The ECMA standard does not say anything about the order of the elements in an object. And specifically Chrome reorders the keys when they look like numbers.
Example:
var example = {
"a": "a",
"b": "b",
"1": "1",
"2": "2"
};
if you print this in Chrome will get something like:
{
1: "1",
2: "2",
"a": "a",
"b": "b"
};
It's a little sour .. but life.
You could use the solution Andy linked as well, basically wrapping these two together in one object.
An alternative that I use a lot is a custom map function that allows you to specify the order in which the object is traversed. Typically you will do sorting when you're printing your data to the user so while you loop and create your table rows (for instance) your iterator will pass the rows in the order your sort function specifies. I thought it was a nice idea :)
The signature looks like:
function map(object, callback, sort_function);
Example usage:
map(object, function (row) {
table.add_row(row.header, row.value);
}, function (key1, key2) {
return object[key1] - object[key2];
});
Rather than coding your own, there are off-the-shelf libraries available to provide "as provided" JSON parsing or "consistently sorted" JSON printing for display.
You might well consider either of these:
The 'json-order' package offers parsing, formatting & pretty-printing with stable ordering. This is based on having ordered input.
The 'fast-json-stable-stringify' package offers deterministic formatting based on sorting.
I am trying to update a value in the nested array but can't get it to work.
My object is like this
{
"_id": {
"$oid": "1"
},
"array1": [
{
"_id": "12",
"array2": [
{
"_id": "123",
"answeredBy": [], // need to push "success"
},
{
"_id": "124",
"answeredBy": [],
}
],
}
]
}
I need to push a value to "answeredBy" array.
In the below example, I tried pushing "success" string to the "answeredBy" array of the "123 _id" object but it does not work.
callback = function(err,value){
if(err){
res.send(err);
}else{
res.send(value);
}
};
conditions = {
"_id": 1,
"array1._id": 12,
"array2._id": 123
};
updates = {
$push: {
"array2.$.answeredBy": "success"
}
};
options = {
upsert: true
};
Model.update(conditions, updates, options, callback);
I found this link, but its answer only says I should use object like structure instead of array's. This cannot be applied in my situation. I really need my object to be nested in arrays
It would be great if you can help me out here. I've been spending hours to figure this out.
Thank you in advance!
General Scope and Explanation
There are a few things wrong with what you are doing here. Firstly your query conditions. You are referring to several _id values where you should not need to, and at least one of which is not on the top level.
In order to get into a "nested" value and also presuming that _id value is unique and would not appear in any other document, you query form should be like this:
Model.update(
{ "array1.array2._id": "123" },
{ "$push": { "array1.0.array2.$.answeredBy": "success" } },
function(err,numAffected) {
// something with the result in here
}
);
Now that would actually work, but really it is only a fluke that it does as there are very good reasons why it should not work for you.
The important reading is in the official documentation for the positional $ operator under the subject of "Nested Arrays". What this says is:
The positional $ operator cannot be used for queries which traverse more than one array, such as queries that traverse arrays nested within other arrays, because the replacement for the $ placeholder is a single value
Specifically what that means is the element that will be matched and returned in the positional placeholder is the value of the index from the first matching array. This means in your case the matching index on the "top" level array.
So if you look at the query notation as shown, we have "hardcoded" the first ( or 0 index ) position in the top level array, and it just so happens that the matching element within "array2" is also the zero index entry.
To demonstrate this you can change the matching _id value to "124" and the result will $push an new entry onto the element with _id "123" as they are both in the zero index entry of "array1" and that is the value returned to the placeholder.
So that is the general problem with nesting arrays. You could remove one of the levels and you would still be able to $push to the correct element in your "top" array, but there would still be multiple levels.
Try to avoid nesting arrays as you will run into update problems as is shown.
The general case is to "flatten" the things you "think" are "levels" and actually make theses "attributes" on the final detail items. For example, the "flattened" form of the structure in the question should be something like:
{
"answers": [
{ "by": "success", "type2": "123", "type1": "12" }
]
}
Or even when accepting the inner array is $push only, and never updated:
{
"array": [
{ "type1": "12", "type2": "123", "answeredBy": ["success"] },
{ "type1": "12", "type2": "124", "answeredBy": [] }
]
}
Which both lend themselves to atomic updates within the scope of the positional $ operator
MongoDB 3.6 and Above
From MongoDB 3.6 there are new features available to work with nested arrays. This uses the positional filtered $[<identifier>] syntax in order to match the specific elements and apply different conditions through arrayFilters in the update statement:
Model.update(
{
"_id": 1,
"array1": {
"$elemMatch": {
"_id": "12","array2._id": "123"
}
}
},
{
"$push": { "array1.$[outer].array2.$[inner].answeredBy": "success" }
},
{
"arrayFilters": [{ "outer._id": "12" },{ "inner._id": "123" }]
}
)
The "arrayFilters" as passed to the options for .update() or even
.updateOne(), .updateMany(), .findOneAndUpdate() or .bulkWrite() method specifies the conditions to match on the identifier given in the update statement. Any elements that match the condition given will be updated.
Because the structure is "nested", we actually use "multiple filters" as is specified with an "array" of filter definitions as shown. The marked "identifier" is used in matching against the positional filtered $[<identifier>] syntax actually used in the update block of the statement. In this case inner and outer are the identifiers used for each condition as specified with the nested chain.
This new expansion makes the update of nested array content possible, but it does not really help with the practicality of "querying" such data, so the same caveats apply as explained earlier.
You typically really "mean" to express as "attributes", even if your brain initially thinks "nesting", it's just usually a reaction to how you believe the "previous relational parts" come together. In reality you really need more denormalization.
Also see How to Update Multiple Array Elements in mongodb, since these new update operators actually match and update "multiple array elements" rather than just the first, which has been the previous action of positional updates.
NOTE Somewhat ironically, since this is specified in the "options" argument for .update() and like methods, the syntax is generally compatible with all recent release driver versions.
However this is not true of the mongo shell, since the way the method is implemented there ( "ironically for backward compatibility" ) the arrayFilters argument is not recognized and removed by an internal method that parses the options in order to deliver "backward compatibility" with prior MongoDB server versions and a "legacy" .update() API call syntax.
So if you want to use the command in the mongo shell or other "shell based" products ( notably Robo 3T ) you need a latest version from either the development branch or production release as of 3.6 or greater.
See also positional all $[] which also updates "multiple array elements" but without applying to specified conditions and applies to all elements in the array where that is the desired action.
I know this is a very old question, but I just struggled with this problem myself, and found, what I believe to be, a better answer.
A way to solve this problem is to use Sub-Documents. This is done by nesting schemas within your schemas
MainSchema = new mongoose.Schema({
array1: [Array1Schema]
})
Array1Schema = new mongoose.Schema({
array2: [Array2Schema]
})
Array2Schema = new mongoose.Schema({
answeredBy": [...]
})
This way the object will look like the one you show, but now each array are filled with sub-documents. This makes it possible to dot your way into the sub-document you want. Instead of using a .update you then use a .find or .findOne to get the document you want to update.
Main.findOne((
{
_id: 1
}
)
.exec(
function(err, result){
result.array1.id(12).array2.id(123).answeredBy.push('success')
result.save(function(err){
console.log(result)
});
}
)
Haven't used the .push() function this way myself, so the syntax might not be right, but I have used both .set() and .remove(), and both works perfectly fine.
I have some data which I originally stored in a generic Javascript object, with the ID as a key:
{
"7": {"id":"7","name":"Hello"},
"3": {"id":"3","name":"World"},
...
}
However, I discovered that browsers do not guarantee a particular object order when looping through them, so in the above "3" would come before "7". I switched to using an array format like this:
[
{"id":"7","name":"Hello"},
{"id":"3","name":"World"},
...
]
Now, I can loop in the correct order but cannot do fast lookups, e.g. data["3"] without having to loop through the array.
Is there a good way to combine both approaches? I would rather avoid using a separate object for each format, because the object is pretty large (hundreds of elements).
I have run across this problem as well. A solution is to keep an ordered array of keys in addition to the original object.
var objects = {
"7": {"id":"7","name":"Hello"},
"3": {"id":"3","name":"World"},
...
}
var order = [ "3", "7", ... ];
Now if you want the second element you can do this lookup:
var second_object = objects[order[1]];
The ECMA standard does not say anything about the order of the elements in an object. And specifically Chrome reorders the keys when they look like numbers.
Example:
var example = {
"a": "a",
"b": "b",
"1": "1",
"2": "2"
};
if you print this in Chrome will get something like:
{
1: "1",
2: "2",
"a": "a",
"b": "b"
};
It's a little sour .. but life.
You could use the solution Andy linked as well, basically wrapping these two together in one object.
An alternative that I use a lot is a custom map function that allows you to specify the order in which the object is traversed. Typically you will do sorting when you're printing your data to the user so while you loop and create your table rows (for instance) your iterator will pass the rows in the order your sort function specifies. I thought it was a nice idea :)
The signature looks like:
function map(object, callback, sort_function);
Example usage:
map(object, function (row) {
table.add_row(row.header, row.value);
}, function (key1, key2) {
return object[key1] - object[key2];
});
Rather than coding your own, there are off-the-shelf libraries available to provide "as provided" JSON parsing or "consistently sorted" JSON printing for display.
You might well consider either of these:
The 'json-order' package offers parsing, formatting & pretty-printing with stable ordering. This is based on having ordered input.
The 'fast-json-stable-stringify' package offers deterministic formatting based on sorting.