Develop Reference

JSON database module for Ruby/Sinatra? A JS lowDB alternative for Ruby/Sinatra?

I have to deal with a huge json like this acting as live datasource, is loaded every 5 min from a url..
sports: [
{
id: 200,
title: "Horse Racing",
meetings: [ ],
is_virtual: false,
events: [...],
pos: 83
},
{
id: 600,
title: "Tennis",
meetings: [ ],
is_virtual: false,
events: [
{
id: 301804310,
is_virtual: false,
outcomes: [
{
id: 32779738900,
description: "Brown/Pliskova",
},
{
id: 32779738900,
description: "Brown/Pliskova",
}]
}]
}]
And need to write methods like
getAllSports() returning an array object with all sports
getSport(sport_id) returning the object with this sport id
getAllEvents(Sport) returning all events list object of this sprot
getEvent(Sport, event_id) returning events that matches with given event_id
getOutcomes(Event, outcomes) ... and so on
Is there is a library that parses the json and already have methods some methods to help me to do this kind of stuff? example: obj.find(sport_id)...
In JS you have LowDB https://github.com/typicode/lowdb for this, any similar in Ruby/Sinatra? Or any approach suggestion? Im not using Rails.
Thanks in advice

You could always use Ruby's built in JSON library. You would be able to do something like
json_string = '{"name": "my name", "age": 5}'
object = JSON.parse(json_string)
object["name"] => "my name"
You can then use regular ruby hash / array functions on the returned object. In your case, you could do something along the lines of
def getSport(json_object, id)
json_object["sports"].select { | h | h["id"] == id }.first
end
Which, assuming you have already parsed the JSON and passed the resulting value into that function, would return the sport that had the given ID.

AngularJS filter based on an array of strings almost there

I am trying to create a custom angular filter based on an array of strings, for example:
$scope.idArray = ['1195986','1195987','1195988']
The data I want to filter is in the format:
$scope.data = {
"moduleName": null,
"contentholder_0": {
"moduleName": "contentholder",
"id": "-1",
"name": "",
"content": ""
},
"webapps_1": {
"moduleName": "webapps",
"items": [{
"itemid": "1195986",
"name": "abc"
},{
"itemid": "1195987",
"name": "def"
},{
"itemid": "1195988",
"name": "ghi"
}]
}
}
I have looked at this Stack Question to try to create a custom filter:
Example
Here is the JSFiddle of the answer
I have not been able to hack it to fit my data structure for what I want to do (filter the "items" if the "itemid" is in the "idArray")
This is a JSfiddle as close as I can get without crashing angular.
Please forgive me if this is a super easy question but I am a beginner and have tried multiple ways to get this to work but haven't been able to. I am not sure where the actual filtering is being done and how to compare the strings in idArray to data.webapps_1.items.itemid
Any help would be greatly appreciated.

I believe this is what you are trying to do.. This filter now checks to see if any of the items have the same ID as any in the ID array.
.filter('selectedTags', function () {
return function (items, tags) {
var filtered = [];
angular.forEach(items, function (item) {
angular.forEach(tags, function (tag) {
if (item.itemid == tag) {
filtered.push(item);
}
});
});
return filtered;
};
})
See this Fiddle

I am not sure if this can help you but I like to use underscore.js to manipulate my JS objects/arrays and there is a function which does exactly that :
_.pluck(data.webapps_1.items, 'itemid');
Based on documentation :
pluck
_.pluck(list, propertyName)
A convenient version of what is perhaps the most common use-case for map: extracting a list of property values.
var stooges = [{name: 'moe', age: 40}, {name: 'larry', age: 50}, {name: 'curly', age: 60}];
_.pluck(stooges, 'name');
=> ["moe", "larry", "curly"]
From what I understood of your question, this will generate idArray from data
Then you can create a custom filter which call this code :)

MongoDb: How to get a field (sub document) from a document?

Consider this example collection:
{
"_id:"0,
"firstname":"Tom",
"children" : {
"childA":{
"toys":{
'toy 1':'batman',
'toy 2':'car',
'toy 3':'train',
}
"movies": {
'movie 1': "Ironman"
'movie 2': "Deathwish"
}
},
"childB":{
"toys":{
'toy 1':'doll',
'toy 2':'bike',
'toy 3':'xbox',
}
"movies": {
'movie 1': "Frozen"
'movie 2': "Barbie"
}
}
}
}
Now I would like to retrieve ONLY the movies from a particular document.
I have tried something like this:
movies = users.find_one({'_id': 0}, {'_id': 0, 'children.ChildA.movies': 1})
However, I get the whole field structure from 'children' down to 'movies' and it's content. How do I just do a query and retrieve only the content of 'movies'?
To be specific I want to end up with this:
{
'movie 1': "Frozen"
'movie 2': "Barbie"
}

The problem here is your current data structure is not really great for querying. This is mostly because you are using "keys" to actually represent "data points", and while it might initially seem to be a logical idea it is actually a very bad practice.
So rather than do something like assign "childA" and "childB" as keys of an object or "sub-document", you are better off assigning these are "values" to a generic key name in a structure like this:
{
"_id:"0,
"firstname":"Tom",
"children" : [
{
"name": "childA",
"toys": [
"batman",
"car",
"train"
],
"movies": [
"Ironman"
"Deathwish"
]
},
{
"name": "childB",
"toys": [
"doll",
"bike",
"xbox",
],
"movies": [
"Frozen",
"Barbie"
]
}
]
}
Not the best as there are nested arrays, which can be a potential problem but there are workarounds to this as well ( but later ), but the main point here is this is a lot better than defining the data in "keys". And the main problem with "keys" that are not consistently named is that MongoDB does not generally allow any way to "wildcard" these names, so you are stuck with naming and "absolute path" in order to access elements as in:
children -> childA -> toys
children -> childB -> toys
And that in a nutshell is bad, and compared to this:
"children.toys"
From the sample prepared above, then I would say that is a whole lot better approach to organizing your data.
Even so, just getting back something such as a "unique list of movies" is out of scope for standard .find() type queries in MongoDB. This actually requires something more of "document manipulation" and is well supported in the aggregation framework for MongoDB. This has extensive capabilities for manipulation that is not present in the query methods, and as a per document response with the above structure then you can do this:
db.collection.aggregate([
# De-normalize the array content first
{ "$unwind": "$children" },
# De-normalize the content from the inner array as well
{ "$unwind": "$children.movies" },
# Group back, well optionally, but just the "movies" per document
{ "$group": {
"_id": "$_id",
"movies": { "$addToSet": "$children.movies" }
}}
])
So now the "list" response in the document only contains the "unique" movies, which corresponds more to what you are asking. Alternately you could just $push instead and make a "non-unique" list. But stupidly that is actually the same as this:
db.collection.find({},{ "_id": False, "children.movies": True })
As a "collection wide" concept, then you could simplify this a lot by simply using the .distinct() method. Which basically forms a list of "distinct" keys based on the input you provide. This playes with arrays really well:
db.collection.distinct("children.toys")
And that is essentially a collection wide analysis of all the "distinct" occurrences for each"toys" value in the collection, and returned as a simple "array".
But as for you existing structure, it deserves a solution to explain, but you really must understand that the explanation is horrible. The problem here is that the "native" and optimized methods available to general queries and aggregation methods are not available at all and the only option available is JavaScript based processing. Which even though a little better through "v8" engine integration, is still really a complete slouch when compared side by side with native code methods.
So from the "original" form that you have, ( JavaScript form, functions have to be so easy to translate") :
db.collection.mapReduce(
// Mapper
function() {
var id this._id;
children = this.children;
Object.keys(children).forEach(function(child) {
Object.keys(child).forEach(function(childKey) {
Object.keys(childKey).forEach(function(toy) {
emit(
id, { "toys": [children[childkey]["toys"][toy]] }
);
});
});
});
},
// Reducer
function(key,values) {
var output = { "toys": [] };
values.forEach(function(value) {
value.toys.forEach(function(toy) {
if ( ouput.toys.indexOf( toy ) == -1 )
output.toys.push( toy );
});
});
},
{
"out": { "inline": 1 }
}
)
So JavaScript evaluation is the "horrible" approach as this is much slower in execution, and you see the "traversing" code that needs to be implemented. Bad news for performance, so don't do it. Change the structure instead.
As a final part, you could model this differently to avoid the "nested array" concept. And understand that the only real problem with a "nested array" is that "updating" a nested element is really impossible without reading in the whole document and modifying it.
So $push and $pull methods work fine. But using a "positional" $ operator just does not work as the "outer" array index is always the "first" matched element. So if this really was a problem for you then you could do something like this, for example:
{
"_id:"0,
"firstname":"Tom",
"childtoys" : [
{
"name": "childA",
"toy": "batman"
}.
{
"name": "childA",
"toy": "car"
},
{
"name": "childA",
"toy": "train"
},
{
"name": "childB",
"toy": "doll"
},
{
"name": "childB",
"toy": "bike"
},
{
"name": "childB",
"toy": "xbox"
}
],
"childMovies": [
{
"name": "childA"
"movie": "Ironman"
},
{
"name": "childA",
"movie": "Deathwish"
},
{
"name": "childB",
"movie": "Frozen"
},
{
"name": "childB",
"movie": "Barbie"
}
]
}
That would be one way to avoid the problem with nested updates if you did indeed need to "update" items on a regular basis rather than just $push and $pull items to the "toys" and "movies" arrays.
But the overall message here is to design your data around the access patterns you actually use. MongoDB does generally not like things with a "strict path" in the terms of being able to query or otherwise flexibly issue updates.

Projections in MongoDB make use of '1' and '0' , not 'True'/'False'.
Moreover ensure that the fields are specified in the right cases(uppercase/lowercase)
The query should be as below:
db.users.findOne({'_id': 0}, {'_id': 0, 'children.childA.movies': 1})
Which will result in :
{
"children" : {
"childA" : {
"movies" : {
"movie 1" : "Ironman",
"movie 2" : "Deathwish"
}
}
}
}

extract part of json into another object and keep link between both

{
   "data": [
      {
         "date": "2013-03-07",
         "id": "2",
         "vt_color": "#548dd4",
         "vt_name": "follow up",
         "duration": "20",
         "time_booked": "12:00:00",
         "stats": "booked",
         "doctor_id": "00002",
         "patient_id": "00003"
      },
      {
         "date": "2013-03-08",
         "id": "3",
         "vt_color": "#76923c",
         "vt_name": "ultrasound",
         "duration": "30",
         "time_booked": "08:00:00",
         "stats": "booked",
         "pt_name": "demo patien",
         "dr_name": "Momen Alzalabany",
         "doctor_id": "00002",
         "patient_id": "00009"
      }
   ] 
}
what i want is to create another array out of this including vt_name,vt_color and index of data
so i use jquery
var words = [];
$.each(arr['data'],function(ref){
words[this.doctor_id].push({name: this.vt_color,color: this.vt_name,index:ref});
});
console.log(koko);
FAIL : words[this.do_id] is not defined....
how can i do this ? i want outcome to be
sorry i'm a newbie with json/js
i want outcome in php would be
['00002'=>[
['name'=>'follow up','color'=>'#548dd4',index=>[0,3,4]],
['name'=>'ultrasound','color'=>'#769dd4',index=>[1,5,8]]
]
]

Change the type of words to object (not array), since you want a map with a string (e.g. '00002') as key:
var words = {};
You want do add to an array which does not exist. You need to create it first:
words[this.doctor_id] = words[this.doctor_id] || []; // creates an array if not already existing
words[this.doctor_id].push({name: this.vt_color,color: this.vt_name,index:ref});

Are references possible in JSON?

Is it possible to assign references to objects in JSON? I have data that looks like this:
[{
name:"name",
Parent:[{
name:"parentName"
Parent:[{
.....//and so on
}]
}]
}]
I need to traverse it in JavaScript and also change the person's name. How can I do this?

Old question but some possibly new answers like JSON Spec and JSON Reference https://json-spec.readthedocs.io/reference.html
[{
"name": "John",
},
{
"name" : "Jack",
"parent": {"$ref": "#/0"}
},
...
]
or possibly better with JSON Path syntax http://goessner.net/articles/JsonPath/
[{
"name": "John",
},
{
"name" : "Jack",
"parent": {"$ref": "$.[?(#.name=='John')]"}
},
...
]

You can't. You can specify the path to the parent as a string and evaluate that at runtime, but as JSON is just strings, integers, arrays, and dictionaries, you can't use references.