Map dependencies in JSON structure - javascript

I am currently building a web tool which enables the user to generate a package of options in the form of a String. To select which options he wants he uses a form with different inputs (radio, checkbox) which is generated from a dictionary.json that currently holds all available options and their codes in the following format (subject to change):
[
{
"id": 0001,
"title":"foo",
"type":"radio",
"options":[
{
"bar":"",
"foo":"489",
"foobar":"489+490"
}
]
},
{
"id": 0002,
"title":"something",
"type":"check",
"options":[
{
"everything":"M016",
"evenmore":"M139"
}
]
},
[...]
As you can see it is basically a small database. The problem is that the options depend on each other so if foo is foobar it might determine that something is definitely evenmore and can NOT be changed to everything. How would I map these dependencies in the dictionary.json so that the generated form can reliably grey out options that are determined by other choices?
The structure has to be flexible so new dependencies can be inserted and would generate the new form reliably or validate existing outputs against them. There could also be options that depend on multiple other options. I can't think of a smart way of saving these dependencies and I wonder if JSON is the right format to go with here.
Any tips or ideas are welcome. Thanks!

You could try to save every option as one object which stores all the options which will be excluded if that option is selected.
So your JSON could look like the following:
[
{
"id": 0001,
"title":"foo",
"type":"radio",
"options":[
{
"bar":"",
"excludes": []
},
{
"foo":"489",
"excludes": []
},
{
"foobar":"489+490",
"excludes": [
{
"id": 0002,
"options": [
"everything"
],
},
{
"id": 0003,
"options": [
"apple",
"cherry"
],
},
]
}
]
},
{
"id": 0002,
"title":"something",
"type":"check",
"options":[
{
"everything":"M016",
"excludes": []
},
{
"evenmore":"M139",
"excludes": []
}
]
},
[...]
Everytime an option is selected you would have to check their excludes list and diable all those options for the specific fields.
To improve the usability you could check there is only one option left for a field, select this option and then disable the whole field.
EDIT:
Additionally you could save a isExcludedBy field to each of the options.
The everything option of id 0002 would then look like this:
"isExcludedBy": [
"id": 0001,
"options": [
"foobar"
]
]
This would be kind of redundant, but depending on what you want your UI to show, it could save you some computing time.

A possible simple solution (which answers your question):
// dictionary.json
{
"options": [
{
"id": 0001,
"title":"foo",
"type":"radio",
"options":[
{
"bar":"",
"foo":"489",
"foobar":"489+490"
}
]
}
// etc.; same as before
],
// this is it:
"dependencies": [
[["0001", "foobar"], ["0002", "evenmore"]],
]
}
dependencies here consist of pairs of [path to option in options that implies another option, path to the implied option].
You could make a Map data structure out of this directly (the implying options are keys, the implied are values).
This assumes that one option can imply only one other option (but it still allows for options that depend on multiple other options).
You could of course easily extend that like so:
[["0001", "foobar"], [["0002", "evenmore"], ["0003", "namaste"]]]
This would mean that "0001"/"foobar" implies both "0002"/"evenmore" and "0003"/"namaste". But perhaps YAGNI. :)

One way to approach this is to model the domain you're actually expressing, and generate the form based on that. For example, we know that apartments have street numbers, and apartment numbers, whereas houseboats don't even have streets.
{
"dwelling": {
"type": "houseboat",
"latitude": null,
"longitude": null,
}
}
or
{
"dwelling": {
"type": "apartment",
"street": "Beech St.",
"street_number": 123,
"apartment_number": 207,
}
}
By modelling the domain rather than the form, you can write rules that apply beyond the form, and you won't have to develop a mini-language for expressing form dependencies.

Related

How to model the rows in getData() since i have nested JSON data?

I want to display these fields :name, age, addresses_id, addresses_city, addresses_primary for each person into data studio.
My JSON data
{
"data": [
{
"name": "Lio",
"age": 30,
"addresses": [
{
"id": 7834,
"city": "ML",
"primary": 1
},
{
"id": 5034,
"city": "MM",
"primary": 1
}
]
},
{
"name": "Kali",
"age": 41,
"addresses": [
{
"id": 3334,
"city": "WK",
"primary": 1
},
{
"id": 1730,
"city": "DC",
"primary": 1
}
]
},
...
]
}
there is no problem if i don't render the addresses field
return {
schema: requestedFields.build(),
rows: rows
};
//rows:
/*
"rows": [
{
"values": ["Lio", 30]
},
{
"values": ["Kali", 41]
},
...
]
*/
The problem is
I'm not able to model the nested JSON data in Google Data Studio. I
have the problem exactly in the "addresses" field.
Could anyone tell me what format should be for the rows in this case?
As you already know, for each name of your dataset, you clearly have more than one row (one person has multiple addresses). Data Studio only accepts a single data for each field, since arrays are not supported at all. So you need to work on this.
There are some ways to solve this, but always keep in mind that:
getSchema() should return all available fields for your connector (the order doesn't really matter, since Data Studio always sort alphabetically the available fields)
getData() should return a list of values. But here the order is relevant: it should be the same as the parameter passed to getData() (which means the results should be dynamic, sometimes you'll return all values, sometimes not, and the order may change).
Solution 1: Return multiple rows per record
Since you can produce multiple rows for each name, just do it.
To achieve this, your field definition (=getSchema()) should include fields address_id, address_city and address_primary (you can also add address_order if you need to know the position of the address in the list).
Supposing getData() is called with all fields in the same order they were discribed, rows array should look like this:
"rows": [
{
"values": ["Lio", 30, "7834", "ML", 1]
},
{
"values": ["Lio", 30, "5034", "MM", 1]
},
{
"values": ["Kali", 41, "3334", "WK", 1]
},
{
"values": ["Kali", 41, "1730", "DC", 1]
},
...
]
IMO, this is the best solution for your data.
Solution 2: Return one address only, ignoring others
If you prefer one row per person, you can get one of the addresses and display only it (usually the main/primary address, or the first one).
To achieve this, your field definition (=getSchema()) should include fields address_id, address_city and address_primary.
Supposing getData() is called with all fields in the same order they were discribed, rows array should look like this:
"rows": [
{
"values": ["Lio", 30, "7834", "ML", 1]
},
{
"values": ["Kali", 41, "3334", "WK", 1]
},
...
]
Solution 3: Return all addresses, serialized in a field
This is helpful if you really need all information but do not want a complex scheme.
Just create a field called addresses in your field definition (=getSchema()) and write the JSON there as a string (or any other format you want).
Supposing getData() is called with all fields in the same order they were discribed, rows array should look like this:
"rows": [
{
"values": ["Lio", 30, "[{\"id\": 7834, \"city\": "ML", \"primary\": 1}, {\"id\": 5034, \"city\": \"MM\", \"primary\": 1}]"]
},
{
"values": ["Kali", 41, "[{\"id\": 3334, \"city\": \"WK\", \"primary\": 1}, {\"id\": 1730, \"city\": \"DC\", \"primary\": 1}]"]
},
...
]
This solution may appear senseless, but it is possible to interact with this data later in DataStudio using REGEX if really needed.
Solution 4: Create a different field for each address
If you're sure all records has a maximum number of addresses (in you example, both names have 2 addresses, for example), you can create multiple fields.
Your field definition (=getSchema()) should include fields address_id1, address_city1, address_primary1, address_id2, ... address_primaryN.
I wouldn't explain how rows should look like in this situation, but it is not hard to guess with the other examples.

Elasticsearch: can I avoid enabling fielddata on text fields?

I'm trying to get the latest records, grouped by the field groupId, which is a String like "group_a".
I followed the accepted answer of this question, but I've got the following error message:
Fielddata is disabled on text fields by default. Set fielddata=true on [your_field_name] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory.
In the Elasticsearch docs is written:
Before you enable fielddata, consider why you are using a text field for aggregations, sorting, or in a script. It usually doesn’t make sense to do so.
I'm using a text field, because groupId is a String. Does it make sense to set fielddata: true if I want to group by it?
Or are there alternatives?
Using "field": "groupId.keyword" (suggested here) didn't work for me.
Thanks in advance!
The suggest answer with .keyword is the correct one.
{
"aggs": {
"group": {
"terms": {
"field": "groupId.raw"
},
"aggs": {
"group_docs": {
"top_hits": {
"size": 1,
"sort": [
{
"timestamp (or wathever you want to sort)": {
"order": "desc"
}
}
]
}
}
}
}
}
}
with a mapping like that:
"groupId": {
"type": "text",
"fields": {
"raw": {
"type": "keyword"
}
}
}

Define JSON-LD #context to join/split values?

I'd like to use the expand and compact methods of the jsonld.js library to translate data from various sources into a common format for processing. If I take a source JSON document, add a #context to it, then pass it through the expand method I'm able to get the common format that I need.
The use case that I haven't been able to find a solution for is when multiple values need to be merged. For example, schema.org defines a PostalAddress with a single field for the streetAddress, but many systems store the street address as separate values (street number, street name, street direction...). To translate the incoming data to the schema.org format I need a way to indicate in my #context that multiple fields make up the streetAddress, in the correct order.
Compacted Document
{
"#context": {
"displaName": "http://schema.org/name",
"website": "http://schema.org/homepage",
"icon": "http://schema.org/image",
"streetNumber": "http://schema.org/streetAddress"
},
"displaName": "John Doe",
"website": "http://example.com/",
"icon": "http://example.com/images/test.png",
"streetNumber": "123",
"streetName": "Main St",
"streetDirection": "South"
}
Expanded Document
{
"http://schema.org/name":[
{
"#value":"John Doe"
}
],
"http://schema.org/image":[
{
"#value":"http://example.com/images/test.png"
}
],
"http://schema.org/streetAddress":[
{
"#value":"123"
}
],
"http://schema.org/homepage":[
{
"#value":"http://example.com/"
}
]
}
I've reviewed all of the JSON-LD specs that I could find and haven't been able to locate anything that indicates a way to split or concatenate values using the #context.
Is anyone aware of a way to map multiple values into one context property, in the correct order, and possibly add whitespace between the values. I also need to find a solution for the reverse scenario, where I need to split one field into multiple values, in the correct order.
Note: Even if I map all three properties to streetAddress, the values will all be included in the array, but there's no guarantee they'll be in the correct order.
One possible way to achieve this is to use a single array field for your address containing the ordered address components (i.e. ["number", "direction", "name"]). Then in the #context you can specify the address with #container: #list, which will ensure the address components are correctly ordered.
So the compacted document would be:
{
"#context": {
"displaName": "http://schema.org/name",
"website": "http://schema.org/homepage",
"icon": "http://schema.org/image",
"address": {
"#id": "http://schema.org/streetAddress",
"#container": "#list"
}
},
"displaName": "John Doe",
"website": "http://example.com/",
"icon": "http://example.com/images/test.png",
"address": ["123", "South", "Main St"]
}
And the expanded one would be
{
"http://schema.org/streetAddress": [
{
"#list": [
{
"#value": "123"
},
{
"#value": "South"
},
{
"#value": "Main St"
}
]
}
],
"http://schema.org/name": [
{
"#value": "John Doe"
}
],
"http://schema.org/image": [
{
"#value": "http://example.com/images/test.png"
}
],
"http://schema.org/homepage": [
{
"#value": "http://example.com/"
}
]
}
I posted an issue on the jsonld.js Github repository. According to #dlongley, the original creator of the jsonld.js library, it's not possible to manipulate properties in this manor, using standard JSON-LD.
https://github.com/digitalbazaar/jsonld.js/issues/115

Update ReactJS state array

I have a ReactJS component that renders a list of orders.
I get orders from a REST API. The format of the data is the following:
{
"count": 2,
"orders": [
{
"order_number": 55981,
"customer_number": 24742
},
{
"order_number": 55980,
"customer_number": 24055
}
]
}
Each order can have a list of items. When I click on an order, I get the list of items in the following format:
{
"count": 2,
"items": [
{
"name": "Green pillow",
"status": "pending"
},
{
"name": "Red pillow",
"status": "delivered"
}
]
}
The orders list is refreshed automatically and can change any time, so I store the orders list in this.state which gets updated via ajax.
this.state looks like this:
{
"orders": [
{
"order_number": 55981,
"customer_number": 24742
},
{
"order_number": 55980,
"customer_number": 24055
}
]
}
My problem is that I would like that, when I click on an order, the state gets updated so that the clicked order contains the items associated to that order. The order list would look like this after clicking on an item:
{
"count": 2,
"orders": [
{
"order_number": 55981,
"customer_number": 24742,
"items": [
{
"name": "Green pillow",
"status": "pending"
}
]
},
{
"order_number": 55980,
"customer_number": 24055
}
]
}
How can I add items to a specific order using this.setState()? The problem is that setState seem to update data using keys, but my orders are in an array. I can probably copy the whole array and put the items key inside, but that seems overkill.
Am I taking the wrong approach?
I'm not entirely sure I got your question, but I think that what you're trying to achieve is add new orders to an array (push) which is located in your state.
If that's the case, you should something like this:
// since you're orders it's not a plain array, you will have
// to deep clone it (e.g. with lodash)
let orders = _.clone(this.state.orders);
orders.push(newOrder);
this.setState(orders);
Why cloning the state before changing it is important?
Immutability comes with some great properties (like easy equality comparison) and the React team is aiming towards that direction to improve performance more and more.
As of React 0.13 mutating state without calling this.setState will trigger a warning, and it will break entirely at some point in React's future (see the docs)
Hope this helps
Make a copy of the current state, modify the copy and use it as the new state.
This is just a simple example. You might want to add some caching logic so you don't have to retrieve the order's item again and again when a user click on the same order multiple times.
var updatedOrder = _.clone(this.state.orders);
updatedOrder[0]["items"] = [ { "name": "Foo", "status":"bar" } ];
this.setState({
orders: updatedOrder
});

MongoDb: How to get a field (sub document) from a document?

Consider this example collection:
{
"_id:"0,
"firstname":"Tom",
"children" : {
"childA":{
"toys":{
'toy 1':'batman',
'toy 2':'car',
'toy 3':'train',
}
"movies": {
'movie 1': "Ironman"
'movie 2': "Deathwish"
}
},
"childB":{
"toys":{
'toy 1':'doll',
'toy 2':'bike',
'toy 3':'xbox',
}
"movies": {
'movie 1': "Frozen"
'movie 2': "Barbie"
}
}
}
}
Now I would like to retrieve ONLY the movies from a particular document.
I have tried something like this:
movies = users.find_one({'_id': 0}, {'_id': 0, 'children.ChildA.movies': 1})
However, I get the whole field structure from 'children' down to 'movies' and it's content. How do I just do a query and retrieve only the content of 'movies'?
To be specific I want to end up with this:
{
'movie 1': "Frozen"
'movie 2': "Barbie"
}
The problem here is your current data structure is not really great for querying. This is mostly because you are using "keys" to actually represent "data points", and while it might initially seem to be a logical idea it is actually a very bad practice.
So rather than do something like assign "childA" and "childB" as keys of an object or "sub-document", you are better off assigning these are "values" to a generic key name in a structure like this:
{
"_id:"0,
"firstname":"Tom",
"children" : [
{
"name": "childA",
"toys": [
"batman",
"car",
"train"
],
"movies": [
"Ironman"
"Deathwish"
]
},
{
"name": "childB",
"toys": [
"doll",
"bike",
"xbox",
],
"movies": [
"Frozen",
"Barbie"
]
}
]
}
Not the best as there are nested arrays, which can be a potential problem but there are workarounds to this as well ( but later ), but the main point here is this is a lot better than defining the data in "keys". And the main problem with "keys" that are not consistently named is that MongoDB does not generally allow any way to "wildcard" these names, so you are stuck with naming and "absolute path" in order to access elements as in:
children -> childA -> toys
children -> childB -> toys
And that in a nutshell is bad, and compared to this:
"children.toys"
From the sample prepared above, then I would say that is a whole lot better approach to organizing your data.
Even so, just getting back something such as a "unique list of movies" is out of scope for standard .find() type queries in MongoDB. This actually requires something more of "document manipulation" and is well supported in the aggregation framework for MongoDB. This has extensive capabilities for manipulation that is not present in the query methods, and as a per document response with the above structure then you can do this:
db.collection.aggregate([
# De-normalize the array content first
{ "$unwind": "$children" },
# De-normalize the content from the inner array as well
{ "$unwind": "$children.movies" },
# Group back, well optionally, but just the "movies" per document
{ "$group": {
"_id": "$_id",
"movies": { "$addToSet": "$children.movies" }
}}
])
So now the "list" response in the document only contains the "unique" movies, which corresponds more to what you are asking. Alternately you could just $push instead and make a "non-unique" list. But stupidly that is actually the same as this:
db.collection.find({},{ "_id": False, "children.movies": True })
As a "collection wide" concept, then you could simplify this a lot by simply using the .distinct() method. Which basically forms a list of "distinct" keys based on the input you provide. This playes with arrays really well:
db.collection.distinct("children.toys")
And that is essentially a collection wide analysis of all the "distinct" occurrences for each"toys" value in the collection, and returned as a simple "array".
But as for you existing structure, it deserves a solution to explain, but you really must understand that the explanation is horrible. The problem here is that the "native" and optimized methods available to general queries and aggregation methods are not available at all and the only option available is JavaScript based processing. Which even though a little better through "v8" engine integration, is still really a complete slouch when compared side by side with native code methods.
So from the "original" form that you have, ( JavaScript form, functions have to be so easy to translate") :
db.collection.mapReduce(
// Mapper
function() {
var id this._id;
children = this.children;
Object.keys(children).forEach(function(child) {
Object.keys(child).forEach(function(childKey) {
Object.keys(childKey).forEach(function(toy) {
emit(
id, { "toys": [children[childkey]["toys"][toy]] }
);
});
});
});
},
// Reducer
function(key,values) {
var output = { "toys": [] };
values.forEach(function(value) {
value.toys.forEach(function(toy) {
if ( ouput.toys.indexOf( toy ) == -1 )
output.toys.push( toy );
});
});
},
{
"out": { "inline": 1 }
}
)
So JavaScript evaluation is the "horrible" approach as this is much slower in execution, and you see the "traversing" code that needs to be implemented. Bad news for performance, so don't do it. Change the structure instead.
As a final part, you could model this differently to avoid the "nested array" concept. And understand that the only real problem with a "nested array" is that "updating" a nested element is really impossible without reading in the whole document and modifying it.
So $push and $pull methods work fine. But using a "positional" $ operator just does not work as the "outer" array index is always the "first" matched element. So if this really was a problem for you then you could do something like this, for example:
{
"_id:"0,
"firstname":"Tom",
"childtoys" : [
{
"name": "childA",
"toy": "batman"
}.
{
"name": "childA",
"toy": "car"
},
{
"name": "childA",
"toy": "train"
},
{
"name": "childB",
"toy": "doll"
},
{
"name": "childB",
"toy": "bike"
},
{
"name": "childB",
"toy": "xbox"
}
],
"childMovies": [
{
"name": "childA"
"movie": "Ironman"
},
{
"name": "childA",
"movie": "Deathwish"
},
{
"name": "childB",
"movie": "Frozen"
},
{
"name": "childB",
"movie": "Barbie"
}
]
}
That would be one way to avoid the problem with nested updates if you did indeed need to "update" items on a regular basis rather than just $push and $pull items to the "toys" and "movies" arrays.
But the overall message here is to design your data around the access patterns you actually use. MongoDB does generally not like things with a "strict path" in the terms of being able to query or otherwise flexibly issue updates.
Projections in MongoDB make use of '1' and '0' , not 'True'/'False'.
Moreover ensure that the fields are specified in the right cases(uppercase/lowercase)
The query should be as below:
db.users.findOne({'_id': 0}, {'_id': 0, 'children.childA.movies': 1})
Which will result in :
{
"children" : {
"childA" : {
"movies" : {
"movie 1" : "Ironman",
"movie 2" : "Deathwish"
}
}
}
}

Categories

Resources