We're building a frontend project for a web app that communicates with a backend written by another team. Some of the developers work on both projects, and have better understanding of changes to the backend and response fields coming back.
Recently we had portions of frontend break as they made changes in parts of the app based on changes to the backend without updating the logic in all places. To mitigate this I want to put in place a concept of a mask/template that all response data would be curated through. That way the rest of the members on the team who're not as familiar with the backend can notice/address these bugs.
To do so, I'm considering using JSON Schema. Instead of simply validating, however, I want to parse the backend data through it (removing the fields not present in the schema). This way the developer making changes in the frontend in response to a backend change would also need to update this template, therefore triggering a test failure until all logic using this schema is updated (not just the logic he touched). I'm playing with https://www.npmjs.com/package/jsonschema, but it doesn't seem to have a way to remove excess fields, just test for them.
Within JSON Schema, I can also set additionalProperties flag. However, it has 2 problems with it:
It doesn't cause the validator to remove the fields, it simply dumps them to error array
It needs to be set individually at each nested level, therefore I need to traverse the entire JSON structure, at which point I basically end up writing my own parser/validator.
Perhaps validator is not the right tool for this, but that's all I'm finding when searching for JSON schema parsers. Can someone guide me in the right direction so that I don't reinvent the wheel? It sounds like this functionality is very similar to what a validator already does and I would rather do this processing in the same pass.
Found a validator that does what I want: https://github.com/acornejo/jjv. It has removalAdditional flag that I can set, here is a quick test I did:
var jjv = require('jjv')();
var addressSchema = {
"id": "address",
"type": "object",
"properties": {
"lines": {
"type": "array",
"items": {"type": "string"}
},
"zip": {"type": "string"},
"city": {"type": "string"},
"country": {"type": "string"}
},
"required": ["country"]
};
var schema = {
"id": "person",
"type": "object",
"properties": {
"name": {"type": "string"},
"address": {"$ref": "address"},
"votes": {"type": "integer", "minimum": 1}
}
};
var p = {
"name": "Barack Obama",
"address": {
"lines": [ "1600 Pennsylvania Avenue Northwest" ],
"zip": "DC 20500",
"city": "Washington",
"foobar": "baz",
"country": "USA"
},
"a": {
"b": 1,
"c": 2
},
"votes": "lots",
"stuff": "yes"
};
jjv.addSchema('address', addressSchema);
jjv.addSchema('schema', schema);
jjv.defaultOptions.checkRequired = true;
jjv.defaultOptions.removeAdditional = true;
console.log(jjv.validate('schema', p));
console.log(p);
And a response:
{ validation: { votes: { type: 'integer' } } }
{ name: 'Barack Obama',
address:
{ lines: [ '1600 Pennsylvania Avenue Northwest' ],
zip: 'DC 20500',
city: 'Washington',
country: 'USA' },
votes: 'lots' }
Related
I have an Apollo Client that I'm using to request data from a service. I want to use the data I get in response to create a network of nodes and links e.g.
// Response data:
{
"Team": [
{
"name": "Example Team",
"members": [
{ "name": "Bob" },
{ "name": "Alice" }
]
}
]
}
// Network data:
{
"nodes": [
{ "name": "Example Team" }
{ "name": "Bob" },
{ "name": "Alice" }
],
"links": [
{ "source": "Example Team", "target": "Bob" },
{ "source": "Example Team", "target": "Alice" }
]
}
Historically, before using GraphQL, I would have used Redux to store the munged API response in state and read from there.
Is it appropriate to take a GraphQL result from Apollo and immediately save it back to Apollo local state in a different form so it can be queried by components in that format?
The main problem I foresee is that I think I'd have to query to check if the data I want exists in local state, then make another query if it didn't. In a Redux-world this would be wrapped up inside my store which would then only make the request off to the API if it didn't have the data it needed which 'feels' much cleaner.
In my case this could be solved using Afterware in Apollo Client, see this answer for more information.
It would allow me to munge the returned data into the form I need, and return it in the response alongside the original data.
currently my webAPI method returns to me a long string for a GeoJSON layer that i need to parse through and separate out into mutliple variables that allows for me to create my necessary GeoJSON layer. Can i accomplish this easily through the str.split function or is a there another route that i need to take
The returned string from the webAPI looks like this:
{"type":"FeatureCollection","bbox":[1108864.9370570681,507099.9482870581,1109064.5346246646,507455.89728260477],"features":[{"type":"Feature","geometry":{"type":"Polygon","coordinates":[[[1109035.4443840273,507308.60334444791],[1108893.4509952962,507307.23951566964],[1108893.1628068388,507337.23816090077],[1109035.1562700756,507338.60198967904],[1109035.4443840273,507308.60334444791]]]},"properties":{"twp":"01","map_num":"8.0","sec":"33.00","qtr_sec":"003","map_block":"007","parcel":"008.00","pid":"018.033.00003007008.00","gis_acres":"0.0977962","debbie_ac":"0.10c","misc":"","scan_deeds":"T:\\Images\\Hyperlinked_Deeds\\00203","latitude":"37.2227488237","longitude":"-89.5082666075","name":"O'RILEY PROPERTIES LLC","site_addre":"519 MADISON ST","shape_leng":"344.000009295","shape_area":"4260.00340015","card_ac":"0.00c 0.00d","date":"1/1/0001 12:00:00 AM","ParcelId":"203","ParcelNumber":"0180330000300700800","_hg_layer":"PARCELS"}}
The features section of the string has an array of features but for use in this example i only copied of one specific feature that. the format I'm looking to set this up in is as follows:
var geojsonObject = {
'type': 'FeatureCollection',
'crs': {
'type': 'name',
'properties': {
//'name': 'EPSG:3857'
}
},
'features': [
{
"type": "Feature", "geometry":
{
"type": "Polygon", "coordinates": [[[1109035.4443840273, 507308.60334444791],
[1108893.4509952962, 507307.23951566964],
[1108893.1628068388, 507337.23816090077],
[1109035.1562700756, 507338.60198967904],
[1109035.4443840273, 507308.60334444791]]]
, "properties": {
"twp": "01",
"map_num": "8.0",
"sec": "33.00",
"qtr_sec": "003",
"map_block": "007",
"parcel": "008.00",
"pid": "018.033.00003007008.00",
"gis_acres": "0.0977962",
"debbie_ac": "0.10c",
"misc": "",
"scan_deeds": "T:\\Images\\Hyperlinked_Deeds\\00203",
"latitude": "37.2227488237",
"longitude": "-89.5082666075",
"name": "O'RILEY PROPERTIES LLC",
"site_addre": "519 MADISON ST",
"shape_leng": "344.000009295",
"shape_area": "4260.00340015",
"card_ac": "0.00c 0.00d",
"date": "1/1/0001 12:00:00 AM",
"ParcelId": "203",
"ParcelNumber": "0180330000300700800",
"_hg_layer": "PARCELS"
}
}
}
Again the features section of this code in my example just shows one specific feature but i will actually have several features that i need to display.
is parsing through this long string the easiest route to go or do i need to restructure the string that my WebAPI actually sends back to me
For a Chrome app, wich stores data in IndexedDB, i have a object like this:
var simplifiedOrderObject = {
"ordernumber": "123-12345-234",
"name": "Mr. Sample",
"address": "Foostreet 12, 12345 Bar York",
"orderitems": [
{
"item": "brush",
"price": "2.00"
},
{
"item": "phone",
"price": "30.90"
}
],
"parcels": [
{
"service": "DHL",
"track": "12345"
},
{
"service": "UPS",
"track": "3254231514"
}
]
}
If i store the hole object in an objectStore, can i use an index for "track", which can be contained multiple times in each order object?
Or is it needed or possibly better/faster to split each object into multiple objectStores like know from relational DBs:
order
orderitem
parcel
The solution should also work in a fast way with 100.000 or more objects stored.
Answering my own question: I have made some tests now. It looks like it is not possible to do this with that object in only 1 objectStore.
An other example object which would work:
var myObject = {
"ordernumber": "123-12345-234",
"name": "Mr. Sample",
"shipping": {"method": "letter",
"company": "Deutsche Post AG" }
}
Creating an index will be done by:
objectStore.createIndex(objectIndexName, objectKeypath, optionalObjectParameters);
With setting objectKeypath it is possible to address a value in the main object like "name":
objectStore.createIndex("name", "name", {unique: false});
It would also be possible to address a value form a subobject of an object like "shipping.method":
objectStore.createIndex("shipping", "shipping.method", {unique: false});
BUT it is not possible to address values like the ones of "track", which are contained in objects, stored in an array. Even something like "parcels[0].track" to get the first value as index does not work.
Anyhow, it would be possible to index all simple elements of an array (but not objects).
So the following more simple structure would allow to create an index entry for each parcelnumber in the array "trackingNumbers":
var simplifiedOrderObject = {
"ordernumber": "123-12345-234",
"name": "Mr. Sample",
"address": "Foostreet 12, 12345 Bar York",
"orderitems": [
{
"item": "brush",
"price": "2.00"
},
{
"item": "phone",
"price": "30.90"
}
],
"trackingNumbers": ["12345", "3254231514"]
}
when creating the index with multiEntry set to true:
objectStore.createIndex("tracking", "trackingNumbers", {unique: false, multiEntry: true});
Anyhow, the missing of the possibility to index object values in arrays, makes using indexedDB really unneeded complicated. It's a failure in design. This forces the developer to do things like in relational DBs, while lacking all the possibilities of SQL. Really bad :(
Previously I was using a json file with the following format:
[{"lat":43.788458853157117,"lng":-79.282781549043008,"category":"volunteer","name":"Rita","url":"", "description":"xxx is a member of 13"},{"lat":43.7,"lng":-79.4,"category":"organization","name":"TCAN","url":"http://tcan.ca","description":"Lorem ipsum"}]
Now I am attempting to generate the json file from a Drupal site and am getting the following structure. How can I reference the lowest level fields. I have looked at examples using d3.net but have not found any that apply.
{
"organizations": [
{
"member": {
"field_category": "organization",
"body": "A network of organizations in Toronto devoted to climate change mitigation and adaptation.",
"URL": "xxx.ca",
"title": "Toronto Climate Action Network",
"field_lat": 43.7,
"field_long": -79.4
}
},
{
"member": {
"field_category": "abc",
"body": "xxx.",
"URL": "",
"title": "yyy",
"field_lat": 43.7,
"field_long": -79.28
}
}
]
}
Assuming that your data is stored in the variable data:
var bottom = data.organizations.map(function(d) { return d.member; });
Consider a JSON like this:
[{
"type": "person",
"name": "Mike",
"age": "29"
},
{
"type": "person",
"name": "Afshin",
"age": "21"
},
{
"type": "something_else",
"where": "NY"
}]
I want to search in the JSON value with a key (for example type='person') and then select a whole object of matched item in JSON. For example when I search for type='person' I expect this value:
[{
"type": "person",
"name": "Mike",
"age": "29"
},
{
"type": "person",
"name": "Afshin",
"age": "21"
}]
Because it's a really big JSON value, I don't want to do a brute-force search in all nodes, so I think the only way is using Regular Expressions but I don't know how can I write a Regex to match something like above.
I'm using NodeJs for the application.
Using underscore.js#where:
var results = _(yourObject).where({ type: 'person' })
If your data set is very very big [e.g. 10k or so], consider filtering / paginating stuff server side.
Plain javascript :
var results = dataset.filter(function(p) {
if(p.type == 'person')
return true;
});
If the requirement is to scan multiple times through the collection, the following one time construction overhead might be of worth.
Use hashing based on values of type.Convert the current data structure to hash map.
var hashMap ={
};
hashMap['person'] =[{},{}];
Hope this helps you.
Use
$.grep(jsonarrayobj,function(n, i){
if(n.type==="person")
{}
})