I would like to sum up two objects into one total object and get a percent change of the two values (add then divide by # of values). I'm having some issues applying this logic. The data is dynamic so there could be more than two objects in the same array.
My preferred method is JavaScript but jQuery is perfectly fine as well if it's simpler! Below is an example and here is a jsfiddle - http://jsfiddle.net/4z63cwhz/4/
Response Data:
$scope.data = [
{
"July":{
"params":{
"frequency":"Monthly",
"category":"Overview",
"year":"2015",
"month":"July"
},
"subcategory":{
"us total":{
"interaction rate":0.51,
"digital interaction rate":8.33,
"impressions":500,
"interactions":256
},
"discover me":{
"digital u.s. discover site sessions":50
},
"me com":{
"total me.com site sessions":50
}
},
"action":"new"
},
"August":{
"params":{
"frequency":"Monthly",
"category":"Overview",
"year":"2015",
"month":"August"
},
"subcategory":{
"us total":{
"interaction rate":0.51,
"digital interaction rate":8.33,
"impressions":500,
"interactions":256
},
"discover me":{
"digital u.s. discover site sessions":50
},
"me com":{
"total me.com site sessions":50
}
},
"action":"new"
}
}
]
JS:
$scope.calculate = function(){
var sumObj = {};
var local = $scope.data;
for(var i in local){
var obj = local[i];
for(var i2 in obj){
for(var i3 in obj[i2].subcategory){
//do the calculation here.
console.log(i3);
}
}
}
};
The sumObj I was hoping to get would be similar to below:
{
"us total":{
"interaction rate":{
total: total#,
'%change': %change
},
"digital interaction rate":{
total: total#,
'%change': %change
},
"impressions":{
total: total#,
'%change': %change
},
"interactions":{
total: total#,
'%change': %change
}
},
"discover me":{
"digital u.s. discover site sessions":{
total: total#,
'%change': %change
}
},
"me com":{
"total me.com site sessions":{
total: total#,
'%change': %change
}
}
}
Thanks in advance!
Well if the json objects all repeat the same format then can't you just do:
var text= {};
local[current].attributeString (some calculation) local[next].attributeString += text;
where current = i and next = i+1.
Then you can use:
var obj = JSON.parse(text);
to convert your String into a json object. I'd probably use some type of key/value pair instead of a string but this may help reduce some of your loops.
Related
I am working on an offer letter template that will replace/modify Dynamic Data Points like Name, Address, Role, Salary, etc based on the candidate selected from a list of candidates. There is a fixed syntax for a dynamic data points i.e they will be enclosed within <<>>, for example :
Welcome to the family, <<Name>>
You will be paid <<Salary>> for the duration of your employment.
In other words, these few data points will change by selecting the candidate we want to offer the job and the rest of the template will remain the same. Here is a demo to help you understand.
This is a dummy array I have created with 1 template, In the real-world app, I can have many templates with different clauseNames, so I am looking for a permanent fix.
.ts file, Template List :
[{
templateId: 1,
templateName: "Offer",
clauses: [
{
clauseName: "Introduction",
clauseId: 1,
texts: [
{
text: "Hello <<Name>>, Welcome to the Machine",
textId: 1,
}]
},
{
clauseName: "Address",
clauseId: 2,
texts: [
{
text: "<<Address>>",
textId: 2,
}]
},
{
clauseName: "Date Of Joining",
clauseId: 3,
texts: [
{
text: "You can join us on <<DateOfJoining>>",
textId: 3,
}]
},
]
}]
and here is the candidate list,
candidateList = [
{ name: "Simba", address: "Some Random Cave" },
{ name: "Doe John", address: "line 4, binary avenue, Mobo" },
{ name: "B Rabbit", address: "8 mile road, Detroit" },
{ name: "Peter Griffin", address: "Spooner Street" },
{ name: "Speedy Gonzales", address: "401, hole 34, Slyvester Cat Road" },
{ name: "Morty", address: "Time Machine XYZ" },
{ name: "Brock", address: "pokeball 420, Medic center" },
]
You can use regular expressions to replace those placeholders such as:
var result = text.text.replace(/\<\<(.*?)\>\>/g, function(match, token) {
return candidate[token.toLowerCase()];
});
One way to incorporate this to your display is by creating a property that returns the formatted text.
I have updated your stackblitz here.
Take a look at this demo
I have modified the logic in below method:
showTeplate(name,address,doj) {
this.clauseList = [];
for (let a of this.templateList) {
if (a.clauses != null) {
for (let cl of a.clauses) {
const tempObj = JSON.parse(JSON.stringify(cl));
tempObj.texts.forEach(textObj => {
textObj.text = textObj.text.replace("<<Name>>",name);
textObj.text = textObj.text.replace("<<Address>>",address);
textObj.text = textObj.text.replace("<<DateOfJoining>>",doj);
})
this.clauseList.push(tempObj)
}
}
}
console.log("Clause list", this.clauseList)
}
I'm using Cloudant's map reduce functionality and I want to find how many events (count of events object) the specific user with name (input from user) has attended for a date range (input from user).
I have docs that look like below.
{
user: {
name: 'peter pan'
},
startEventDateTime: <timestamp>,
endDateDateTime: <timestamp>,
events: [
{
name: 'La la land',
text: 'more info'
},
{
name: 'La la land',
text: 'more info'
}
]
}
Above means, user attended 2 events between between that start and end time. There are many documents for the same user for a different date range too with the events attended list.
How can I achieve this in Cloudant map reduce?
My Attempt:
unable to get map correctly. I can filter by name by doing
map:
function (doc) {
emit([doc.user, doc.events, startEventDateTime, endDateDateTime], doc)
}
reduce:
function (keys, values, rereduce) {
if (rereduce) {
return sum(values);
} else {
return values.length;
}
}
I would suggest considering a different format for your documents. Instead of having a user document with a list of events, make a separate document for each event, timestamped for the time at which it happened, such as:
{
"_id": "c48ee0881ce7c5d39243d2243d2e63cb",
"_rev": "1-c2f71fba5f09b129f1db20785f2429b2",
"user": "bob",
"datetime": "Thu 30 Nov 2017 09:46:02 GMT",
"event": {
"name": "lalaland",
"text": "more info"
}
}
Then you can rely on MapReduce to pick out date ranges per user. Here's a map function that does just that:
function (doc) {
if (doc && doc.user && doc.datetime) {
var when = new Date(Date.parse(doc.datetime));
emit([doc.user, when.getFullYear(), when.getMonth(), when.getDay()], 1);
}
}
and using the built-in reduce _sum. You can now use key ranges to slice the data. Say you want the events attended by user bob in Aug, 2017:
curl 'https://ACCT.cloudant.com/DBNAME/_design/DDOC/_view/VIEWNAME?startkey=\["bob", 2017, 7\]&endkey=\["bob", 2017, 8]&group=true&inclusive_end=false&reduce=true'
{
"rows": [
{
"key": [
"bob",
2017,
7,
4
],
"value": 1
}
]
}
I have a quick question on map-reduce with mongodb. I have this following document structure
{
"_id": "ffc74819-c844-4d61-8657-b6ab09617271",
"value": {
"mid_tag": {
"0": {
"0": "Prakash Javadekar",
"1": "Shastri Bhawan",
"2": "Prime Minister's Office (PMO)",
"3": "Narendra Modi"
},
"1": {
"0": "explosion",
"1": "GAIL",
"2": "Andhra Pradesh",
"3": "N Chandrababu Naidu"
},
"2": {
"0": "Prime Minister",
"1": "Narendra Modi",
"2": "Bharatiya Janata Party (BJP)",
"3": "Government"
}
},
"total": 3
}
}
when I am doing my map reduce code on this collection of documents I want to specify total as the sort field in this command
db.ana_mid_big.mapReduce(map, reduce,
{
out: "analysis_result",
sort: {"value.total": -1}
}
);
But this does not seem to work. How can I specify a key which is nested for sorting? Please help.
----------------------- EDIT ---------------------------------
as per the comments I am posting my whole problem here. I have started with a collection with a little more than 3.5M documents (this is just an old snap shot of the live one, which already crossed 5.5 M) which looks like this
{
"_id": ObjectId("53b394d6f9c747e33d19234d"),
"autoUid": "ffc74819-c844-4d61-8657-b6ab09617271"
"createDate": ISODate("2014-07-02T05:12:54.171Z"),
"account_details": {
"tag_cloud": {
"0": "FIFA World Cup 2014",
"1": "Brazil",
"2": "Football",
"3": "Argentina",
"4": "Belgium"
}
}
}
So, there can be many documents with the same autoUid but with different (or partially same or even same) tag_cloud.
I have written this following map-reduce to generate an intermediate collection which looks like the one at the start of the question. So, evidently that is collection of all the tag_clouds belongs to one person in a single document. To achieve this the MR code i used looks like the following
var map = function(){
final_val = {
tag_cloud: this.account_details.tag_cloud,
total: 1
};
emit(this.autoUid, final_val)
}
var reduce = function(key, values){
var fv = {
mid_tags: [],
total: 0
}
try{
for (i in values){
fv.mid_tags.push(values[i].tag_cloud);
fv.total = fv.total + 1;
}
}catch(e){
fv.mid_tags.push(values)
fv.total = fv.total + 1;
}
return fv;
}
db.my_orig_collection.mapReduce(map, reduce,
{
out: "analysis_mid",
sort: {createDate: -1}
}
);
Here comes problem Number-1 when somebody has more than one record it obeys reduce function. But when somebody has only one instead of naming it "mid_tag" it retains the name "tag_cloud". I understand that there is some problem with the reduce code but can not find what.
Now I want to reach to a final result which looks like
{"_id": "ffc74819-c844-4d61-8657-b6ab09617271",
"value": {
"tags": {
"Prakash Javadekar": 1,
"Shastri Bhawan": 1,
"Prime Minister's Office (PMO)": 1,
"Narendra Modi": 2,
"explosion": 1,
"GAIL": 1,
"Andhra Pradesh": 1,
"N Chandrababu Naidu": 1,
"Prime Minister": 1,
"Bharatiya Janata Party (BJP)": 1,
"Government": 1
}
}
Which is finally one document for each person representing the tag density they have used. The MR code I am trying to use (not tested yet) looks like this---
var map = function(){
var val = {};
if ("mid_tags" in this.value){
for (i in this.value.mid_tags){
for (j in this.value.mid_tags[i]){
k = this.value.mid_tags[i][j].trim();
if (!(k in val)){
val[k] = 1;
}else{
val[k] = val[k] + 1;
}
}
}
var final_val = {
tag: val,
total: this.value.total
}
emit(this._id, final_val);
}else if("tag_cloud" in this.value){
for (i in this.value.tag_cloud){
k = this.value.tag_cloud[i].trim();
if (!(k in val)){
val[k] = 1;
}else{
val[k] = val[k] + 1;
}
}
var final_val = {
tag: val,
total: this.value.total
}
emit(this._id, final_val);
}
}
var reduce = function(key, values){
return values;
}
db.analysis_mid.mapReduce(map, reduce,
{
out: "analysis_result"
}
);
This last piece of code is not tested yet. That is all I want to do. Please help
Your PHP background appears to be showing. The data structures you are representing are not showing arrays in typical JSON notation, however there are noted calls to "push" in your mapReduce code that at least in your "interim document" the values are actually arrays. You seem to have "notated" them the same way so it seems reasonable to presume they are.
Actual arrays are your best option for storage here, especially considering your desired outcome. So even if they do not, your original documents should look like this, as they would be represented in the shell:
{
"_id": ObjectId("53b394d6f9c747e33d19234d"),
"autoUid": "ffc74819-c844-4d61-8657-b6ab09617271"
"createDate": ISODate("2014-07-02T05:12:54.171Z"),
"account_details": {
"tag_cloud": [
"FIFA World Cup 2014",
"Brazil",
"Football",
"Argentina",
"Belgium"
]
}
}
With documents like that or if you change them to be like that, then your right tool for doing this is the aggregation framework. That works in native code and does not require JavaScript interpretation, hence it is much faster.
An aggregation statement to get to your final result is like this:
db.collection.aggregate([
// Unwind the array to "de-normalize"
{ "$unwind": "$account_details.tag_cloud" },
// Group by "autoUid" and "tag", summing totals
{ "$group": {
"_id": {
"autoUid": "$autoUid",
"tag": "$account_details.tag_cloud"
},
"total": { "$sum": 1 }
}},
// Sort the results to largest count per user
{ "$sort": { "_id.autoUid": 1, "total": -1 }
// Group to a single user with an array of "tags" if you must
{ "$group": {
"_id": "$_id.autoUid",
"tags": {
"$push": {
"tag": "$_id.tag",
"total": "$total"
}
}
}}
])
Slightly different output, but much simpler to process and much faster:
{
"_id": "ffc74819-c844-4d61-8657-b6ab09617271",
"tags": [
{ "tag": "Narendra Modi", "total": 2 },
{ "tag": "Prakash Javadekar", "total": 1 },
{ "tag": "Shastri Bhawan", "total": 1 },
{ "tag": "Prime Minister's Office (PMO)", "total": 1 },
{ "tag": "explosion", "total": 1 },
{ "tag": "GAIL", "total": 1 },
{ "tag": "Andhra Pradesh", "total": 1 },
{ "tag": "N Chandrababu Naidu", "total": 1 },
{ "tag": "Prime Minister", "total": 1 },
{ "tag": "Bharatiya Janata Party (BJP)", "total": 1 },
{ "tag": "Government", "total": 1 }
]
}
Also sorted by "tag relevance score" for the user for good measure, but you can look at dropping that or even both of the last stages as is appropriate to your actual case.
Still, by far the best option. Get to learn how to use the aggregation framework. If your "output" will still be "big" ( over 16MB ) then try to look at moving to MongoDB 2.6 or greater. Aggregate statements can produce a "cursor" which can be iterated rather than pull all results at once. Also there is the $out operator which can create a collection just like mapReduce does.
If your data is actually in the "hash" like format of sub-documents how you indicate in your notation of this ( which follows a PHP "dump" convention for arrays ), then you need to use mapReduce as the aggregation framework cannot traverse "hash-keys" the way these are represented. Not the best structure, and you should change it if this is the case.
Still there are several corrections to your approach and this does in fact become a single step operation to the final result. Again though, the final output will contain and "array" of "tags", since it really is not good practice to use your "data" as "key" names:
db.collection.mapReduce(
function() {
var tag_cloud = this.account_details.tag_cloud;
var obj = {};
for ( var k in tag_cloud ) {
obj[tag_cloud[k]] = 1;
}
emit( this.autoUid, obj );
},
function(key,values) {
var reduced = {};
// Combine keys and totals
values.forEach(function(value) {
for ( var k in value ) {
if (!reduced.hasOwnProperty(k))
reduced[k] = 0;
reduced[k] += value[k];
}
});
return reduced;
},
{
"out": { "inline": 1 },
"finalize": function(key,value) {
var output = [];
// Mapped to array for output
for ( var k in value ) {
output.push({
"tag": k,
"total": value[k]
});
}
// Even sorted just the same
return output.sort(function(a,b) {
return ( a.total < b.total ) ? -1 : ( a.total > b.total ) ? 1 : 0;
});
}
}
)
Or if it actually is an "array" of "tags" in your original document but your end output will be too big and you cannot move up to a recent release, then the initial array processing is just a little different:
db.collection.mapReduce(
function() {
var tag_cloud = this.account_details.tag_cloud;
var obj = {};
tag_cloud.forEach(function(tag) {
obj[tag] = 1;
});
emit( this.autoUid, obj );
},
function(key,values) {
var reduced = {};
// Combine keys and totals
values.forEach(function(value) {
for ( var k in value ) {
if (!reduced.hasOwnProperty(k))
reduced[k] = 0;
reduced[k] += value[k];
}
});
return reduced;
},
{
"out": { "replace": "newcollection" },
"finalize": function(key,value) {
var output = [];
// Mapped to array for output
for ( var k in value ) {
output.push({
"tag": k,
"total": value[k]
});
}
// Even sorted just the same
return output.sort(function(a,b) {
return ( a.total < b.total ) ? -1 : ( a.total > b.total ) ? 1 : 0;
});
}
}
)
Everything essentially follows the same principles to get to the end result:
De-normalize to a "user" and "tag" combination with "user" and the grouping key
Combine the results per user with a total on "tag" values.
In the mapReduce approach here, apart from being cleaner than what you seemed to be trying, the other main point to consider here is that the reducer needs to "output" exactly the same sort of "input" that comes from the mapper. The reason is actually well documented, as the "reducer" can in fact get called several times, basically "reducing again" output that has already been through reduce processing.
This is generally how mapReduce deals with "large inputs", where there are lots of values for a given "key" and the "reducer" only processes so many of them at one time. For example a reducer may actually only take 30 or so documents emitted with the same key, reduce two sets of those 30 down to 2 documents and then finally reduce to a single output for a single key.
The end result here is the same as the other output shown above, with the mapReduce difference that everything is under a "value" key as that is just how it works.
So a couple of ways to do it depending on your data. Do try to stick with the aggregation framework where possible as it is much faster and modern versions can consume and output just as much data as you can throw at mapReduce.
I'm trying to build a nested array in jQuery based on a user's selection from a drop down menu. This will be used in a JSON request at a later date.
So far my code does produce (almost) the required result, however no matter order i select the options from my drop down menu, the output (which i log in the console at the end) is always the same.
$('#comboGenre').change(function () {
var values = $('#comboGenre').val();
var parsedJSON = JSON.parse($data); //Data returned from ajax request
for (var i = 0; i < values.length; i += 1) {
$genreList = parsedJSON.genre[i];
console.log($genreList);
}
});
So if i select RPG and Action from my drop down, the output gives me RPG and Driving. If i selected RPG, Driving and Action (in that order), i get what i would expect RPG, Driving and Action.
So it's just iterating through my JSON, when really it should be returning the 'selected' option.
How can i achieve this?
My JSON looks like this if it's useful:
{"genres": [{
"genre": "RPG",
"publishers": [{
"publisher": "Square",
"games": [{
"game": "FFX",
"rating": [
12, 15
]
}]
}]
},
{
"genre": "Driving",
"publishers": [{
"publisher": "Turn10",
"games": [{
"game": "Forza",
"rating": [
5
]
}]
}]
},
{
"genre": "Action",
"publishers": [{
"publisher": "EA",
"games": [{
"game": "COD",
"rating": [
18, 20
]
}]
}]
}
]}
EDIT:
I've also tried this:
$('#comboGenre').change(function () {
var parsedJSON = JSON.parse($data);
$genreList = "";
$.each(parsedJSON.genres, function(index, value){
$genreList = parsedJSON.genres[index];
console.log($genreList);
});
});
And i end up getting ALL the objects in my JSON, so from here, i'm only wanting to add the selected object to the $genreList variable.
If you broke out some of the logic and created a genre finding function and used the selected string to find the proper object you could then put the object into the variable you will use later. I do some checking to ensure that the genre that has been selected isn't already in my array which is because I am using the multiple select
JSFiddle: http://jsfiddle.net/vkTFq/
Code:
$(function(){
var selectedGenres = [];
var genres =[{"genre":"RPG","publishers":[{"publisher":"Square","games":[{"game":"FFX","rating":[12,15]}]}]},{"genre":"Driving","publishers":[{"publisher":"Turn10","games":[{"game":"Forza","rating":[5]}]}]},{"genre":"Action","publishers":[{"publisher":"EA","games":[{"game":"COD","rating":[18,20]}]}]}]
$('#comboGenre').change(function() {
$(this).find(":selected").each(function() {
var selectedGenre = findGenre($(this).val())
if (!genreAlreadySelected(selectedGenre.genre)) {
selectedGenres.push(selectedGenre);
};
});
console.log (JSON.stringify(selectedGenres));
});
function genreAlreadySelected(genre){
for(var i = 0; i < selectedGenres.length; i++){
if (genre == selectedGenres[i].genre) {
return true;
};
return false;
}
}
function findGenre(genre){
for(var i = 0; i < genres.length; i ++){
console.log(genre)
if(genre == genres[i].genre){
return genres[i];
}
}
};
});
I'm working with a response from the Webtrends API in Google apps script and I have a JSON/JS object that looks like this:
"data": [
{
"period": "Month",
"start_date": "2013-12",
"end_date": "2013-12",
"attributes": {},
"measures": {
"Visits": 500
},
"SubRows": [
{
"facebook.com": {
"attributes": {},
"measures": {
"Visits": 100
},
"SubRows": null
},
"google.co.uk": {
"attributes": {},
"measures": {
"Visits": 100
},
"SubRows": null
},
"newsnow.co.uk": {
"attributes": {},
"measures": {
"Visits": 100
},
"SubRows": null
},
"No Referrer": {
"attributes": {},
"measures": {
"Visits": 100
},
"SubRows": null
},
"t.co": {
"attributes": {},
"measures": {
"Visits": 100
},
"SubRows": null
}
}
]
}
]
What I need to access is the names i.e facebook.com etc... and visit numbers for each of the SubRows.
I'm able to get the visit numbers, but I can't work out how to get the names. Please note the names will change constantly as different sites will send different amounts of traffic each day.
Section of my code at the moment where I get the visit numbers:
for(i in dObj){
var data = dObj[i].SubRows;
var sd = dObj[i].start_date;
var ed = dObj[i].end_date;
if(sd == ed){
var timep = ""+ sd;
}
else{
var timep = ""+ sd + "-" + ed;
}
var subRows = data[0];
Logger.log(subRows);
for(i in subRows){
var row = subRows[i];
var rmeasures = row.measures;
var rvis = rmeasures.Visits;
values = [timep,"",rvis]; //Blank string for where the name of the site would go
}
}
I've tried the following links, but none of them seem to have the answer:
Getting JavaScript object key list
How to access object using dynamic key?
How to access key itself using javascript
How do I access properties of a javascript object if I don't know the names?
I'm just using vanilla google apps script as I don't have any experience with Jquery etc...
Any help would be much appreciated!
I usually use a little helper function that looks like this:
var keyVal = function(o) {
var key = Object.keys(o)[0];
return {"key": key, "val":o[key]};
} ;
This will map an object with a variable key to a key/value object {key:...., val:{}}, which is usually convenient enough to work with.
describe.only ("stack overflow answer", function(){
it ("is should create a key/value pair" , function(){
var res = keyVal( {
"facebook.com": {
"attributes": {},
"measures": {
"Visits": 100
},
"SubRows": null
}});
res.key.should.equal('facebook.com');
res.val.attributes.should.deep.equal({});
});
Within the loop, the variable i contains the current key. Replacing the empty string with i should give you what you need.
You might also want to look at some of the more functional tools built into Javascript. Some more concise code might also be more explicit:
data.map(function(datum) {
var timep = datum.start_date == datum.end_date ? datum.end_date :
(data.start_date + "-" + datum.end_date);
return datum.SubRows.map(function(subRow) {
return Object.keys(subRow).map(function(key) {
return [timep, key, subRow[key].measures.Visits];
});
});
});
would return an object something like this:
[
[
[
["2013-12", "facebook.com", 100],
["2013-12", "google.co.uk", 100],
["2013-12", "newsnow.co.uk", 100],
["2013-12", "No Referrer", 100],
["2013-12", "t.co", 100 ]
]
]
]
This just uses map and Object.keys to simplify some of what you're doing with explicit loops.