I am writing my second mapReduce to get the top ten songs played for every user for the last week from a collection that contains "activity" nested document that has an array of song_id, counter and date. Counter means the "play times" of the song.
I tried to use mapReduce and I was able to accomplish this task and output the needed results using only "map" without the need to reduce the emitted values. Is this a wrong approach I am using? what is the best approach of doing this.
Here is the map function:
var map = function() {
user_top_songs = [];
user_songs = [];
limit = 10;
if(this.activities !== undefined){
key = {user_id:this.id};
for (var i=0; i < this.activities.songs.length; i++){
if (this.activities.songs !== undefined && this.activities.songs[i].date.getDate() > (new Date().getDate()-7))
user_songs.push([this.activities.songs[i].song_id, this.activities.songs[i].counter]);
}
if(user_songs.length !== 0){
user_songs.sort(function(a,b){return b[1]-a[1]});
if(user_songs.length < 10 )
limit = user_songs.length;
for(var j=0; j < limit; j++)
user_top_songs.push(user_songs[j]);
}
value = {songs:user_top_songs};
emit(key,value);
}
}
Here is the empty reduce method:
var reduce = function(key, values) {};
You shouldn't need a reduce function. Based on the input data it won't be necessary, and I'll explain why.
To recall in a simplified manner, in MapReduce the mapper function takes the input and splits it up by key then passes the (key,value) pairs to the reducer. The reducer then aggregates the (key, [list of values]) pairs into some useful output.
In your case, the key is the user ID, and the value is top 10 songs they listened to. Just by the way the data is laid out, it is already organized into (key,[list of values]) pairs. You already have the key with the list of every value that is associated with it following it. The user ID is listed with every song they listenend to right after it, so there is no need to reduce.
Basically, the reduce step would be combining each (user ID, song) pair into a list of the user's songs. But that's already been done. It's inherent in the data. So, in this specific case, the mapper is the only necessary function to accomplish what you need in this case.
Related
I have an array of 11 objects which contain JSON data. I wrote a function in which a new key with a zero value is added to each of the objects. Now I want to update the value of the said key in all 11 objects. The data is stored in an array2 with 11 numbers. My for loop doesn't seem to work for this, and the only way to do it (so far) is to hard code it. Does anyone has a suggestion how this can be done?
The desired outcome would be this:
array[0].new_key = array2[0];
array[1].new_key = array2[1];
The first art of the function, before the for loop with j, is for adding the new key into the original array and that part works.
for (i = 0; i < array.length; i++) {
array.map(i => i.new_key = 0);
console.log(array)
for (j = 0; j < array2.length; j++) {
array[i].new_key = array2[j];
console.log(array)
}
}
}```
I split it into two functions, I realized that I made it too complicated and it didn't made sense. I wrote a second function that only updates all the key values, so indeed, I removed the inner loop as it was not needed. Thank you for the help.
.map() does not modify the original array:
The map() method creates a new array with the results of calling a
provided function on every element in the calling array.
You will want to get the result of the map by assigning it to a variable, and see what is happening there. Right now you don't do anything with it, so it will just disappear.
While the above is true for maps, in this case the original array is being modified as we access the object's properties and modify them there.
As the title says, I need to sort my data depending on my first column which contains som rank names. I do not wan't it sorted alphabetically but rather in a order I define myself.
Example:
Admin_contact
Commissioner
Battalion Chief
Paramedic
I want them listed in that order using google scripts so it automatically sorts when data gets edited.
Hope someone can help me achieve this
Absolutely, you can create a google script function onEdit() in the spreadsheet which will fire each time you edit something.
There are a few ways to do it. One of the most obvious ones (probably not the best as it loops through the data a lot):
Using getLastColumn() and getLastRow() grab all the data. You then use 2 arrays. 1 is current and 1 is sorted.
Loop through the current array and if data[i][0] matches the rank, output it to the sorted array
After you finish one loop, change to the rank you expect next (keep them in an array and you can use a for loop with rank[n])
Loop the array until you run out of the ranks.lenght
So it would be something along the lines of
function onEdit()
//Get the data
rank = [rank1, rank2, rank3]
for (n=0 ; n < rank.lenght; n++) {
for (i = 0; i < data.lenght; i++)
if (data[i][0] === rank[n]) {
sorted[x] = data[i];
x++;
}
}
}
Though I would recommend to make it so you can manually run it instead of an onEdit. Or you can set it so only if the last row is edited, then to fire it (make a return at the start of the script if the edited row is less than getLastRow()+1
Try this code:
function customSort(array, arrayOrder, numColumn) {
var order = [];
// transpose vertical 2d array into 1d array
for (var i = 0; i < arrayOrder.length; i++) {
order.push(arrayOrder[i][0]);
}
// sort
return array.sort(function (a, b) {
return order.indexOf(a[numColumn-1]) - order.indexOf(b[numColumn-1]);
});
}
Here's the use like custom function:
Same example for column 2:
I am using expressjs to render my routes and pass on database information.
I have two arrays 1. paintingJobs 2. customerList . The (1) paintingJobs array has a foreign key, which contains the customer ID.I want to create an array of customers who currently HAVE painting jobs and pass that array onto the view. To filter the (2)customerList array down to a list of only those customers who currently have painting jobs; I use a series of nested for loops like so.....
for(var i=0; i<paintingJobs.length; i++) {
for(j=0; j<customerList.length; j++) {
if (customerList[j].ID == paintingJobs[i].CustomerID) {
customerRecords.push(customerList[j]);
}
}
}
Creating the new filtered array of customerRecords, which is then passed to the view.
My Question is: If I sort the (1)paintingJobs array by date prior to using it as a filter for my (1)customerList, will the resulting array (customerRecords) be sorted by date as well or more specifically is this a reliable method? If not, would appending the date to the customerList and then sorting the final array be the next best solution?
I am passing this information to my view and then creating an Unordered List based on the number of records.
Thank you for your time and any suggestions
Your double-loop preserves the order of paintingJobs in customerRecords, though unless you have duplicate customerList ID's, looks somewhat inefficient (why no break?).
This means changes to the order of paintingJobs before your double loop will be reflected in customerRecords. Changes after will not.
paintingJobs.sort(however);
for(var i=0; i<paintingJobs.length; i++) {
for(j=0; j<customerList.length; j++) {
if (customerList[j].ID == paintingJobs[i].CustomerID) {
customerRecords.push(customerList[j]);
break; // next paintingJob
}
}
}
It looks like the items in customerList are Objects, so if you make a change to an Object in customerList, this change will also appear in customerRecords even after your double loop.
This is not a answer to your original question, but concerning performance i would create something like a test- object with the ID as key,like
for(var i=0; i<paintingJobs.length; i++) {
testArray[paintingJobs[i].ID] += 1;
}
for(j=0; j<customerList.length; j++) {
testArray[customerList[j].ID] += 1;
}
and check where the testObject has the value 2.
Then with these keys run your operations.
This method will be MUCH faster as your nested loop.
I have a long array of strings that will have a varying length because I'm pulling the strings from a database table that will be filled dynamically by user submission, however, that long array will be comprised of a repeating set of 6 variables. So I want to slice it at every 6th index, turning what I've sliced into an array, and then printing the list of those sub arrays to the page.
This is what I have so far:
//I'm using Parse as my database and the find() method is one way you query it.
var search = [];
query.find({
success: function(results) {
//loop through results to get the key-value pair of each row
for (i = 0; i < results.length; i++){
activity = results[i].get("activity");
scene = results[i].get("location");
neighborhood = results[i].get("neighborhood");
date = results[i].get("date");
details = results[i].get("details");
time = results[i].get("time");
//Here I'm pushing the row of the six variables into an array, but since there will be multiple rows on the table from multiple submissions, the array will contain however many rows there are.
search.push(activity, scene, neighborhood, date, details, time);
//my second array
var search2 = [];
//This is that part I'm unsure of. I want to loop through the search array slicing it at every 6th index creating multiple sub arrays. I then want to write the list of those sub arrays to the page
for(i=0; i < search.length; i+=6){
search2 = search.slice(0,6);
}
//this is the div I want to write the arrays to.
$("#mainDiv").text(search2);
};// closes success function
Instead of pushing each individual field onto the search array and then slicing each set out at the end, why don't you just add each record's set of data as an array to your search array?
search.push([activity, scene, neighborhood, date, details, time]);
That way, when you loop through the search array to print out each record's values you don't have to do any slicing or incrementing by 6. For each loop you already have the array slice you want:
for (i = 0; i < search.length; i++) {
$('#mainDiv').append(search[i]);
}
Maybe I've misread your question, but it looks like you want to add all the records to your #mainDiv. If you use the jQuery text() method for each iteration of your loop, you'll just overwrite the text for each iteration, making the result the final value of your loop instead of all of the values of all of the items. To add each record to your #mainDiv, you want to use the append() method.
You might also want to add some spaces between the values, assuming you are using my approach to storing sets of arrays instead of individual field array items:
for (i = 0; i < search.length; i++) {
$('#mainDiv').append(search[i].join(' '));
}
I am in a situation that feels a little odd ... I have a list of keys deliberately containing duplicates. For the sake of the arguments lets assume this list looks like [1,2,3,2,1]. Currently the code to fetch the documents belonging to these Ids loops over the list of keys, calls findOne() and pushes the document into an array.
So we have a construct like this:
for (var i = 0; i < keys.length; i++) {
documents.push(db.items.findOne(keys[i]);
}
I am wondering whether there is a way to do this in a more ... elegant ... fashion, preferably with a single query? Keeping the order would be a plus but is not strictly required.
Edit:
Please note that this is a MongoDB question. I am looking for a way to substitute the above loop with a single call to db.items.find().
I don't think there's a direct way to retrieve a list for duplicate keys, but since you said that keys are unique you could do it using $in and some looping:
var keys = [1,2,3,2,1];
var docHash = {}; //auxiliary hashtable containing mapping id -> document
var documents = []; //the result
db.items.find({id : {$in : keys}})
.forEach(
function(doc) {
//if multiple documents can have the same key change it to:
// if (docHash[doc.id]) {
// docHash[doc.id].push(doc);
// } else {
// docHash[doc.id] = [doc];
// }
docHash[doc.id] = doc;
});
keys.forEach(function(i) {
// if multiple documents can have the same key change it to:
// docHash[i].forEach(function(item) {documents.push(item);});
documents.push(docHash[i]);
}
Note that, while it's longer than the original, it queries the database only once. As a bonus it returns documents in the same order as the given keys.
use for-each function in java-script that should help you a bit:
for (var i in keys) {// i will be 1 ,2 ,3 ,3 ,1
documents.push(db.items.findOne(i);
}
or i might've missed your questions intention complety?
but then again duplicate keys do not exist in any array..
var used=Array();
for (var i = 0; i < keys.length; i++) {
if (!used[keys[i]]){
used[keys[i]]=1;
documents.push(db.items.findOne(keys[i]);}
}
that will ignore any duplicates and count them as one eather that or you can run query on the keys rendering them in to a new array that does not contains duplicates