Json and group by solution - javascript

I'm trying to select the last messages of an inbox and group them in a list by the topic_id. i want to display the last message of each topic.
the array looks like this:
[{
"id":"5",
"topic_id":"4",
"message_from":"24",
"message":"how do you do?",
"date":"2015-01-13 15:34:59"
},
{
"id":"6",
"topic_id":"1",
"message_from":"33",
"message":"go go go!!",
"date":"2015-01-13 13:35:06"
},
{
"id":"7",
"topic_id":"4",
"message_from":"33",
"message":"Je suis charlie",
"date":"2015-01-14 16:24:46"
},....
is there a solution to do it without a loop?

You can't do this without loops, but you can make this easier by breaking down the sequence of events into smaller functions. You might not like this approach, but it's the cleanest imo. Alternatively you can use a third-party library (underscore, perhaps?) that allows you to run groupings on data.
Basically, get a list of all the topic_ids for all records, loop over that topic_id array and pull out the last record for each and add that to an output array.
// Get a list of all the topic ids - no duplicates
function getTopicIds(arr) {
var out = [];
arr.forEach(function (el) {
if (out.indexOf(el.topic_id) === -1) out.push(el.topic_id);
});
return out;
}
// Given a topic_id, filter the array for only those records
// sort in desc order by id, and return the first record.
// Given that each record has a unique id, and we know that older
// messages will have higher ids, it's easier to sort by id than
// date here
function getLastMsg(id, arr) {
return arr.filter(function (el) {
return el.topic_id === id;
}).sort(function (a, b) { return +b.id - +a.id; })[0];
}
// return a array of the last messages for each topic_id
// in the records array
function getLastMsgs(arr) {
return getTopicIds(arr).map(function (id) {
return getLastMsg(id, arr);
});
}
var result = getLastMsgs(arr);
DEMO

Related

Remove all keys except current one in foreach, javascript

order.get_orderlines() is an array of three objects.
I want to call the print_xml function everytime I loop on an orderline where product.is_gift_product == true, but I also want to clean the array before.
Each time, I want to remove all other rows except the one I'm looping on.
I tried to remove all key on which I'm looping where product.is_gift_product != true, and when I loop on a orderline on which product.is_gift_product == true, to remove everything after, but this way I'm not printing the same tickets number as of the number of products marked as gift.
// START GIFT PRODUCT MANAGEMENT
var order_saved = order;
order.get_orderlines().forEach( function (orderline, i) {
if (orderline.product.is_gift_product != true) {
order.get_orderlines().splice(i, 1);
order_saved = order;
} else {
order.get_orderlines().splice(i, 9e9);
setTimeout(function(){
self.print_xml_gift(order, 'XmlGiftReceipt');
order._printed = true;
}, 2000);
}
});
// END GIFT PRODUCT MANAGEMENT
Try chaining a .filter(orderLine => orderLine.product.is_gift_product) before calling your forEach.
you should filter all elements to take a gift products.
During filtering, you can call for print of them.
const order_saved = order.get_orderlines().filter(orderline =>
orderline.product.is_gift_product).forEach(() => {
setTimeout(function(){
self.print_xml_gift(order, 'XmlGiftReceipt');
order._printed = true;
}, 2000);
});
I'm not sure if you need this timeout

Search through a big collection of objects

i have a really big collection of objects that i want to search through.
The array have > 60.000 items and the search performance can be really slow from time to time.
One object in that array looks like this:
{
"title": "title"
"company": "abc company"
"rating": 13 // internal rating based on comments and interaction
...
}
I want to search for the title and the company info and order that by the rating of the items.
This is what my search currently look like:
onSearchInput(searchTerm) {
(<any>window).clearTimeout(this.searchInputTimeout);
this.searchInputTimeout = window.setTimeout(() => {
this.searchForFood(searchTerm);
}, 500);
}
searchForFood(searchTerm) {
if (searchTerm.length > 1) {
this.searchResults = [];
this.foodList.map(item => {
searchTerm.split(' ').map(searchTermPart => {
if (item.title.toLowerCase().includes(searchTermPart.toLowerCase())
|| item.company.toLowerCase().includes(searchTermPart.toLowerCase())) {
this.searchResults.push(item);
}
});
});
this.searchResults = this.searchResults.sort(function(a, b) {
return a.rating - b.rating;
}).reverse();
} else {
this.searchResults = [];
}
}
Question: Is there any way to improve the search logic and performance wise?
A bunch of hints:
It's a bit excessive to put searching through 60,000 items on the front-end. Any way you can perform part of the search on the back-end? If you really must do it on the front-end considering searching in chunks of e.g. 10,000 and then using a setImmediate() to perform the next part of the search so the user's browser won't completely freeze during processing time.
Do the splitting and lowercasing of the search term outside of the loop.
map() like you're using it is weird as you don't use the return value. Better to use forEach(). Better still, is use filter() to get the items that match.
When iterating over the search terms, use some() (as pointed out in the comments) as it's an opportunity to early return.
sort() mutates the original array so you don't need to re-assign it.
sort() with reverse() is usually a smell. Instead, swap the sides of your condition to be b - a.
At this scale, it may make sense to do performance tests with includes(), indexOf(), roll-your-own-for-loop, match() (can almost guarantee it will be slower though)
Alex's suggestions are good. My only suggestion would be, if you could afford to pre-process the data during idle time (preferably don't hold up first render or interaction) you could process the data into a modified prefix trie. That would let you search for the items in O(k) time where k is the length of the search term (right now you are searching in O(kn) time because you look at every item and then do an includes which takes k time (it's actually a little worse because of the toLowerCase's but I don't want to get into the weeds of it).
If you aren't familiar with what a trie is, hopefully the code below gives you the idea or you can search for information with your search engine of choice. It's basically a mapping of characters in a string in nested hash maps.
Here's some sample code of how you might construct the trie:
function makeTries(data){
let companyTrie = {};
let titleTrie = {};
data.forEach(item => {
addToTrie(companyTrie, item.company, item, 0);
addToTrie(titleTrie, item.title, item, 0);
});
return {
companyTrie,
titleTrie
}
}
function addToTrie(trie, str, item, i){
trie.data = trie.data || [];
trie.data.push(item);
if(i >= str.length)
return;
if(! trie[str[i]]){
trie[str[i]] = {};
}
addToTrie(trie[str[i]], str, item, ++i);
}
function searchTrie(trie, term){
if(trie == undefined)
return [];
if(term == "")
return trie.data;
return searchTrie(trie[term[0]], term.substring(1));
}
var testData = [
{
company: "abc",
title: "def",
rank: 5
},{
company: "abd",
title: "deg",
rank: 5
},{
company: "afg",
title: "efg",
rank: 5
},{
company: "afgh",
title: "efh",
rank: 5
},
];
const tries = makeTries(testData);
console.log(searchTrie(tries.companyTrie, "afg"));

rx: unfold array to multiple streams

I have a stream holding an array, each element of which has an id. I need to split this into a stream per id, which will complete when the source stream no longer carries the id.
E.g. input stream sequence with these three values
[{a:1}, {b:1}] [{a:2}, {b:2}, {c:1}] [{b:3}, {c:2}]
should return three streams
a -> 1 2 |
b -> 1 2 3
c -> 1 2
Where a has completed on the 3rd value, since its id is gone, and c has been created on the 2nd value, since its id has appeared.
I'm trying groupByUntil, a bit like
var input = foo.share();
var output = input.selectMany(function (s) {
return rx.Observable.fromArray(s);
}).groupByUntil(
function (s) { return s.keys()[0]; },
null,
function (g) { return input.filter(
function (s) { return !findkey(s, g.key); }
); }
)
So, group by the id, and dispose of the group when the input stream no longer has the id. This seems to work, but the two uses of input look odd to me, like there could a weird order dependency when using a single stream to control the input of the groupByUntil, and the disposal of the groups.
Is there a better way?
update
There is, indeed, a weird timing problem here. fromArray by default uses the currentThread scheduler, which will result in events from that array being interleaved with events from input. The dispose conditions on the group are then evaluated at the wrong time (before the groups from the previous input have been processed).
A possible workaround is to do fromArray(.., rx.Scheduler.immediate), which will keep the grouped events in sync with input.
yeah the only alternative I can think of is to manage the state yourself. I don't know that it is better though.
var d = Object.create(null);
var output = input
.flatMap(function (s) {
// end completed groups
Object
.keys(d)
.filter(function (k) { return !findKey(s, k); })
.forEach(function (k) {
d[k].onNext(1);
d[k].onCompleted();
delete d[k];
});
return Rx.Observable.fromArray(s);
})
.groupByUntil(
function (s) { return s.keys()[0]; },
null,
function (g) { return d[g.key] = new Rx.AsyncSubject(); });

Sort an array by a preferred order

I'd like to come up with a good way to have a "suggested" order for how to sort an array in javascript.
So say my first array looks something like this:
['bob','david','steve','darrel','jim']
Now all I care about, is that the sorted results starts out in this order:
['jim','steve','david']
After that, I Want the remaining values to be presented in their original order.
So I would expect the result to be:
['jim','steve','david','bob','darrel']
I have an API that I am communicating with, and I want to present the results important to me in the list at the top. After that, I'd prefer they are just returned in their original order.
If this can be easily accomplished with a javascript framework like jQuery, I'd like to hear about that too. Thanks!
Edit for clarity:
I'd like to assume that the values provided in the array that I want to sort are not guaranteed.
So in the original example, if the provided was:
['bob','steve','darrel','jim']
And I wanted to sort it by:
['jim','steve','david']
Since 'david' isn't in the provided array, I'd like the result to exclude it.
Edit2 for more clarity:
A practical example of what I'm trying to accomplish:
The API will return something looking like:
['Load Average','Memory Usage','Disk Space']
I'd like to present the user with the most important results first, but each of these fields may not always be returned. So I'd like the most important (as determined by the user in some other code), to be displayed first if they are available.
Something like this should work:
var presetOrder = ['jim','steve','david']; // needn't be hardcoded
function sortSpecial(arr) {
var result = [],
i, j;
for (i = 0; i < presetOrder.length; i++)
while (-1 != (j = $.inArray(presetOrder[i], arr)))
result.push(arr.splice(j, 1)[0]);
return result.concat(arr);
}
var sorted = sortSpecial( ['bob','david','steve','darrel','jim'] );
I've allowed for the "special" values appearing more than once in the array being processed, and assumed that duplicates should be kept as long as they're shuffled up to the front in the order defined in presetOrder.
Note: I've used jQuery's $.inArray() rather than Array.indexOf() only because that latter isn't supported by IE until IE9 and you've tagged your question with "jQuery". You could of course use .indexOf() if you don't care about old IE, or if you use a shim.
var important_results = {
// object keys are the important results, values is their order
jim: 1,
steve: 2,
david: 3
};
// results is the orig array from the api
results.sort(function(a,b) {
// If compareFunction(a, b) is less than 0, sort a to a lower index than b.
// See https://developer.mozilla.org/en/JavaScript/Reference/Global_Objects/Array/sort
var important_a = important_results[a],
important_b = important_results[b],
ret;
if (important_a && !important_b) {ret = -1}
else if (important_b && !important_a) {ret = 1}
else if (important_a && important_b) {ret = important_a - important_b}
else {ret = 0}; // keep original order if neither a or b is important
return(ret);
}
)
Use a sorting function that treats the previously known important results specially--sorts them to the head of the results if present in results.
items in important_results don't have to be in the results
Here's a simple test page:
<html>
<head>
<script language="javascript">
function test()
{
var items = ['bob', 'david', 'steve', 'darrel', 'jim'];
items.sort(function(a,b)
{
var map = {'jim':-3,'steve':-2,'david':-1};
return map[a] - map[b];
});
alert(items.join(','));
}
</script>
</head>
<body>
<button onclick="javascript:test()">Click Me</button>
</body>
</html>
It works in most browsers because javascript typically uses what is called a stable sort algorithm, the defining feature of which is that it preserves the original order of equivalent items. However, I know there have been exceptions. You guarantee stability by using the array index of each remaining item as it's a1/b1 value.
http://tinysort.sjeiti.com/
I think this might help. The $('#yrDiv').tsort({place:'start'}); will add your important list in the start.
You can also sort using this function the way you like.
Live demo ( jsfiddle seems to be down)
http://jsbin.com/eteniz/edit#javascript,html
var priorities=['jim','steve','david'];
var liveData=['bob','david','steve','darrel','jim'];
var output=[],temp=[];
for ( i=0; i<liveData.length; i++){
if( $.inArray( liveData[i], priorities) ==-1){
output.push( liveData[i]);
}else{
temp.push( liveData[i]);
}
}
var temp2=$.grep( priorities, function(name,i){
return $.inArray( name, temp) >-1;
});
output=$.merge( temp2, output);
there can be another way of sorting on order base, also values can be objects to work with
const inputs = ["bob", "david", "steve", "darrel", "jim"].map((val) => ({
val,
}));
const order = ["jim", "steve", "david"];
const vMap = new Map(inputs.map((v) => [v.val, v]));
const sorted = [];
order.forEach((o) => {
if (vMap.has(o)) {
sorted.push(vMap.get(o));
vMap.delete(o);
}
});
const result = sorted.concat(Array.from(vMap.values()));
const plainResult = result.map(({ val }) => val);
Have you considered using Underscore.js? It contains several utilities for manipulating lists like this.
In your case, you could:
Filter the results you want using filter() and store them in a collection.
var priorities = _.filter(['bob','david','steve','darrel','jim'],
function(pName){
if (pName == 'jim' || pName == 'steve' || pName == 'david') return true;
});
Get a copy of the other results using without()
var leftovers = _.without(['bob','david','steve','darrel','jim'], 'jim', 'steve', 'david');
Union the arrays from the previous steps using union()
var finalList = _.union(priorities, leftovers);

Selecting n elements belonging to a user in a MapReduce for CouchDB

Please bear with me, I'm pretty new to the whole CouchDb stuff.
The db looks like:
** item ** count ** user **
A 20 bob
B 30 bob
C 10 bob
D 15 john
I want to write a MapReduce that selects all the items belonging to bob and only return the top 2, sorted. so it should return [{item:"B",count:"30"},{item:"A",count:"20}]
I'm not sure how this can be done? Seems like I have to emit(doc.item, doc.count), but how do I know if the user owns the doc? How do I run another MapReduce to select the top elements?
One solution would be to write your view to use a complex key, such as:
function (doc) {
emit([doc.user, doc.count], doc.item);
}
If you add descending=true to your query string, that would give you a view result like:
{"total_rows":4,"offset":0,"rows":[
{"id":"53f359b7cd360da296dd9aab3d0029bd","key":["john",15],"value":"D"},
{"id":"53f359b7cd360da296dd9aab3d001a0e","key":["bob",30],"value":"B"},
{"id":"53f359b7cd360da296dd9aab3d000fec","key":["bob",20],"value":"A"},
{"id":"53f359b7cd360da296dd9aab3d002668","key":["bob",10],"value":"C"}
]}
It's sorted already by user, then count. (with the item type as the value)
Then you can use a _list function to do the rest. The code below basically loops through the view, and returns the top 2 results for each user. If you specify user=bob in the query string, you'll only get the results for bob.
function (head, req) {
// specify that we're sending JSON as our response
provides('json', function () {
var results = [],
result, user, count, row;
while (row = getRow()) {
// if the user doesn't match the last iteration, reset our counter
if (user != row.key[0]) {
user = row.key[0];
count = 0;
}
// we only need the top 2
if (count++ >= 2) {
continue;
}
// start building a result object
result = {
item: row.value,
count: row.key[1]
};
// if we provide user=?
if (req.query.user) {
// check to see if it matches the current user
if (req.query.user === user) {
// if so, add it to the results
results.push(result);
}
// by default, we'll return the top 2 for every user
} else {
// add the user key to the result object
result.user = row.key[0];
// and add it to the result set
results.push(result);
}
}
// send outside the loop, since it needs to be sent as valid JSON
send(JSON.stringify(results));
});
}
If you put user and count in the key of the view, you can use startkey=["bob",""] and endkey=["bob"] to select the user, and descending=true and limit=2 to get the top two items.
I tried the following map function:
function(doc) {
if(doc.user && doc.count && doc.item) {
emit([doc.user, doc.count], doc);
}
}
with the query string ?startkey=["bob",""]&endkey=["bob"]&descending=true&limit=2 it returns:
{"total_rows":4,"offset":1,"rows":[
{"id":"item_B_bob","key":["bob",30],"value":{"_id":"item_B_bob","_rev":"1-b23bd22fb719c7d59b045bce0932df8c","item":"B","count":30,"user":"bob"}},
{"id":"item_A_bob","key":["bob",20],"value":{"_id":"item_A_bob","_rev":"2-515bca46eab383cfeaaa2a101d180291","item":"A","count":20,"user":"bob"}}
]}
Please note:
startkey and endkey are reversed because descending=true.
["bob",""] is a key greater then ["bob", ANY NUMBER] as specified in view collation.

Categories

Resources