Good day stack people,
I'm doing research for my self on how to combine two Json feeds in one and display them in one timeline by date using JS or jQuery.
For example we will have two json files file1.json and file2.json (one from twitter and another from filckr).
I need "n" numbers of latest items and show append them to show by items time.
Any ideas or hints?
Thank you!
P.S. Example feeds: http://twitter.com/status/user_timeline/ignaty.json?count=5 and http://api.flickr.com/services/feeds/groups_pool.gne?%20id=675729#N22&lang=en-us&format=json
Lets pick only any one value from each.
Here's some method that should do that (you need to tweak it though).
Essentially you request the two APIs and then (once both requests are completed) you sort an array of normalized objects.
var all = [];
var waiting = 2; // number of services you request
// once you get response1 or response2
function parseFlickr(data) {
$.each(data, function(index, item) {
// normalize item here depending on service format (parse date)
var normalized = {};
normalized.date = new Date(Date.parse(item.date));
all.push(normalized);
});
if(--waiting == 0) { onDone(); }
}
function onDone() {
all.sort(function(a,b) {
// switch -1 and +1 to invert ordering
return (a.date < b.date ? -1 : (a.date > b.date ? +1 : 0));
});
// do the rendering/appending (you might limit the amount here)
}
Related
I am using an external API to fetch a list of events happening between two dates. I have then used array.reduce to group the events happening on the same day into one array.
const time = events && events.reduce((acc, item) => {
if (!acc[item.fixture.date.split('T')[1]]) {
acc[item.fixture.date.split('T')[1]] = [];
}
acc[item.fixture.date.split('T')[1]].push(item);
return acc;
}, {})
They are labelled by the time in which the event occurs. If I console.log time then you can see in the image below how the data is returned for one day.
Example of returned data
I am trying to work out how to loop through these objects and find the ones that are within 30 minutes of each other. For example: It would find 16:05:00+01:00 and 16:30:00+01:00 and place these into a new array called Interval together.
What would be the easiest way to achieve this?
const datesInRange = (date1, date2, range) => //range = difference in minutes
(date1 - date2) / 1000 / 60 <= range
? true : false
I'm trying to override the sort logic in dgrid as suggested by kfranqueiro in this link - https://github.com/SitePen/dgrid/issues/276.
I get the data from the server in sorted order and just want to update the UI of column header. I'm doing this -
On(mygrid, 'dgrid-sort', lang.hitch( this,function(event){
var sort = event.sort[0];
var order = this.sort.descending ? "descending" : "ascending";
console.log("Sort "+ this.sort.property + " in " +order+" order.");
event.preventDefault();
mygrid.updateSortArrow(event.sort, true);
myFunctionToRefreshGrid();
}));
...
myFunctionToRefreshGrid: function() {
...//get data from server in sorted order
var mystore = new Memory({data: sortedDataFromServer, idProperty: 'id'});
mygrid.set("collection", mystore);
...
}
Memory here is "dstore/Memory". I'm using dgrid 0.4, dstore 1.1 and dojo 1.10.4
Before calling set('collection',...) I see that sortedDataFromServer is in the desired sorted order. But for some reason, the order in the grid is different. For example, when sorted in descending order, I see that the values starting with lower case appear first in descending order and then the values starting with upper case appear in sorted order. It looks like dstore is doing something more.
What could be going on? Am I doing something wrong? Is there a different/better way to do custom sorting?
Thanks,
This is how I ended up addressing the situation -
As suspected, the collection/store was further sorting my data and hence the inconsistency. I customized the store (Memory) as shown below and use the custom store when setting data to my grid.
var CustomGridStore = declare([Memory],{
sort: function (sorted) {
sorted = [];//Prevent the collection from sorting the data
return this.inherited(arguments);
}
});
I think you are doing the right thing, only problem here is that you are not resetting sort property of grid, one you re-initiate memory with sorted order, it get's sorted automatically
after you are calling
event.preventDefault();
call this
mygrid.set("sort", null);
I am doing custom sorting in my one of grid as following
self.xrefGrid.on("dgrid-sort", function (event) {
var sort = event.sort[0];
event.preventDefault();
self.xrefGrid.set('sort', function (a, b) {
var aValue,bValue;
if (a[sort.attribute] && typeof a[sort.attribute] == "string")
aValue = a[sort.attribute].toLowerCase();
if (b[sort.attribute] && typeof b[sort.attribute] == "string")
bValue = b[sort.attribute].toLowerCase();
var result = aValue > bValue ? 1 : -1;
return result * (sort.descending ? -1 : 1);
});
self.xrefGrid.updateSortArrow(event.sort, true);
});
I am playing around with Node and Express. My current problem is not "how to do things", as I have my paging and filtering/searching working in my "mini API" that I have made from scratch and that I am playing with. My question is more about "good practices" and "proper way" of doing things.
I will put some snippets of code below, which I am sure will bring some critics. The API is memory based, no database involved. I have an array of hard-coded users that I am pulling data from and pushing data in.
Below is my code (as you can see, I have also implemented basic authentication using passport):
//This array contains all my user data...
var users = [
{
"id": "1",
"firstName": "john",
"lastName": "doe"
}
];
//This is the route I have configured in order to retrieve all users.
//I am retrieving the users with the getUsers() function and then returning it.
//in the response object.
router.get('/users', passport.authenticate('basic', { session: false }),
function(req, res, next) {
var result = users.getUsers(req);
res.status(200).json({ users: result });
});
//This method will get the page and items parameters and will try to parse
//them. After that, it will call the search function that will filter the data
//Finally, I am passing the result array, page param and items param to the
//sliceUsers() function that will take care of slicing the result array depending
//on the values of page and items.
exports.getUsers = function(req) {
console.log(req.query);
var page = req.query.page;
items = req.query.items;
page = page !== 'undefined' ? parseInt(page, 10) : undefined;
items = items !== 'undefined' ? parseInt(items, 10) : undefined;
//The search method will filter the data
var searchResults = exports.search(req.query);
//Then, I call sliceUsers(), passing the filtered data, page and items parameters
return exports.sliceUsers(searchResults , page, items);
}
//This method will slice the array to return the page and # of items specified
//The "data" array that is passed as the first parameters is the array that contains
//the data that have already been filtered.
exports.sliceUsers= function(data, page, items) {
page = (page < 1 ? 1 : page) || 1;
items = (items < 1 ? 5 : items) || 5;
console.log('page', page, 'items', items);
var indexStart, indexEnd;
indexStart = (page - 1) * items;
indexEnd = indexStart + items;
return data.slice(indexStart, indexEnd);
};
//Those 2 methods take care of filtering
exports.search = function(query) {
return users.filter(search(query));
}
function search(query) {
console.log('search function');
return function(element) {
for(var i in query) {
//Please note here how I am checking the the parameter I am currently
//checking is NOT 'page' nor 'items'
if(query[i] != element[i] && i !== 'page' && i !== 'items') {
return false;
}
}
return true;
}
}
A few questions arise here:
Is the way I am dealing with filter/search and THEN, dealing with paging the right way?
In the search() function, when I am looping over req.query, I know that the way I check if the current param is different than 'page' or 'items' is not very efficient, but I don't know I could do that differently.
My goal here is to learn node and express and get better at javascript, what would you advice me to do next in order to pursue that goal? Any resources greatly appreciated, as the only stuff I found on APIs are basic operations that don't really deal with search/filtering. When I do find those, it's never in addition to paging for example. I have never found a complete example.
I have heard that underscore could help me do the filtering, but once again, did not really find any good example, any snippets somewhere?
Any critic GREATLY appreciated.
P.S: I apologize in advance for any grammatical error in this question.
There's a standard called OData for thing like filtering, searching, selecting and of course REST. There are few options for using it at the moment. For node backend part it's node-odata. For more, see here: http://www.odata.org/libraries/
Im creating my batch and inserting it to collection using command i specified below
batch = []
time = 1.day.ago
(1..2000).each{ |i| a = {:name => 'invbatch2k'+i.to_s, :user_id => BSON::ObjectId.from_string('533956cd4d616323cf000000'), :out_id => 'out', :created_at => time, :updated_at => time, :random => '0.5' }; batch.push a; }
Invitation.collection.insert batch
As stated above, every single invitation record has user_id fields value set to '533956cd4d616323cf000000'
after inserting my batch with created_at: 1.day.ago i get:
2.1.1 :102 > Invitation.lte(created_at: 1.week.ago).count
=> 48
2.1.1 :103 > Invitation.lte(created_at: Date.today).count
=> 2048
also:
2.1.1 :104 > Invitation.lte(created_at: 1.week.ago).where(user_id: '533956cd4d616323cf000000').count
=> 14
2.1.1 :105 > Invitation.where(user_id: '533956cd4d616323cf000000').count
=> 2014
Also, I've got a map reduce which counts invitations sent by each unique User (both total and sent to unique out_id)
class Invitation
[...]
def self.get_user_invites_count
map = %q{
function() {
var user_id = this.user_id;
emit(user_id, {user_id : this.user_id, out_id: this.out_id, count: 1, countUnique: 1})
}
}
reduce = %q{
function(key, values) {
var result = {
user_id: key,
count: 0,
countUnique : 0
};
var values_arr = [];
values.forEach(function(value) {
values_arr.push(value.out_id);
result.count += 1
});
var unique = values_arr.filter(function(item, i, ar){ return ar.indexOf(item) === i; });
result.countUnique = unique.length;
return result;
}
}
map_reduce(map,reduce).out(inline: true).to_a.map{|d| d['value']} rescue []
end
end
The issue is:
Invitation.lte(created_at: Date.today.end_of_day).get_user_invites_count
returns
[{"user_id"=>BSON::ObjectId('533956cd4d616323cf000000'), "count"=>49.0, "countUnique"=>2.0} ...]
instead of "count" => 2014, "countUnique" => 6.0 while:
Invitation.lte(created_at: 1.week.ago).get_user_invites_count returns:
[{"user_id"=>BSON::ObjectId('533956cd4d616323cf000000'), "count"=>14.0, "countUnique"=>6.0} ...]
Data provided by query, is accurate before inserting the batch.
I cant wrap my head around whats going on here. Am i missing something?
The part that you seemed to have missed in the documentation seem to be the problem here:
MongoDB can invoke the reduce function more than once for the same key. In this case, the previous output from the reduce function for that key will become one of the input values to the next reduce function invocation for that key.
And also later:
the type of the return object must be identical to the type of the value emitted by the map function to ensure that the following operations is true:
So what you see is your reduce function is returning a signature different to the input it receives from the mapper. This is important since the reducer may not get all of the values for a given key in a single pass. Instead it gets some of them, "reduces" the result and that reduced output may be combined with other values for the key ( possibly also reduced ) in a further pass through the reduce function.
As a result of your fields not matching, subsequent reduce passes do not see those values and do not count towards your totals. So you need to align the signatures of the values:
def self.get_user_invites_count
map = %q{
function() {
var user_id = this.user_id;
emit(user_id, {out_id: this.out_id, count: 1, countUnique: 0})
}
}
reduce = %q{
function(key, values) {
var result = {
out_id: null,
count: 0,
countUnique : 0
};
var values_arr = [];
values.forEach(function(value) {
if (value.out_id != null)
values_arr.push(value.out_id);
result.count += value.count;
result.countUnique += value.countUnique;
});
var unique = values_arr.filter(function(item, i, ar){ return ar.indexOf(item) === i; });
result.countUnique += unique.length;
return result;
}
}
map_reduce(map,reduce).out(inline: true).to_a.map{|d| d['value']} rescue []
end
You also do not need user_id in the values emitted or kept as it is already the "key" value for the mapReduce. The remaining alterations consider that both "count" and "countUnique" can contain an exiting value that needs to be considered, where you were simply resetting the value to 0 on each pass.
Then of course if the "input" has already been through a "reduce" pass, then you do not need the "out_id" values to be filtered for "uniqueness" as you already have the count and that is now included. So any null values are not added to the array of things to count, which is also "added" to the total rather than replacing it.
So the reducer does get called several times. For 20 key values the input will likely not be split, which is why your sample with less input works. For pretty much anything more than that, then the "groups" of the same key values will be split up, which is how mapReduce optimizes for large data processing. As the "reduced" output will be sent back to the reducer again, you need to be mindful that you are considering the values you already sent to output in the previous pass.
I'd like to come up with a good way to have a "suggested" order for how to sort an array in javascript.
So say my first array looks something like this:
['bob','david','steve','darrel','jim']
Now all I care about, is that the sorted results starts out in this order:
['jim','steve','david']
After that, I Want the remaining values to be presented in their original order.
So I would expect the result to be:
['jim','steve','david','bob','darrel']
I have an API that I am communicating with, and I want to present the results important to me in the list at the top. After that, I'd prefer they are just returned in their original order.
If this can be easily accomplished with a javascript framework like jQuery, I'd like to hear about that too. Thanks!
Edit for clarity:
I'd like to assume that the values provided in the array that I want to sort are not guaranteed.
So in the original example, if the provided was:
['bob','steve','darrel','jim']
And I wanted to sort it by:
['jim','steve','david']
Since 'david' isn't in the provided array, I'd like the result to exclude it.
Edit2 for more clarity:
A practical example of what I'm trying to accomplish:
The API will return something looking like:
['Load Average','Memory Usage','Disk Space']
I'd like to present the user with the most important results first, but each of these fields may not always be returned. So I'd like the most important (as determined by the user in some other code), to be displayed first if they are available.
Something like this should work:
var presetOrder = ['jim','steve','david']; // needn't be hardcoded
function sortSpecial(arr) {
var result = [],
i, j;
for (i = 0; i < presetOrder.length; i++)
while (-1 != (j = $.inArray(presetOrder[i], arr)))
result.push(arr.splice(j, 1)[0]);
return result.concat(arr);
}
var sorted = sortSpecial( ['bob','david','steve','darrel','jim'] );
I've allowed for the "special" values appearing more than once in the array being processed, and assumed that duplicates should be kept as long as they're shuffled up to the front in the order defined in presetOrder.
Note: I've used jQuery's $.inArray() rather than Array.indexOf() only because that latter isn't supported by IE until IE9 and you've tagged your question with "jQuery". You could of course use .indexOf() if you don't care about old IE, or if you use a shim.
var important_results = {
// object keys are the important results, values is their order
jim: 1,
steve: 2,
david: 3
};
// results is the orig array from the api
results.sort(function(a,b) {
// If compareFunction(a, b) is less than 0, sort a to a lower index than b.
// See https://developer.mozilla.org/en/JavaScript/Reference/Global_Objects/Array/sort
var important_a = important_results[a],
important_b = important_results[b],
ret;
if (important_a && !important_b) {ret = -1}
else if (important_b && !important_a) {ret = 1}
else if (important_a && important_b) {ret = important_a - important_b}
else {ret = 0}; // keep original order if neither a or b is important
return(ret);
}
)
Use a sorting function that treats the previously known important results specially--sorts them to the head of the results if present in results.
items in important_results don't have to be in the results
Here's a simple test page:
<html>
<head>
<script language="javascript">
function test()
{
var items = ['bob', 'david', 'steve', 'darrel', 'jim'];
items.sort(function(a,b)
{
var map = {'jim':-3,'steve':-2,'david':-1};
return map[a] - map[b];
});
alert(items.join(','));
}
</script>
</head>
<body>
<button onclick="javascript:test()">Click Me</button>
</body>
</html>
It works in most browsers because javascript typically uses what is called a stable sort algorithm, the defining feature of which is that it preserves the original order of equivalent items. However, I know there have been exceptions. You guarantee stability by using the array index of each remaining item as it's a1/b1 value.
http://tinysort.sjeiti.com/
I think this might help. The $('#yrDiv').tsort({place:'start'}); will add your important list in the start.
You can also sort using this function the way you like.
Live demo ( jsfiddle seems to be down)
http://jsbin.com/eteniz/edit#javascript,html
var priorities=['jim','steve','david'];
var liveData=['bob','david','steve','darrel','jim'];
var output=[],temp=[];
for ( i=0; i<liveData.length; i++){
if( $.inArray( liveData[i], priorities) ==-1){
output.push( liveData[i]);
}else{
temp.push( liveData[i]);
}
}
var temp2=$.grep( priorities, function(name,i){
return $.inArray( name, temp) >-1;
});
output=$.merge( temp2, output);
there can be another way of sorting on order base, also values can be objects to work with
const inputs = ["bob", "david", "steve", "darrel", "jim"].map((val) => ({
val,
}));
const order = ["jim", "steve", "david"];
const vMap = new Map(inputs.map((v) => [v.val, v]));
const sorted = [];
order.forEach((o) => {
if (vMap.has(o)) {
sorted.push(vMap.get(o));
vMap.delete(o);
}
});
const result = sorted.concat(Array.from(vMap.values()));
const plainResult = result.map(({ val }) => val);
Have you considered using Underscore.js? It contains several utilities for manipulating lists like this.
In your case, you could:
Filter the results you want using filter() and store them in a collection.
var priorities = _.filter(['bob','david','steve','darrel','jim'],
function(pName){
if (pName == 'jim' || pName == 'steve' || pName == 'david') return true;
});
Get a copy of the other results using without()
var leftovers = _.without(['bob','david','steve','darrel','jim'], 'jim', 'steve', 'david');
Union the arrays from the previous steps using union()
var finalList = _.union(priorities, leftovers);