Develop Reference

Meteor: Lazyload, load after rendering. Best practise

i have a Meteor Application which is very "slow" as there are a lot of API-Calls.
What i try to do is to break apart the loading/calls.
What i just did is:
i have loading template via iron-router
i waitOn for the first API-Call has finished
then i start the next API-calls in the Template.myTemplate.rendered - function
This was already a big benefit for the speed of my Application, but i want to break it up even more as the second call is in fact more like 5-25 API-calls.
So what i try to do now is inside the rendered function is a self-calling function which calls itself as long as there are no more to do and saves the response inside a session. (Until now it just rewrites, but even to this point i can´t get)
Template.detail.rendered = function(){
//comma separated list of numbers for the API-Call
var cats = $(this.find(".extra")).attr('data-extra').split(',');
var shop = $(this.find(".extra")).attr('data-shop');
var counter = 0;
var callExtras = function(_counter){
var obj = {
categories : [cats[_counter]],
shop : shop
};
if(_counter <= cats.length){
Meteor.subscribe('extra', obj,function(result){
//TODO dickes todo... nochmal nachdenken und recherchieren
//console.log(_counter);
Session.set('extra',Extra.find('extra').fetch()[0].results);
counter++;
callExtras(counter);
});
}
};
callExtras(counter);
Session.set('loading_msg', '' );
};
Now i have again problems with my reactive parts of the app desscribed here - Meteor: iron-router => waitOn without subscribe As i can´t find a proper way to update my client-side per user base collection. Also in the docs it is described the publish method also creates a new collection. (The new document´s ID) here - http://docs.meteor.com/#/full/publish_added
here is the publish from server
Meteor.publish('extra', function(obj){
var that = this;
Meteor.call('extra', obj, function(error, result){
if (result){
//console.log(result);
that.added("extra", "extra", {results: result});
//that.changed('extra','extra',{results: result});
that.ready();
} else {
//that.ready();
}
});
});
So my question is: Is there from scratch a better way to structuring my code means solving the problem somehow different? If not how can i achive it the cleanest way? Because for my understanding this is just strange way to do it.
EDIT:
For example.
Can i do a per-user-collection (maybe only client-side like now) and push data from the server and just subscribe to this collection? But then how can i check when the async API-Call has finshed to start the next round. So the view gets data piece by piece. I am just confused right now.

My fault was simple as i thaught: You don´t need to use subscribe.
I just added "error,result" in the callback of Meteor.call
Only "result" leads to the result is always undefined.
var cats = $(this.find(".extra")).attr('data-extra').split(',');
var shop = $(this.find(".extra")).attr('data-shop');
var counter = 0;
var callExtras = function(_counter){
var obj = {
categories : [cats[_counter]],
shop : shop
};
if(_counter <= cats.length){
Meteor.call('extra', obj,function(error,result){
var actual_session = Session.get('extra');
if(actual_session === false){
actual_session = [];
}
actual_session = actual_session.concat(result);
Session.set('extra',actual_session);
counter++;
callExtras(counter);
});
}
};
callExtras(counter);
Then in the template helper
"extra" : function(){
return Session.get('extra');
},

Assemble paginated ajax data in a Bacon FRP stream

I'm learning FRP using Bacon.js, and would like to assemble data from a paginated API in a stream.
The module that uses the data has a consumption API like this:
// UI module, displays unicorns as they arrive
beautifulUnicorns.property.onValue(function(allUnicorns){
console.log("Got "+ allUnicorns.length +" Unicorns");
// ... some real display work
});
The module that assembles the data requests sequential pages from an API and pushes onto the stream every time it gets a new data set:
// beautifulUnicorns module
var curPage = 1
var stream = new Bacon.Bus()
var property = stream.toProperty()
var property.onValue(function(){}) # You have to add an empty subscriber, otherwise future onValues will not receive the initial value. https://github.com/baconjs/bacon.js/wiki/FAQ#why-isnt-my-property-updated
var allUnicorns = [] // !!! stateful list of all unicorns ever received. Is this idiomatic for FRP?
var getNextPage = function(){
/* get data for subsequent pages.
Skipping for clarity */
}
var gotNextPage = function (resp) {
Array.prototype.push.apply(allUnicorns, resp) // just adds the responses to the existing array reference
stream.push(allUnicorns)
curPage++
if (curPage <= pageLimit) { getNextPage() }
}
How do I subscribe to the stream in a way that provides me a full list of all unicorns ever received? Is this flatMap or similar? I don't think I need a new stream out of it, but I don't know. I'm sorry, I'm new to the FRP way of thinking. To be clear, assembling the array works, it just feels like I'm not doing the idiomatic thing.
I'm not using jQuery or another ajax library for this, so that's why I'm not using Bacon.fromPromise
You also may wonder why my consuming module wants the whole set instead of just the incremental update. If it were just appending rows that could be ok, but in my case it's an infinite scroll and it should draw data if both: 1. data is available and 2. area is on screen.

This can be done with the .scan() method. And also you will need a stream that emits items of one page, you can create it with .repeat().
Here is a draft code (sorry not tested):
var itemsPerPage = Bacon.repeat(function(index) {
var pageNumber = index + 1;
if (pageNumber < PAGE_LIMIT) {
return Bacon.fromCallback(function(callback) {
// your method that talks to the server
getDataForAPage(pageNumber, callback);
});
} else {
return false;
}
});
var allItems = itemsPerPage.scan([], function(allItems, itemsFromAPage) {
return allItems.concat(itemsFromAPage);
});
// Here you go
allItems.onValue(function(allUnicorns){
console.log("Got "+ allUnicorns.length +" Unicorns");
// ... some real display work
});
As you noticed, you also won't need .onValue(function(){}) hack, and curPage external state.

Here is a solution using flatMap and fold. When dealing with network you have to remember that the data can come back in a different order than you sent the requests - that's why the combination of fold and map.
var pages = Bacon.fromArray([1,2,3,4,5])
var requests = pages.flatMap(function(page) {
return doAjax(page)
.map(function(value) {
return {
page: page,
value: value
}
})
}).log("Data received")
var allData = requests.fold([], function(arr, data) {
return arr.concat([data])
}).map(function(arr) {
// I would normally write this as a oneliner
var sorted = _.sortBy(arr, "page")
var onlyValues = _.pluck(sorted, "value")
var inOneArray = _.flatten(onlyValues)
return inOneArray
})
allData.log("All data")
function doAjax(page) {
// This would actually be Bacon.fromPromise($.ajax...)
// Math random to simulate the fact that requests can return out
// of order
return Bacon.later(Math.random() * 3000, [
"Page"+page+"Item1",
"Page"+page+"Item2"])
}
http://jsbin.com/damevu/4/edit

Integrating a link to my database within the Win 8 App Search Contract

In my Win 8 app, based on a blank template, I have successfully added search contract and it seems to work despite the fact that I have not linked it to any data yet, so, for now, when I search any term in my app it simply takes me to the searchResults page with the message "No Results Found" this is what I was expecting initially.
Now what I wish to do is link my database into the searchResults.js file so that I can query my database. Now outside of the search contract I have tested and connected my Db and it works; I did this using WinJS.xhr, to connect to my web-service which in turn queries my database and returns a JSON object.
In my test I only hardcoded the url, however I now need to do two things. Move the test WinJS.xr data for connecting my DB into the search contract code, and second - change the hardcoded url to a dynamic url that accepts the users search term.
From what I understand of Win 8 search so far the actual data querying part of the search contract is as follows:
// This function populates a WinJS.Binding.List with search results for the provided query.
_searchData: function (queryText) {
var originalResults;
// TODO: Perform the appropriate search on your data.
if (window.Data) {
originalResults = Data.items.createFiltered(function (item) {
return (item.termName.indexOf(queryText) >= 0 || item.termID.indexOf(queryText) >= 0 || item.definition.indexOf(queryText) >= 0);
});
} else {`enter code here`
originalResults = new WinJS.Binding.List();
}
return originalResults;
}
});
The code that I need to transfer into this section is as below; now I have to admit I do not currently understand the code block above and have not found a good resource for breaking it down line by line. If someone can help though it will be truly awesome! My code below, I basically want to integrate it and then make searchString be equal to the users search term.
var testTerm = document.getElementById("definition");
var testDef = document.getElementById("description");
var searchString = 2;
var searchFormat = 'JSON';
var searchurl = 'http://www.xxx.com/web-service.php?termID=' + searchString +'&format='+searchFormat;
WinJS.xhr({url: searchurl})
.done(function fulfilled(result)
{
//Show Terms
var searchTerm = JSON.parse(result.responseText);
// var terms is the key of the object (terms) on each iteration of the loop the var terms is assigned the name of the object key
// and the if stament is evaluated
for (terms in searchTerm) {
//terms will find key "terms"
var termName = searchTerm.terms[0].term.termName;
var termdefinition = searchTerm.terms[0].term.definition;
//WinJS.Binding.processAll(termDef, termdefinition);
testTerm.innerText = termName;
testDef.innerText = termdefinition;
}
},
function error(result) {
testDef.innerHTML = "Got Error: " + result.statusText;
},
function progress(result) {
testDef.innerText = "Ready state is " + result.readyState;
});

I will try to provide some explanation for the snippet that you didn't quite understand. I believe the code you had above is coming from the default code added by Visual Studio. Please see explanation as comments in line.
/**
* This function populates a WinJS.Binding.List with search results
* for the provided query by applying the a filter on the data source
* #param {String} queryText - the search query acquired from the Search Charm
* #return {WinJS.Binding.List} the filtered result of your search query.
*/
_searchData: function (queryText) {
var originalResults;
// window.Data is the data source of the List View
// window.Data is an object defined in YourProject/js/data.js
// at line 16 WinJS.Namespace.define（"Data" ...
// Data.items is a array that's being grouped by functions in data.js
if (window.Data) {
// apply a filter to filter the data source
// if you have your own search algorithm,
// you should replace below code with your code
originalResults = Data.items.createFiltered(function (item) {
return (item.termName.indexOf(queryText) >= 0 ||
item.termID.indexOf(queryText) >= 0 ||
item.definition.indexOf(queryText) >= 0);
});
} else {
// if there is no data source, then we return an empty WinJS.Binding.List
// such that the view can be populated with 0 result
originalResults = new WinJS.Binding.List();
}
return originalResults;
}
Since you are thinking about doing the search on your own web service, then you can always make your _searchData function async and make your view waiting on the search result being returned from your web service.
_searchData: function(queryText) {
var dfd = new $.Deferred();
// make a xhr call to your service with queryText
WinJS.xhr({
url: your_service_url,
data: queryText.toLowerCase()
}).done(function (response) {
var result = parseResultArrayFromResponse(response);
var resultBindingList = WinJS.Binding.List(result);
dfd.resolve(result)
}).fail(function (response) {
var error = parseErrorFromResponse(response);
var emptyResult = WinJS.Binding.List();
dfd.reject(emptyResult, error);
});
return dfd.promise();
}
...
// whoever calls searchData would need to asynchronously deal with the service response.
_searchData(queryText).done(function (resultBindingList) {
//TODO: Display the result with resultBindingList by binding the data to view
}).fail(function (resultBindingList, error) {
//TODO: proper error handling
});

Returning a value from a jQuery Ajax method

I'm trying to use Javascript in an OO style, and one method needs to make a remote call to get some data so a webpage can work with it. I've created a Javascript class to encapsulate the data retrieval so I can re-use the logic elsewhere, like so:
AddressRetriever = function() {
AddressRetriever.prototype.find = function(zip) {
var addressList = [];
$.ajax({
/* setup stuff */
success: function(response) {
var data = $.parseJSON(response.value);
for (var i = 0; i < data.length; i++) {
var city = data[i].City; // "City" column of DataTable
var state = data[i].State; // "State" column of DataTable
var address = new PostalAddress(postalCode, city, state); // This is a custom JavaScript class with simple getters, a DTO basically.
addressList.push(address);
}
}
});
return addressList;
}
}
The webpage itself calls this like follows:
$('#txtZip').blur(function() {
var retriever = new AddressRetriever();
var addresses = retriever.find($(this).val());
if (addresses.length > 0) {
$('#txtCity').val(addresses[0].getCity());
$('#txtState').val(addresses[0].getState());
}
});
The problem is that sometimes addresses is inexplicably empty (i.e. length = 0). In Firebug the XHR tab shows a response coming back with the expected data, and if I set an alert inside of the success method the length is correct, but outside of that method when I try to return the value, it's sometimes (but not always) empty and my textbox doesn't get populated. Sometimes it shows up as empty but the textbox gets populated properly anyways.
I know I could do this by getting rid of the separate class and stuffing the whole ajax call into the event handler, but I'm looking for a way to do this correctly so the function can be reused if necessary. Any thoughts?

In a nutshell, you can't do it the way you're trying to do it with asynchronous ajax calls.
Ajax methods usually run asynchronous. Therefore, when the ajax function call itself returns (where you have return addressList in your code), the actual ajax networking has not yet completed and the results are not yet known.
Instead, you need to rework how the flow of your code works and deal with the results of the ajax call ONLY in the success handler or in functions you call from the success handler. Only when the success handler is called has the ajax networking completed and provided a result.
In a nutshell, you can't do normal procedural programming when using asynchronous ajax calls. You have to change the way your code is structured and flows. It does complicate things, but the user experience benefits to using asynchronous ajax calls are huge (the browser doesn't lock up during a networking operation).
Here's how you could restructure your code while still keeping the AddressRetriever.find() method fairly generic using a callback function:
AddressRetriever = function() {
AddressRetriever.prototype.find = function(zip, callback) {
$.ajax({
/* setup stuff */
success: function(response) {
var addressList = [];
var data = $.parseJSON(response.value);
for (var i = 0; i < data.length; i++) {
var city = data[i].City; // "City" column of DataTable
var state = data[i].State; // "State" column of DataTable
var address = new PostalAddress(postalCode, city, state); // This is a custom JavaScript class with simple getters, a DTO basically.
addressList.push(address);
}
callback(addressList);
}
});
}
}
$('#txtZip').blur(function() {
var retriever = new AddressRetriever();
retriever.find($(this).val(), function(addresses) {
if (addresses.length > 0) {
$('#txtCity').val(addresses[0].getCity());
$('#txtState').val(addresses[0].getState());
}
});
});

AddressRetriever = function() {
AddressRetriever.prototype.find = function(zip) {
var addressList = [];
$.ajax({
/* setup stuff */
success: function(response) {
var data = $.parseJSON(response.value);
for (var i = 0; i < data.length; i++) {
var city = data[i].City; // "City" column of DataTable
var state = data[i].State; // "State" column of DataTable
var address = new PostalAddress(postalCode, city, state); // This is a custom JavaScript class with simple getters, a DTO basically.
addressList.push(address);
processAddresss(addressList);
}
}
});
}
}
function processAddresss(addressList){
if (addresses.length > 0) {
$('#txtCity').val(addresses[0].getCity());
$('#txtState').val(addresses[0].getState());
}
}
or if you want don't want to make another function call, make the ajax call synchronous. Right now, it is returning the array before the data is pushed into the array

Not inexplicable at all, the list won't be filled until an indeterminate amount of time in the future.
The canonical approach is to do the work in your success handler, perhaps by passing in your own callback. You may also use jQuery's .when.

AJAX calls are asynchroneous, which means they don't run with the regular flow of the program. When you execute
if (addresses.length > 0) {
addresses is in fact, empty, as the program did not wait for the AJAX call to complete.

Problems making GET request from jQuery

I'm trying to make an HTTP GET request using the jQuery get() function, but I'm having some trouble.
Here's what my code looks like:
// get the links on the page
var pageLinks = $.find('#pageLinks');
// loop through each of the links
$(pageLinks).find('a').each(function(){
if($(this).attr('title') !== "Next Page"){
// make a GET request to the URL of this link
$.get($(this).attr("href"), function(data) {
console.log("here");
var temp = parse_page(data);
// concatenate the return string with another
bdy = bdy+String(temp);
console.log("done");
});
}
});
There are multiple pages that I need to get data from. Since the get() function is asynchronous, I get the pages in a random order. Secondly, the concatenation does not work. Even though I get each of the pages, they're not put into bdy.
Can anyone suggest how I might deal with this?
Thanks a lot!!

Construct bdy after all pages are retrieved, i.e. store get results in a dictionary or array; wait for all gets to finish; then assemble them in the correct order.

I tried this one and it works:
// get the links on the page
var pageLinks = $('a');
var bdy
// loop through each of the links
$(pageLinks).each(function(){
console.log(this);
// make a GET request to the URL of this link
$.get($(this).attr("href"), function(data) {
// concatenate the return string with another
bdy = bdy + data.toString();
console.log(bdy);
});
});

As an example of what #muratgu has said:
var results = [];
var count = 0;
function allDone() {
var bdy = results.join("");
// do stuff with bdy
}
// get the links on the page
var pageLinks = $.find('#pageLinks');
// filter the links so we're left with the links we want
var wantedLinks = $(pageLinks).find('a').filter(function (idx) {
return $(this).attr('title') !== "Next Page";
});
// remember how many links we're working on
count = wantedLinks.length;
// loop through each of the links
wantedLinks.each(function (idx) {
// make a GET request to the URL of this link
$.get($(this).attr("href"), function (data) {
console.log("here");
var temp = parse_page(data);
results[idx] = temp;
// Decrement the count.
count--;
if (count === 0) {
// All done.
allDone();
}
});
});
You could go further and abstract this into a data type that can perform N async downloads, and then notify you when all are complete.

I just found that there are modules that allow one to manage the control flow in JS. The ones I found are:
Async
Step
For help using the above modules, see my follow up question here.