Develop Reference

Assemble paginated ajax data in a Bacon FRP stream

I'm learning FRP using Bacon.js, and would like to assemble data from a paginated API in a stream.
The module that uses the data has a consumption API like this:
// UI module, displays unicorns as they arrive
beautifulUnicorns.property.onValue(function(allUnicorns){
console.log("Got "+ allUnicorns.length +" Unicorns");
// ... some real display work
});
The module that assembles the data requests sequential pages from an API and pushes onto the stream every time it gets a new data set:
// beautifulUnicorns module
var curPage = 1
var stream = new Bacon.Bus()
var property = stream.toProperty()
var property.onValue(function(){}) # You have to add an empty subscriber, otherwise future onValues will not receive the initial value. https://github.com/baconjs/bacon.js/wiki/FAQ#why-isnt-my-property-updated
var allUnicorns = [] // !!! stateful list of all unicorns ever received. Is this idiomatic for FRP?
var getNextPage = function(){
/* get data for subsequent pages.
Skipping for clarity */
}
var gotNextPage = function (resp) {
Array.prototype.push.apply(allUnicorns, resp) // just adds the responses to the existing array reference
stream.push(allUnicorns)
curPage++
if (curPage <= pageLimit) { getNextPage() }
}
How do I subscribe to the stream in a way that provides me a full list of all unicorns ever received? Is this flatMap or similar? I don't think I need a new stream out of it, but I don't know. I'm sorry, I'm new to the FRP way of thinking. To be clear, assembling the array works, it just feels like I'm not doing the idiomatic thing.
I'm not using jQuery or another ajax library for this, so that's why I'm not using Bacon.fromPromise
You also may wonder why my consuming module wants the whole set instead of just the incremental update. If it were just appending rows that could be ok, but in my case it's an infinite scroll and it should draw data if both: 1. data is available and 2. area is on screen.

This can be done with the .scan() method. And also you will need a stream that emits items of one page, you can create it with .repeat().
Here is a draft code (sorry not tested):
var itemsPerPage = Bacon.repeat(function(index) {
var pageNumber = index + 1;
if (pageNumber < PAGE_LIMIT) {
return Bacon.fromCallback(function(callback) {
// your method that talks to the server
getDataForAPage(pageNumber, callback);
});
} else {
return false;
}
});
var allItems = itemsPerPage.scan([], function(allItems, itemsFromAPage) {
return allItems.concat(itemsFromAPage);
});
// Here you go
allItems.onValue(function(allUnicorns){
console.log("Got "+ allUnicorns.length +" Unicorns");
// ... some real display work
});
As you noticed, you also won't need .onValue(function(){}) hack, and curPage external state.

Here is a solution using flatMap and fold. When dealing with network you have to remember that the data can come back in a different order than you sent the requests - that's why the combination of fold and map.
var pages = Bacon.fromArray([1,2,3,4,5])
var requests = pages.flatMap(function(page) {
return doAjax(page)
.map(function(value) {
return {
page: page,
value: value
}
})
}).log("Data received")
var allData = requests.fold([], function(arr, data) {
return arr.concat([data])
}).map(function(arr) {
// I would normally write this as a oneliner
var sorted = _.sortBy(arr, "page")
var onlyValues = _.pluck(sorted, "value")
var inOneArray = _.flatten(onlyValues)
return inOneArray
})
allData.log("All data")
function doAjax(page) {
// This would actually be Bacon.fromPromise($.ajax...)
// Math random to simulate the fact that requests can return out
// of order
return Bacon.later(Math.random() * 3000, [
"Page"+page+"Item1",
"Page"+page+"Item2"])
}
http://jsbin.com/damevu/4/edit

AngularJS and Restangular, trying to convert update method to API

I'm trying to convert my basic crud operations into an API that multiple components of my application can use.
I have successfully converted all methods, except the update one because it calls for each property on the object to be declared before the put request can be executed.
controller
$scope.update = function(testimonial, id) {
var data = {
name: testimonial.name,
message: testimonial.message
};
dataService.update(uri, data, $scope.id).then(function(response) {
console.log('Successfully updated!');
},
function(error) {
console.log('Error updating.');
});
}
dataService
dataService.update = function(uri, data, id) {
var rest = Restangular.one(uri, id);
angular.forEach(data, function(value, key) {
// needs to be in the format below
// rest.key = data.key
});
// needs to output something like this, depending on what the data is passed
// rest.name = data.name;
// rest.message = data.message;
return rest.put();
}
I tried to describe the problem in the codes comments, but to reiterate I cannot figure out how to generate something like rest.name = data.name; without specifying the name property because the update function shouldn't need to know the object properties.
Here is what the update method looked like before I started trying to make it usable by any of my components (this works)
Testimonial.update = function(testimonial, id) {
var rest = Restangular.one('testimonials', id);
rest.name = testimonial.name;
rest.message = testimonial.message;
return rest.put();
}
How can I recreate this without any specific properties parameters hard-coded in?
Also, my project has included lo-dash, if that helps, I don't know where to start with this problem. Thanks a ton for any advice!

Try like
angular.extend(rest,testimonial)
https://docs.angularjs.org/api/ng/function/angular.extend

What problems would I run into if I make all Data available to all Controllers?

My AngularJS CRUD application processes it's information over a WebSocket Server. (This was mainly so that updates from one user would get automatically pushed to all users without the need for massive HTTP polling)
I realized early on that I would have to set up my services differently than I normally do with HTTP services. Normally, for each Model that I am working with, I give them their own service to populate that particular Model. However, this is not feasible with a Websocket Connection, because I don't want a separate connection for each service. Therefore, there are a couple of solutions.
1) set up a single service that establishes a connection, then share that connection with other services that will use that service to make their specific queries
2) make a single, type-agnostic service that will be used by all controllers that need access to the connection and data.
Option 2 seemed much easier to manage and would be reusable across applications, so I started on that. That was when I realized that this was actually an opportunity. Rather than explicitly creating models for each type of data that the Client could receive, I could create a master data object, and dynamically create child objects of myService.data as needed when data flows in from requests. Thus, if I ever need to update my Model, I just update the Model at the server level, and the client already knows how to receive it; it will just need a Controller that knows how to use it.
However, this opportunity brings a drawback. Apparently, because myService.Data is an empty, childless object at creation, any Scope that wants to reference its future children have to simple reference the object itself.
For example, $scope.user = myService.data.user throws an error, because that object doesn't exist at declaration. it would appear that my only option is for each controller to simply have $scope.data = myservice.data, and the view for each controller will simply have to use
< ng-model='data'>, with the declarations being something like {{data.user.username}}. I have tested it, and this does work.
My question is this; Is there any way I can get the best of both worlds? Can I have my service update it's data model dynamically, yet still have my controllers access only the part that they need? I? I was feeling quite clever until I realized that all of my Controllers were going to have access to the entire data model... But I honestly can't decide if that is even a huge problem.
Here is my Service:
app.factory('WebSocketService', ['$rootScope', function ($rootScope) {
var factory = {
socket: null,
data: {},
startConnection: function () {
//initialize Websocket
socket = new WebSocket('ws://localhost:2012/')
socket.onopen = function () {
//todo: Does anything need to happen OnOpen?
}
socket.onclose = function () {
//todo: Does anything need to happen OnClose?
}
socket.onmessage = function (event) {
var packet = JSON.parse(event.data);
////Model of Packet:
////packet.Data: A serialised Object that contains the needed data
////packet.Operation: What to do with the Data
////packet.Model: which child object of Factory.data to use
////packet.Property: used by Update and Delete to find a specific object with a property who's name matches this string, and who's value matches Packet.data
//Deserialize Data
packet.Data = JSON.parse(packet.Data);
//"Refresh" is used to completely reload the array
// of objects being stored in factory.data[packet.Model]
// Used for GetAll commands and manual user refreshes
if (packet.Operation == "Refresh") {
factory.data[packet.Model] = packet.Data
}
//Push is used to Add an object to an existing array of objects.
//The server will send this after somebody sends a successful POST command to the WebSocket Server
if (packet.Operation == "Push") {
factory.data[packet.Model].push(packet.Data)
}
if (packet.Operation == "Splice") {
for (var i = 0; i < factory.data[packet.Model].length; i++) {
for (var j = 0; j < packet.Data.length; j++){
if (factory.data[packet.Model][i][packet.Property] == packet.Data[j][packet.Property]) {
factory.data[packet.Model].splice(i, 1);
i--;
}
}
}
}
// Used to update existing objects within the Array. Packet.Data will be an array, although in most cases it will usually only have one value.
if (packet.Operation == "Update") {
for (var i = 0; i < factory.data[packet.Model].length; i++) {
for (var j = 0; j < packet.Data.length; j++) {
if (factory.data[packet.Model][i][packet.Property] == packet.Data[j][packet.Property]) {
factory.data[packet.Model][i] = packet.Data[j]
i--;
}
}
}
}
//Sent by WebSocket Server after it has properly authenticated the user, sending the user information that it has found.
if (packet.Operation == "Authentication") {
if (packet.Data == null) {
//todo: Authentication Failed. Alert User Somehow
}
else {
factory.data.user = packet.Data;
factory.data.isAuthenticated = true;
}
}
$rootScope.$digest();
}
},
stopConnection: function () {
if (socket) {
socket.close();
}
},
//sends a serialised command to the Websocket Server according to it's API.
//The DataObject must be serialised as a string before it can be placed into Packet object,which will also be serialised.
//This is because the Backend Framework is C#, which must see what Controller and Operation to use before it knows how to properly Deserialise the DataObject.
sendPacket: function (Controller, Operation, DataObject) {
if (typeof Controller == "string" && typeof Operation == "string") {
var Data = JSON.stringify(DataObject);
var Packet = { Controller: Controller, Operation: Operation, Data: Data };
var PacketString = JSON.stringify(Packet);
socket.send(PacketString);
}
}
}
return factory
}]);
Here is a Simple Controller that Accesses User Information. It is actually used in a permanent header <div> in the Index.html, outside of the dynamic <ng-view>. It is responsible for firing up the Websocket Connection.
App.controller("AuthenticationController", function ($scope, WebSocketService) {
init();
function init() {
WebSocketService.startConnection();
}
//this is the ONLY way that I have found to access the Service Data.
//$scope.user = WebSocketService.data.user doesn't work
//$scope.user = $scope.data.user doesn't even work
$scope.data = WebSocketService.data
});
And here is the HTML that uses that Controller
<div data-ng-controller="AuthenticationController">
<span data-ng-model="data">{{data.user.userName}}</span>
</div>

One thing you could do is store the data object on the root scope, and set up watches on your various controllers to watch for whatever controller-specific keys they need:
// The modules `run` function is called once the
// injector is finished loading all its modules.
App.run(function($rootScope, WebSocketService) {
WebSocketService.startConnection();
$rootScope.socketData = WebSocketService.data;
});
// Set up a $watch in your controller
App.controller("AuthenticationController", function($scope) {
$scope.$watch('socketData.user', function(newUser, oldUser) {
// Assign the user when it becomes available.
$scope.user = newUser;
});
});

Backbone - Check if Record Already Exists - Is there a better way than this?

In Backbone, I need to check if a model record already exists. Right now, I am doing it by fetching the model by id, and seeing if its "created_at" attribute is undefined. This feels brittle to me. Does anyone have any better recommendations?
var dealProgram = new WhiteDeals.Models.DealProgram({id: servant_id});
dealProgram.fetch({
success: function() {
var program = dealProgram.toJSON();
var datecheck = program.created_at
if(typeof datecheck === 'undefined'){
dealPrograms.create({
title: "",
servant_id: servant.servant_id,
servant_name: servant.name,
servant_master: servant.master
},
{
success: function () {
self.manageServants(servants);
}
}); // End of dealPrograms.create
} else if (datecheck !== undefined) {
console.log("is defined, success!")
self.manageServants(servants);
}; // End of if statement for non-existant dealPrograms
} // End of success
}); // End of dealProgram.fetch

You'll obviously have to check by using a request (whatever the form of the request, you'll have a low amount of data anyway). Guess you should still wonder if it wouldn't be worth fetching all your models at once in a collection so you can make the check client-side (or only the id if it'd be too big to fetch everything).

Returning a value from a jQuery Ajax method

I'm trying to use Javascript in an OO style, and one method needs to make a remote call to get some data so a webpage can work with it. I've created a Javascript class to encapsulate the data retrieval so I can re-use the logic elsewhere, like so:
AddressRetriever = function() {
AddressRetriever.prototype.find = function(zip) {
var addressList = [];
$.ajax({
/* setup stuff */
success: function(response) {
var data = $.parseJSON(response.value);
for (var i = 0; i < data.length; i++) {
var city = data[i].City; // "City" column of DataTable
var state = data[i].State; // "State" column of DataTable
var address = new PostalAddress(postalCode, city, state); // This is a custom JavaScript class with simple getters, a DTO basically.
addressList.push(address);
}
}
});
return addressList;
}
}
The webpage itself calls this like follows:
$('#txtZip').blur(function() {
var retriever = new AddressRetriever();
var addresses = retriever.find($(this).val());
if (addresses.length > 0) {
$('#txtCity').val(addresses[0].getCity());
$('#txtState').val(addresses[0].getState());
}
});
The problem is that sometimes addresses is inexplicably empty (i.e. length = 0). In Firebug the XHR tab shows a response coming back with the expected data, and if I set an alert inside of the success method the length is correct, but outside of that method when I try to return the value, it's sometimes (but not always) empty and my textbox doesn't get populated. Sometimes it shows up as empty but the textbox gets populated properly anyways.
I know I could do this by getting rid of the separate class and stuffing the whole ajax call into the event handler, but I'm looking for a way to do this correctly so the function can be reused if necessary. Any thoughts?

In a nutshell, you can't do it the way you're trying to do it with asynchronous ajax calls.
Ajax methods usually run asynchronous. Therefore, when the ajax function call itself returns (where you have return addressList in your code), the actual ajax networking has not yet completed and the results are not yet known.
Instead, you need to rework how the flow of your code works and deal with the results of the ajax call ONLY in the success handler or in functions you call from the success handler. Only when the success handler is called has the ajax networking completed and provided a result.
In a nutshell, you can't do normal procedural programming when using asynchronous ajax calls. You have to change the way your code is structured and flows. It does complicate things, but the user experience benefits to using asynchronous ajax calls are huge (the browser doesn't lock up during a networking operation).
Here's how you could restructure your code while still keeping the AddressRetriever.find() method fairly generic using a callback function:
AddressRetriever = function() {
AddressRetriever.prototype.find = function(zip, callback) {
$.ajax({
/* setup stuff */
success: function(response) {
var addressList = [];
var data = $.parseJSON(response.value);
for (var i = 0; i < data.length; i++) {
var city = data[i].City; // "City" column of DataTable
var state = data[i].State; // "State" column of DataTable
var address = new PostalAddress(postalCode, city, state); // This is a custom JavaScript class with simple getters, a DTO basically.
addressList.push(address);
}
callback(addressList);
}
});
}
}
$('#txtZip').blur(function() {
var retriever = new AddressRetriever();
retriever.find($(this).val(), function(addresses) {
if (addresses.length > 0) {
$('#txtCity').val(addresses[0].getCity());
$('#txtState').val(addresses[0].getState());
}
});
});

AddressRetriever = function() {
AddressRetriever.prototype.find = function(zip) {
var addressList = [];
$.ajax({
/* setup stuff */
success: function(response) {
var data = $.parseJSON(response.value);
for (var i = 0; i < data.length; i++) {
var city = data[i].City; // "City" column of DataTable
var state = data[i].State; // "State" column of DataTable
var address = new PostalAddress(postalCode, city, state); // This is a custom JavaScript class with simple getters, a DTO basically.
addressList.push(address);
processAddresss(addressList);
}
}
});
}
}
function processAddresss(addressList){
if (addresses.length > 0) {
$('#txtCity').val(addresses[0].getCity());
$('#txtState').val(addresses[0].getState());
}
}
or if you want don't want to make another function call, make the ajax call synchronous. Right now, it is returning the array before the data is pushed into the array

Not inexplicable at all, the list won't be filled until an indeterminate amount of time in the future.
The canonical approach is to do the work in your success handler, perhaps by passing in your own callback. You may also use jQuery's .when.

AJAX calls are asynchroneous, which means they don't run with the regular flow of the program. When you execute
if (addresses.length > 0) {
addresses is in fact, empty, as the program did not wait for the AJAX call to complete.