Develop Reference

How to retrieve both caller-name and carrier from Twilio in node.js?

Im having problems working with the twilio-api for node.
i wrote this code:
let typeArray = ['caller-name','carrier'];
this.client.phoneNumbers(phoneNumberToCheck).get({
type: typeArray
}, (error, number) => {
// working on the number data results
// ...
});
The problem is that i dont get ANY of them(carrier/caller-name) - although passing array to argument 'type' is the way to do it in other languages(php,c#..) but it doesnt work on node.js, instead i get this:
// -> get
{
"caller_name":null,
"country_code":"US",
"phone_number":"+123456789",
"national_format":"(248) 123-456",
"carrier":null,
"add_ons":null,
"url":"https://lookups.twilio.com/v1/PhoneNumbers/+123456789",
"callerName":null,
"countryCode":"US",
"phoneNumber":"+123456789",
"nationalFormat":"(248) 123-456",
"addOns":null
}
note: if i send each one separately (only carrier or only caller-name) - i get the partial information for each.
how can i get both in one request in node.js?

Twilio developer evangelist here.
You should be calling the Lookups API in Node this way:
client.lookups.phoneNumbers.get(phoneNumber)
.fetch({
type: ['carrier', 'caller-name']
},
function(err, result) {
// do something
}
)
The docs are a little lacking in Node.js on the Lookups documentation and I will raise that with the team.

Querying a parse table and eagerly fetching Relations for matching

Currently, I have a table named Appointments- on appointments, I have a Relation of Clients.
In searching the parse documentation, I haven't found a ton of help on how to eagerly fetch all of the child collection of Clients when retrieving the Appointments. I have attempted a standard query, which looked like this:
var Appointment = Parse.Object.extend("Appointment");
var query = new Parse.Query(Appointment);
query.equalTo("User",Parse.User.current());
query.include('Rate'); // a pointer object
query.find().then(function(appointments){
let appointmentItems =[];
for(var i=0; i < appointments.length;i++){
var appt = appointments[i];
var clientRelation = appt.relation('Client');
clientRelation.query().find().then(function(clients){
appointmentItems.push(
{
objectId: appt.id,
startDate : appt.get("Start"),
endDate: appt.get("End"),
clients: clients, //should be a Parse object collection
rate : appt.get("Rate"),
type: appt.get("Type"),
notes : appt.get("Notes"),
scheduledDate: appt.get("ScheduledDate"),
confirmed:appt.get("Confirmed"),
parseAppointment:appt
}
);//add to appointmentitems
}); //query.find
}
});
This does not return a correct Clients collection-
I then switched over to attempt to do this in cloud code- as I was assuming the issue was on my side for whatever reason, I thought I'd create a function that did the same thing, only on their server to reduce the amount of network calls.
Here is what that function was defined as:
Parse.Cloud.define("GetAllAppointmentsWithClients",function(request,response){
var Appointment = Parse.Object.extend("Appointment");
var query = new Parse.Query(Appointment);
query.equalTo("User", request.user);
query.include('Rate');
query.find().then(function(appointments){
//for each appointment, get all client items
var apptItems = appointments.map(function(appointment){
var ClientRelation = appointment.get("Clients");
console.log(ClientRelation);
return {
objectId: appointment.id,
startDate : appointment.get("Start"),
endDate: appointment.get("End"),
clients: ClientRelation.query().find(),
rate : appointment.get("Rate"),
type: appointment.get("Type"),
notes : appointment.get("Notes"),
scheduledDate: appointment.get("ScheduledDate"),
confirmed:appointment.get("Confirmed"),
parseAppointment:appointment
};
});
console.log('apptItems Count is ' + apptItems.length);
response.success(apptItems);
})
});
and the resulting "Clients" returned look nothing like the actual object class:
clients: {_rejected: false, _rejectedCallbacks: [], _resolved: false, _resolvedCallbacks: []}
When I browse the data, I see the related objects just fine. The fact that Parse cannot eagerly fetch relational queries within the same call seems a bit odd coming from other data providers, but at this point I'd take the overhead of additional calls if the data was retrieved properly.
Any help would be beneficial, thank you.

Well, in your Cloud code example - ClientRelation.query().find() will return a Parse.Promise. So the output clients: {_rejected: false, _rejectedCallbacks: [], _resolved: false, _resolvedCallbacks: []} makes sense - that's what a promise looks like in console. The ClientRelation.query().find() will be an async call so your response.success(apptItems) is going to be happen before you're done anyway.
Your first example as far as I can see looks good though. What do you see as your clients response if you just output it like the following? Are you sure you're getting an array of Parse.Objects? Are you getting an empty []? (Meaning, do the objects with client relations you're querying actually have clients added?)
clientRelation.query().find().then(function(clients){
console.log(clients); // Check what you're actually getting here.
});
Also, one more helpful thing. Are you going to have more than 100 clients in any given appointment object? Parse.Relation is really meant for very large related collection of other objects. If you know that your appointments aren't going to have more than 100 (rule of thumb) related objects - a much easier way of doing this is to store your client objects in an Array within your Appointment objects.
With a Parse.Relation, you can't get around having to make that second query to get that related collection (client or cloud). But with a datatype Array you could do the following.
var query = new Parse.Query(Appointment);
query.equalTo("User", request.user);
query.include('Rate');
query.include('Clients'); // Assumes Client column is now an Array of Client Parse.Objects
query.find().then(function(appointments){
// You'll find Client Parse.Objects already nested and provided for you in the appointments.
console.log(appointments[0].get('Clients'));
});

I ended up solving this using "Promises in Series"
the final code looked something like this:
var Appointment = Parse.Object.extend("Appointment");
var query = new Parse.Query(Appointment);
query.equalTo("User",Parse.User.current());
query.include('Rate');
var appointmentItems = [];
query.find().then(function(appointments){
var promise = Parse.Promise.as();
_.each(appointments,function(appointment){
promise = promise.then(function(){
var clientRelation = appointment.relation('Clients');
return clientRelation.query().find().then(function(clients){
appointmentItems.push(
{
//...object details
}
);
})
});
});
return promise;
}).then(function(result){
// return/use appointmentItems with the sub-collection of clients that were fetched within the subquery.
});
You can apparently do this in parallel, but that was really not needed for me, as the query I'm using seems to return instantaniously. I got rid of the cloud code- as it didnt seem to provide any performance boost. I will say, the fact that you cannot debug cloud code seems truly limiting and I wasted a bit of time waiting for console.log statements to show themselves on the log of the cloud code panel- overall the Parse.Promise object was the key to getting this to work properly.

Recursion and asynchronous methods with a flag variable

I have the following method:
function getRelevantArticles(amount,
userSuggestions,
suggestionPage,
relevantArticles,
continueFlag,
success,
error)
{
if(continueFlag)
{
getSuggestedArticles(suggestionPage, userSuggestions, function (articles)
{
if(articles.length == 0)
getRelevantArticles(amount, userSuggestions, suggestionPage, relevantArticles, false, success, error); // continueFlag= false
getUnvisitedArticles(articles, function (unvisited)
{
for(var i = 0; i < unvisited.length; i++)
relevantArticles.push(unvisited[i]);
relevantArticles= filterRelevant(amount, userSuggestions, relevantArticles);
if(relevantArticles.length < amount)
getRelevantArticles(amount, userSuggestions, suggestionPage + 1, relevantArticles, true, success, error); // continueFlag= true
else
getRelevantArticles(amount, userSuggestions, suggestionPage, relevantArticles, false, success, error); // continueFlag= false
}, error);
}, error);
}
else if(success)
{
fillWithContent(relevantArticles, success, error); //Should be last method to execute
}
}
The context
I know it may be hard to understand and can be optimized a lot, I'll try my best to explain what it's doing (or trying to):
The method is first called with a flag of continueFlag= true, so it starts by calling getSuggestedArticles, which is an async method that makes an AJAX request. I pass in a callback function with the result of the request.
getSuggestedArticles gets me the article Ids related to the user suggestions. (The user suggestions is a list of topics that may be of interest to the user).
I pass in a suggestionPage because the suggestions can be a lot, and I should be able to get the relevant articles with just a few (the first page).
If no articles are retrieved, then it means we are out of suggestions (every suggestion has at least one article), i. e., we reached the last page, so we set the continueFlag flag to false, to call the finalizer method.
If there is at least one article, I call getUnvisitedArticles, which is another asynchronous method that makes an AJAX request. This method gives me the articles that were not visited or read by the user, which are the ones I care about.
I have a relevantArticles variable which keeps track of the articles that I found out to be relevant and will be presented to the user. Once I get the relevant ones from my current page of unvisited articles, and have appended them to the previous page's relevant ones, I check if I have the minimum amount of articles to show.
If I don't yet satisfy the minimum amount, then I go on with the next page (suggestionPage + 1);
If I reached the minimum threshold, then I go on with the finalizer method (continueFlag= false)
fillWithContent is the method which will be called when I finished identifying the relevant articles. It is an asynchronous method that will make an AJAX request, and will fill my article objects with additional information.
getSuggestedArticles(Number: suggestionPage, Array: userSuggestions, Function: success, Function: error)
Receives an array of user suggestions, and takes the nth page of this array (page size 100).
Let's suppose we call the method like this:
getSuggestedArticles(0, [ 745, 4567, 1500 ], function (data) {
var articles = data;
}, error);
The method makes request to Database Web API, and pass an array of suggested articles to the success function. In the preceding example, the articles variable would have an array like this (Note that all the returned suggested articles have at least one user suggestion among their topics):
[
{
id: 12345,
topics: [ 998, 1500, 323 ] //has user suggestion 1500
},
{
id: 45778,
topics: [ 009, 1500, 745] //Has user suggestion 745 and 1500
},
...
]
getUnvisitedArticles(Array: articles, Function: success, Function: error)
Receives an array of articles and returns all the ones who weren't visited by the user.
Let's suppose we call this method like this:
//We are using the same "articles" variable from the previous example
getUnvisitedArticles(articles, function (data) {
var unvisited = data;
}, error);
The function makes request to the Database Web Api and pass an array with the unvisited articles to the success function. In the preceding example, the variable unvisited would have an array like this:
[
{
id: 45778,
topics: [ 009, 1500, 745]
}
]
Notice that article with id 12345 is gone. This is because it has been visited by the user.
fillWithContent(Array: relevantArticles, Function: success, Function: error)
Receives an array of articles and fill these objects with additional information.
Let's suppose we call this method like this:
//We are using the same "unvisited" variable from the previous example
fillWithContent(unvisited, function (data) {
filledArticles = data;
}, error);
The function makes request to the Database Web Api and pass an array with the filled articles to the success function. In the preceding example, the variable filledArticles would have an array like this:
[
{
id: 45778,
topics: [009, 1500, 745],
title: 'Article title',
publicationDate: 'Some date',
author: 'Some author',
...
}
]
This is the array my caller is expeting, by caller I mean the one calling my getRelevantArticles function.
The problem
The problem with this method is that fillWithContent is called infinitely, therefore causing a great deal of requests to be made, the browser to crash, and a recursion overflow to arise.
I'm not calling this method from another place, so it has to be a problem with this function.
I writed a console.log(suggestionPage), and it seems it keeps on incrementing the variable infinitely too. It should have stopped at page 3, because articles.length == 0. But it's not stopping.
What is going on here?

I think you should split your workload into independent parts that are easy reason about and easy to combine for more complex results.
As I understand it, the work consists three basic parts, all done through Ajax:
[topic IDs] to [article stubs for those topics] (getSuggestedArticles)
[article stubs] to [relevant (unread) article stubs] (getUnvisitedArticles)
[article stubs] to [full articles] (fillWithContent)
All of those Ajax requests should be made in a paged manner, e.g. 1000 items are handled by 10 requests with 100 items each.
For that we define a utility function that takes a list of items, pages them, makes an Ajax request for each page (through a worker function we pass as an argument) and returns the combined result of all requests.
// utility: runs an ajax request and handles errors on the low level
function ajax(options) {
return $.ajax(options).fail(function(jqXHR, textStatus, errorThrown) {
console.log(textStatus, errorThrown, jqXHR);
});
}
// utility: makes paged Ajax requests through a worker function
function pagedAjax(ajaxFunc, items, pageSize) {
var temp = [].slice.call(items), page,
requests = [];
// start as many parallel Ajax requests as we have pages
while (temp.length) {
page = temp.splice(0, pageSize);
requests.push( ajaxFunc(page) );
}
// wait until all requests have finished, return combined result
return $.when.apply($, requests).then(function (results) {
var combined = [];
$.each(results, function (i, result) {
// result is an array [data, textStatus, jqXhr]
// push all contained items onto combined
// (the following assumes that data is an array of objects)
[].push.apply(combined, result[0]);
});
return combined;
});
}
Now we can set up our three worker functions. They take arbitrary amounts of input, because all paging is done by the utility function above:
// worker: retrieves a list of article IDs from topic IDs
function getSuggestedArticles(topics) {
return ajax({method: 'post', url: '/articlesByTopic', data: topics});
// or whatever API request returns a list of articles IDs from topic IDs
}
// worker: takes a list of article IDs, returns a list of _unread_ article IDs
function getUnvisitedArticles(articles) {
return ajax({method: 'post', url: '/unvisitedArticles', data: articles});
// or whatever API request returns a list of unvisited articles from IDs
}
// worker: takes a list of article IDs, returns a list of articles
function fillWithContent(articles) {
return ajax({method: 'post', url: '/articles', data: articles});
// or whatever API request fills articles with content
}
After that, combining the functions is not difficult anymore:
// takes a list of topic IDs, requests article IDs, filters them, returns actual articles
function getUnvisitedArticlesByTopic(topicIds) {
var pageSize = 100;
return pagedAjax(getSuggestedArticles, topicIds, pageSize)
.then(function (allArticles) {
return pagedAjax(getUnvisitedArticles, allArticles, pageSize);
})
.then(function (unvisitedArticles) {
return pagedAjax(fillWithContent, unvisitedArticles, pageSize);
});
}
And we can use it all through a very simple call:
// renders unvisited articles
function renderUnvisitedArticles() {
var topicIds = [9, 1500, 745];
getUnvisitedArticlesByTopic(topicIds).done(function (articles) {
$.each(articles, function (i, article) {
// show article on page
});
});
}
Benefits of this promise-based approach:
No callback hell.
Short functions that do exactly one thing.
No self-calling functions.
Good re-usability and testability of the individual parts.
Recommended reading is of course jQuery's documentation on Deferred objects.
Disclaimer: The code is indeed untested. If you find mistakes, tell me.

Chaining models relations in node.js (node-orm) like in Rails

I'm building a relatively big NodeJS application, and I'm currently trying to figure out how to fetch the data I need from the DB. Here is a part of my models :
One user has one role, which has access to many modules (where there's a table role_modules to link roles and modules).
In Rails, I would do something like user.role.modules to retrieve the list of the modules he has access to. In NodeJS it's a bit more complicated. I'm using node-orm2 along with PostgreSQL. Here is what I have so far:
req.models.user.find({email: req.body.user}, function(err, user) {
user[0].getRole(function(err, role) {
role.getModules(function(err, modules) {
var list_modules = Array();
modules.forEach(function(item) {
console.log(item);
list_modules.push(item.name);
})
But I can't do this, because item only contains role_id and module_id. If I want to have the name, I would have to do item.getModule(function() {...}), but the results would be asynchronous ... so I don't really see how I could end up with an array containing the names of the modules a user has access to ... have any idea?
Also, isn't that much slower than actually running a single SQL query with many JOIN? Because as I see it, the ORM makes multiple queries to get the data I want here...
Thank you!

I wrote an ORM called bookshelf.js that aims to simplify associations and eager loading relations between models in SQL. This is what your query would probably look like to load the role & modules on a user given your description:
var Module = Bookshelf.Model.extend({
tableName: 'modules'
});
var Role = Bookshelf.Model.extend({
tableName: 'roles',
modules: function() {
return this.belongsToMany(Module);
}
});
var User = Bookshelf.Model.extend({
tableName: 'users'
role: function() {
return this.hasOne(Role);
}
});
User.forge({email: req.body.user})
.fetch({
require: true,
withRelated: ['role.modules']
})
.then(function(user) {
// user, now has the role and associated modules eager loaded
console.log(user.related('role'));
console.log(user.related('role').related('modules'))
}, function(err) {
// gets here if no user was found.
});
Might be worth taking a look at.

backbone.js - getting extra data along with the request

I have a collection which holds some of the users. Some information that is needed is how many total there are, how many pages, etc. How do I pass these back to the client? Or do they have to come from a separate view in which case I will need more than one ajax call? I'd like to have the collection fetch() and also receive some of this "meta data". What's a good way for handling this?

Generally, you need to handle this in the collection class' parse method. Its responsibility is to take the response and return back an array of model attributes. However, you could do more than that if you wished if you didn't mind the parse method having this additional responsibility.
UserList = Backbone.Collection.extend({
model: User,
url: '/users',
parse: function(data) {
if (!data) {
this.registered_users = 0;
return [];
}
this.registered_users = data.registered_users;
var users = _(data.users).map(
function(user_data) {
var user = {};
user['name'] = user_data.name;
return user;
}
);
return users;
    }
});
So in the trivial example above, presume the server returns a response which contains a count of registered users and and an array of user attributes. You would both parse through and return the user attributes and you would pick off the registered user count and just set it as a variable on the model.
The parse method would get called as part of a fetch. So no need to modify the fetch, just use the built-in hook method that you have.
Purists would say that you are giving the parse method a secondary responsibility which might surprise some people (e.g. returning something and modifying model state). However, I think this is okay.

One way to do this is to override the Collection::fetch() method so that it parses this metadata out of the response. You could have your backend return a response like this:
{
"collection": [
{ ... model 1 ... },
{ ... model 2 ... },
...
],
"total_rows": 98765,
"pages": 43
}
In your fetch method which overrides the original Backbone.Collection::fetch() method, you can handle each property of the object separately. Here's you could do the override with a slightly modified fetch method:
_.extend(Backbone.Collection.prototype, {
fetch : function(options) {
options || (options = {});
var collection = this;
var success = options.success;
options.success = function(resp) {
// Capture metadata
if (resp.total_rows) collection.total_rows = resp.total_rows;
if (resp.pages) collection.pages = resp.pages;
// Capture actual model data
collection[options.add ? 'add' : 'refresh'](
collection.parse(resp.collection), options);
// Call success callback if necessary
if (success) success(collection, resp);
};
options.error = wrapError(options.error, collection, options);
(this.sync || Backbone.sync).call(this, 'read', this, options);
return this;
});
Note that this approach using _.extend will affect all your classes which extend Backbone.Collection.
This way, you don't have to make 2 separate calls to the backend.

I would bootstrap the information at pagecreation. Write the information into the html document when the server creates the site. Like that you don't have to have an ajax call at all. I do that with the whole collection in ordner not to first load the page and then wait for the ajax call to return the information needed.
Code example with Python:
Line 64: https://github.com/ichbinadrian/maps/blob/master/python/main.py <- from here
Line 43: https://github.com/ichbinadrian/maps/blob/master/templates/index.html <- into here