Develop Reference

aws personalize putevents does not update recommendations

I'm trying to use AWS Personalize. After creating dataset and batch inference, I am updating the user-item-interactions with personalize.putEvents (using Javascript SDK, docs)
Snippet:
const awsOpts = { apiVersion, accessKeyId, secretAccessKey, region }
const pEvents = new AWS.PersonalizeEvents(awsOpts)
// ...
const params = {
trackingId, userId, sessionId,
eventList: [{
eventId: (+sentAt) + "",
sentAt,
eventType,
properties: { itemId }
}]
}
pEvents.putEvents(params, (err, data) => err ? reject(err) : resolve(data))
The events seem to be registered. No errors. After that when I create another batch-inference, I would expect that the new user-items would not appear in the recommendations anymore. But the recommendations in the next batch-inference are unchanged. Am I doing something wrong or am I misunderstanding the putEvents-API-call?
Schema for reference:
{
"type": "record",
"name": "Interactions",
"namespace": "com.amazonaws.personalize.schema",
"fields": [
{
"name": "USER_ID",
"type": "string"
},
{
"name": "ITEM_ID",
"type": "string"
},
{
"name": "EVENT_TYPE",
"type": "string"
},
{
"name": "TIMESTAMP",
"type": "long"
}
],
"version": "1.0"
}
One thing seems a bit strange: Cloud watch reports that the lambda was executed twice despite no errors nor timeout exceeded (timeout is set to 10s, and the lambda takes less than 2s). Also Retry attempts is set to 2.

#D.J.Duff (sadly I can't comment)
Are you sure the events added using PutEvent API are considered without retraining ? I have been through the AWS Personalize Doc looking for exactly that and it looked to me that you need to retrain to get those events included and for the runtime api to be able to consider them. Could you point to me where you saw that they are considered without having to retrain ? Thanks

User-item recommendations, at least from the runtime API should change without retraining after some events (that is what it is for and that is how we use it), though perhaps (it is something to check) you need to use the runtime API to see the new recommendations.
(edited to clarify that that the API is called the "runtime" API - thanks to PatrykMilewski for seeking the clarification - and that I am by no means certain that the particular API used is the important thing - I do know that when using the runtime API, the events do have an effect though).

I have tried out the very same usecase with personalize using boto3 sdk. Yes!! Put events can be used to update user-item-interactions. No need to retrain the model after put events. Looks like aws personalize solutions are made compatible to handle the data updated with put events. When I run the next batch inference, after updating interactions with put events API, I could see the change in the recommendations. I verified this with personalized ranking recipe. I could see the recent interacted items getting re-ranked according users recent interactions.

Need Help to implement Tincan Javascript API

I'm working on tincan JavaScript API. The issue my data format is total change and TinCan have specified a why to pass data along with call. Help me to adjust my data in TinCan Api format. Here is sample data one of my call.
var data = {
"groupId": "groupId",
"groupName": "gNameEncrypt",
"tutorNames": "tutorNames",
"actorNames": "actorNames",
"otherNames": "otherNames"
};
Current what i do i simply decode this data and send it like this.
var actionList = new TinCan(
{
recordStores: [{
endpoint: "http://example.com",
username: username,
password: password,
allowFail: false
}]
});
var action = new TinCan.Agent({
"name": "insert"
});
actionList.getStatements({
'params': {
'agent': action,
'verb': {
'id': $.base64.encode(data)
}
},
'callback': function (err, data) {
console.info(data.more);
var urlref = "http://<?php echo $_SERVER['SERVER_NAME'] . ":" . $_SERVER['SERVER_PORT'] . $uriParts[0] . "?" ?>t=" + data.more.TutorToken;
window.location.href = urlref;
}
});
crypt.finish();
});

There are really two parts here:
need to get data into an xAPI (formerly Tin Can) format, and
the code itself.
In depth,
I think you need to take another look at how xAPI is used in general. Data is stored a JSON "Statement" object that has 3 required properties and various other optional ones. These properties often contain complex objects that are very extensible. It is hard to tell from what you've shown what you are really trying to capture and what the best approach would be. I suggest reading some material about the xAPI statement format. http://experienceapi.com/statements-101/ is a good starting point, and to get at least some coverage of all the possibilities continue with http://experienceapi.com/statements/ .
The code you've listed is attempting to get already stored statements based on two parameters rather than trying to store a statement. The two parameters being "agent" and "verb". In this case We can't tell what the verb is supposed to be since we don't know what data contains, I suspect this isn't going to make sense as a verb which is intended to be the action of a statement. Having said that the fact that the "actor" has a value of action is questionable, as that really sounds more like what a "verb" should contain. Getting the statements right as part of #1 should make obvious how you would retrieve those statements. As far as storing those statements, if you're using the TinCan interface object you would need to use the sendStatement method of that object. But this interface is no longer recommended, the recommended practice is to construct a TinCan.LRS object and interact directly with it, in which case you'd be using the saveStatement method.
I would recommend looking at the "Basic Usage" section of the project home page here: http://rusticisoftware.github.io/TinCanJS/ for more specifics look at the API doc: http://rusticisoftware.github.io/TinCanJS/doc/api/latest/

Best practice creating a key/value map dev/prod node.js

I have a Node.js app, APP-A, that communicates with another C# app, APP-B, using APP-B's API. APP-B has a RESTful API that returns JSON. Other than a few standard fields e.g., name, description, APP-B's keys are defined when the user creates the field in the system. The resulting JSON looks like this:
{
"name": "An example name",
"description": "Description for the example",
"cust_fields": {
"cust_123": "Joe Bloggs",
"cust_124": "Essex"
}
}
I have two instances of APP-B, a dev and prod environment, which are separate installations. As a result, the JSON from the prod environment is as above, and the JSON from the dev environment looks like this:
{
"name": "An example name",
"description": "Description for the example",
"cust_fields": {
"cust_782": "Joe Bloggs",
"cust_793": "Essex"
}
}
This is dealt with in APP-A (the Node.js app) by having a JSON map like this:
{
"name": "name",
"description": "description",
"cust_fields": {
"full_name": "cust_123",
"city": "cust_124"
}
}
Which is loaded like this:
var map;
switch(env) {
case 'dev':
map = require('../env/dev/map.json');
break;
case 'prod':
map = require('../env/prod/map.json');
break;
};
module.exports = {
name: map.name,
description: map.description,
cust_fields: {
full_name: map.cust_fields.full_name,
city: map.cust_fields.city,
}
}
So I am wondering, is there is a better way of dealing with this? I don't see a way around having to create some kind of manual relationship between the key names across prod and dev, as there is no way to find out what field corresponds to what, but it seems like a lot of work.
Thanks for reading.
Update:
I have created a jsFiddle to better illustrate my question: http://jsfiddle.net/7k9k03o6.

If the mapping is unavoidable and everything is done manually right now, the next best progression would be to automate the building of those lookup maps, through some persistent storage, i.e. a database.
The general flow would be:
When APP-B creates a new form, that field information is stored in the database with all the identifying information. You could store production and dev data in the same db (as a flag) but likely they would just be different databases. Structure might be like customerId, formId, fieldName, fieldMapping, fieldValue, isProduction --> 123, 2, 'cust_124', 'city', 'Essex', true
When APP-A needs a field listing, it queries the DB for the relevant field lists."Find mapping customer X for form Y in production" --> WHERE custId = 123 AND formId = 2 AND isProduction = true would yield a list of fields and their mapping values (which you would post process/reduce into the mapping you need).
This automated process will leave less work for you manually. You shouldn't accidentally miss or forget a mapping from the hand generated file.
This will add a tiny bit of work to the server processing, as you'll need the field mapping from the DB every time a request is processed. (You could back off a bit and do one big query each time a customer is loaded, or further back is each time the server starts . . . depends how dynamic these custom fields are). Plus you would have to map DB results into a usable listing for your purposes.
Depending how many customers and custom forms you are monitoring, an automated process for that will save you a lot of time and avoid a lot of mistakes of all things hand generated.

Rails REST API permissions (CanCan) with Angular JS client. How to render UI based on permissions?

I'm building a project which works only via JSON API (Rails 4.0, PostgreSQL). It's a big application with database based permissions. And I have AngularJS app which works with this REST API.
Simplified structure:
employees >--- position ---< permission
Employee.rb
belongs_to :position
Position.rb
has_many :employees
has_many :permissions, dependent: :destroy
Permission.rb
belongs_to :position
## Columns
# action (:manage, :read, :update, :create, etc...)
# subject_class
# subject
I have a problem with action buttons/links on AngularJS client side.
For example I don't want to show link "Add Order" somewhere in Angular app because Employee's position permission allows only to read resource and not to modify it:
id action subject_class subject
1 :read Order
How I tried to solve this problem
I create a resource GET api/v1/employees/me which returns current_employee with all his permissions:
"employee": {
...
:position": {
...
"permissions": {
{"id": 1, "action": "read", "subject_class": "Order", "subject": ""},
{"id": 6, "action": "manage", "subject_class": "Waybill", "subject": ""}
}
}
}
So I have all permissions on client side, but what is the best way of beautiful integration received permissions with AngularJS applications UI?

After some digging I think I have found a workable solution. It is a combination of your approach with Jon Samwell's comment above:
http://jonsamwell.com/url-route-authorization-and-security-in-angular/
and Eric Zou's blog post:
http://blog.ericzou.com/2013/05/03/using-angularjs-for-existing-rails-app/
To summarize:
create a rails service that responds with the cancan abilities of the current user on the rails side in json format. Looks like that is what you have started doing with your GET request for permissions. Like this(from Eric's blog) when calling /abilities.json for current user:
{
"manage": {
"User": true,
"Post": false,
...
},
"read": {
"User": true,
"Post": true
...
},
"Update": {
"User": true,
"Post": false
}
...
}
To get the Abilities for a specific user you would do something like Ability.new(current_user) in the controller. It returns an object with a bunch of rules. The rules have conditions which will allow you to limit the access to specific posts.
create a service in angular that handles parsing that works as an abstraction layer between what you get from the rails side(determines if you can access a route or can manage, read, etc a specific model( e.g., can?(:manage, post) ). You can make this service as complicated as you'd like.
also create a directive in angular to handle what to show. Eric suggests using ng-show to do this which is fine, but cleaning it up with something similar to Jon's suggestion: <div access="post" access-type="read" access-options="options">Secret Post</div> makes sense to me.
Make sure that you don't serve up anything from the server that you wouldn't want your users to see after they have hacked your javascript to change their permissions. This is not security, it is UI management.

Angular.js accessing and displaying nested models efficiently

I'm building a site at the moment where there are many relational links between data. As an example, users can make bookings, which will have booker and bookee, along with an array of messages which can be attached to a booking.
An example json would be...
booking = {
id: 1,
location: 'POST CDE',
desc: "Awesome stackoverflow description."
booker: {
id: 1, fname: 'Lawrence', lname: 'Jones',
},
bookee: {
id: 2, fname: 'Stack', lname: 'Overflow',
},
messages: [
{ id: 1, mssg: 'For illustration only' }
]
}
Now my question is, how would you model this data in your angular app? And, while very much related, how would you pull it from the server?
As I can see it I have a few options.
Pull everything from the server at once
Here I would rely on the server to serialize the nested data and just use the given json object. Downsides are that I don't know what users will be involved when requesting a booking or similar object, so I can't cache them and I'll therefore be pulling a large chunk of data every time I request.
Pull the booking with booker/bookee as user ids
For this I would use promises for my data models, and have the server return an object such as...
booking = {
id: 1,
location: 'POST CDE',
desc: "Awesome stackoverflow description."
booker: 1, bookee: 2,
messages: [1]
}
Which I would then pass to a Booking constructor, which would resolve the relevant (booker,bookee and message) ids into data objects via their respective factories.
The disadvantages here are that many ajax requests are used for a single booking request, though it gives me the ability to cache user/message information.
In summary, is it better practise to rely on a single ajax request to collect all the nested information at once, or rely on various requests to 'flesh out' the initial response after the fact.
I'm using Rails 4 if that helps (maybe Rails would be more suited to a single request?)

I'm going to use a system where I can hopefully have the best of both worlds, by creating a base class for all my resources that will be given a custom resolve function, that will know what fields in that particular class may require resolving. A sample resource function would look like this...
class Booking
# other methods...
resolve: ->
booking = this
User
.query(booking.booker, booking.bookee)
.then (users) ->
[booking.booker, booking.bookee] = users
Where it will pass the value of the booker and bookee fields to the User factory, which will have a constructor like so...
class User
# other methods
constructor: (data) ->
user = this
if not isNaN(id = parseInt data, 10)
User.get(data).then (data) ->
angular.extend user, data
else angular.extend this, data
If I have passed the User constructor a value that cannot be parsed into a number (so this will happily take string ids as well as numerical) then it will use the User factorys get function to retrieve the data from the server (or through a caching system, implementation is obviously inside the get function itself). If however the value is detected to be non-NaN, then I'll assume that the User has already been serialized and just extend this with the value.
So it's invisible in how it caches and is independent of how the server returns the nested objects. Allows for modular ajax requests and avoids having to redownload unnecessary data via its caching system.
Once everything is up and running I'll write some tests to see whether the application would be better served with larger, chunked ajax requests or smaller modular ones like above. Either way this lets you pass all model data through your angular factories, so you can rely on every record having inherited any prototype methods you may want to use.