I'm trying to use AWS Personalize. After creating dataset and batch inference, I am updating the user-item-interactions with personalize.putEvents (using Javascript SDK, docs)
Snippet:
const awsOpts = { apiVersion, accessKeyId, secretAccessKey, region }
const pEvents = new AWS.PersonalizeEvents(awsOpts)
// ...
const params = {
trackingId, userId, sessionId,
eventList: [{
eventId: (+sentAt) + "",
sentAt,
eventType,
properties: { itemId }
}]
}
pEvents.putEvents(params, (err, data) => err ? reject(err) : resolve(data))
The events seem to be registered. No errors. After that when I create another batch-inference, I would expect that the new user-items would not appear in the recommendations anymore. But the recommendations in the next batch-inference are unchanged. Am I doing something wrong or am I misunderstanding the putEvents-API-call?
Schema for reference:
{
"type": "record",
"name": "Interactions",
"namespace": "com.amazonaws.personalize.schema",
"fields": [
{
"name": "USER_ID",
"type": "string"
},
{
"name": "ITEM_ID",
"type": "string"
},
{
"name": "EVENT_TYPE",
"type": "string"
},
{
"name": "TIMESTAMP",
"type": "long"
}
],
"version": "1.0"
}
One thing seems a bit strange: Cloud watch reports that the lambda was executed twice despite no errors nor timeout exceeded (timeout is set to 10s, and the lambda takes less than 2s). Also Retry attempts is set to 2.
#D.J.Duff (sadly I can't comment)
Are you sure the events added using PutEvent API are considered without retraining ? I have been through the AWS Personalize Doc looking for exactly that and it looked to me that you need to retrain to get those events included and for the runtime api to be able to consider them. Could you point to me where you saw that they are considered without having to retrain ? Thanks
User-item recommendations, at least from the runtime API should change without retraining after some events (that is what it is for and that is how we use it), though perhaps (it is something to check) you need to use the runtime API to see the new recommendations.
(edited to clarify that that the API is called the "runtime" API - thanks to PatrykMilewski for seeking the clarification - and that I am by no means certain that the particular API used is the important thing - I do know that when using the runtime API, the events do have an effect though).
I have tried out the very same usecase with personalize using boto3 sdk. Yes!! Put events can be used to update user-item-interactions. No need to retrain the model after put events. Looks like aws personalize solutions are made compatible to handle the data updated with put events. When I run the next batch inference, after updating interactions with put events API, I could see the change in the recommendations. I verified this with personalized ranking recipe. I could see the recent interacted items getting re-ranked according users recent interactions.
Related
I have a problem with Azure AD Graph API integration where we try to create user to existing Active directory. Used azure documentation to work on this (https://msdn.microsoft.com/en-us/library/azure/ad/graph/api/users-operations#UpdateUser)
For some reason the example dataset is not working, but I'm receiving:
{
"odata.error": {
"code": "Request_BadRequest",
"message": {
"lang": "en",
"value": "One or more property values specified are invalid."
},
"values": null
}
}
The dataset I'm passing is
{
"accountEnabled": true,
"displayName": "Test User",
"mailNickname": "TestUser",
"passwordProfile": {
"password": "Test1234",
"forceChangePasswordNextLogin": false
},
"userPrincipalName": "Test#_MY_DOMAIN_.onmicrosoft.com"
}
_MY_DOMAIN_ is one of the domains defined in the Active directory's domains.
If I try e.g. removing any of the required fields, I'm receiving error which defines the missing property.
Listing and updating users is working just fine, there is some weird issue with adding.
Usually, we will get this issue when we haven't set the Url in the correct format. I tested your dataset on my side, it worked fine. Please double check the url.
Here is the capture of my test request:
Any further concern, please feel free to let me know.
Most probably problem was related to Azure AD permissions. Somehow started suddenly working.
One thing figured out is that the error message above will also come e.g. if you have property key mistyped. Sometimes response describes the missing property, but not always.
Thanks everyone for your time..
Trying to query against my db to get all docs with all info. The db.list functionality gets the overview of the docs but does not return all the data for the docs. Currently have to get the high level data then loop through the rows and query each individual doc. There must be a better way...
Is there a way to get all docs with the full set of info for each doc?
getting:
{
"id": "0014ee0d-7551-4639-85ef-778f74365d05",
"key": "0014ee0d-7551-4639-85ef-778f74365d05",
"value": {
"rev": "59-4f01f89e12c488ba5b8aba4643982c45"
}
}
want:
{
"_id": "14fb92ad75b8694c05b98d89de6e9b2d",
"_rev": "1-6067c00b37a18ad8bab6744d258e6439",
"offeringId": "ae0146696d1d3a90fe400cc55a97a60e",
"timestamp": 1464165870848,
"srcUrl": "",
"score": 9,
...
}
The repository you linked to for nano looks like an outdated mirror. The official repository includes documentation for db.list which also includes a params object. I would double-check your version of nano, but I would guess you already have a more recent version.
You can simply add { include_docs: true } as the first argument to db.list and alongside id, key and value, you'll get a doc property that has the entire document.
I'm accessing the Facebook Graph API for posts and am trying to figure out the pagination handling. I understand the use of paging.next and paging.previous properties of the results but I'd like to know when there are actually previous results. Particularly, when I make the first 'posts' call, I get back a paging.previous url even though there are no previous values. Upon calling that url I get a response with no results.
For example, calling "168073773388372/posts?limit=2" returns the following:
{
"data": [
{
"story": "Verticalmotion test added a new photo.",
"created_time": "2015-12-02T17:04:56+0000",
"id": "168073773388372_442952469233833"
},
{
"message": "http://www.youtube.com/watch?v=QD2Rdeo8vuE",
"created_time": "2013-12-16T23:19:30+0000",
"id": "168073773388372_184840215045061"
}
],
"paging": {
"previous": "https://graph.facebook.com/v2.6/168073773388372/posts?limit=2&format=json&since=1449075896&access_token=****&__paging_token=enc_AdA69SApv4VoBZB0PPZA7W5EivCYQal8KMFmRNkyhr8ZBk4w0YmFEQUJWV3JZBS70ihyMpbqieQaERhY50enqNCMBuIZATadeopYj8xPvQL7Y8KueaQZDZD&__previous=1",
"next": "https://graph.facebook.com/v2.6/168073773388372/posts?limit=2&format=json&access_token=****&until=1387235970&__paging_token=enc_AdAVMaUlPmpxjBmq5ZClVdNpFp7f9MyMFWjE7ygqsMLW7zvSx3eGHLkfwDxdCx0uO3ooAZCKDmCwMWHZA9RNyxkYUPJyjMtO3kynKm5uF2PhoPZB2gZDZD"
}
}
How can I tell if it's the first set of results?
From tidbits scattered around the documentation and web, it seems like the previous url shouldn't be there.
I don't think it matters because I get the same results in the Graph Explorer but I'm using OpenFB to access the API.
You can set the order to be reverse then get the 1st result
https://developers.facebook.com/docs/graph-api/using-graph-api
Ordering
You can order certain data sets chronologically. For example you may sort a photo's comments in reverse chronological order using the key reverse_chronological:
GET graph.facebook.com
/{photo-id}?
fields=comments.order(reverse_chronological)
order must be one of the following values:
*chronological*
*reverse_chronological*
I have a Node.js app, APP-A, that communicates with another C# app, APP-B, using APP-B's API. APP-B has a RESTful API that returns JSON. Other than a few standard fields e.g., name, description, APP-B's keys are defined when the user creates the field in the system. The resulting JSON looks like this:
{
"name": "An example name",
"description": "Description for the example",
"cust_fields": {
"cust_123": "Joe Bloggs",
"cust_124": "Essex"
}
}
I have two instances of APP-B, a dev and prod environment, which are separate installations. As a result, the JSON from the prod environment is as above, and the JSON from the dev environment looks like this:
{
"name": "An example name",
"description": "Description for the example",
"cust_fields": {
"cust_782": "Joe Bloggs",
"cust_793": "Essex"
}
}
This is dealt with in APP-A (the Node.js app) by having a JSON map like this:
{
"name": "name",
"description": "description",
"cust_fields": {
"full_name": "cust_123",
"city": "cust_124"
}
}
Which is loaded like this:
var map;
switch(env) {
case 'dev':
map = require('../env/dev/map.json');
break;
case 'prod':
map = require('../env/prod/map.json');
break;
};
module.exports = {
name: map.name,
description: map.description,
cust_fields: {
full_name: map.cust_fields.full_name,
city: map.cust_fields.city,
}
}
So I am wondering, is there is a better way of dealing with this? I don't see a way around having to create some kind of manual relationship between the key names across prod and dev, as there is no way to find out what field corresponds to what, but it seems like a lot of work.
Thanks for reading.
Update:
I have created a jsFiddle to better illustrate my question: http://jsfiddle.net/7k9k03o6.
If the mapping is unavoidable and everything is done manually right now, the next best progression would be to automate the building of those lookup maps, through some persistent storage, i.e. a database.
The general flow would be:
When APP-B creates a new form, that field information is stored in the database with all the identifying information. You could store production and dev data in the same db (as a flag) but likely they would just be different databases. Structure might be like customerId, formId, fieldName, fieldMapping, fieldValue, isProduction --> 123, 2, 'cust_124', 'city', 'Essex', true
When APP-A needs a field listing, it queries the DB for the relevant field lists."Find mapping customer X for form Y in production" --> WHERE custId = 123 AND formId = 2 AND isProduction = true would yield a list of fields and their mapping values (which you would post process/reduce into the mapping you need).
This automated process will leave less work for you manually. You shouldn't accidentally miss or forget a mapping from the hand generated file.
This will add a tiny bit of work to the server processing, as you'll need the field mapping from the DB every time a request is processed. (You could back off a bit and do one big query each time a customer is loaded, or further back is each time the server starts . . . depends how dynamic these custom fields are). Plus you would have to map DB results into a usable listing for your purposes.
Depending how many customers and custom forms you are monitoring, an automated process for that will save you a lot of time and avoid a lot of mistakes of all things hand generated.
Is interacting with the database in BreezeJS secure?
For example, if I use the following code, it clearly shows the Database name, tables and the query itself directly in the javascript. Does it make a secure connection to the database?
var manager = new breeze.EntityManager('api/northwind');
var query = new breeze.EntityQuery()
.from("Employees");
manager.executeQuery(query).then(function(data){
ko.applyBindings(data);
}).fail(function(e) {
alert(e);
});
The line "var manager = new breeze.EntityManager('api/northwind');" doesn't says anything about the database. It is the route to the MVC controller ( webapi in this case ).
And the line "var query = new breeze.EntityQuery().from("Employees");" does not have any relation to the database, it's the name of a methd in you controller.
Having in consideration that you can use the mechanics that mvc provides to securize the controller ( like the Authorize attribute ), I don't see any risk using breeze.
The security of breeze.js in the end falls to the programming language used to actually run the queries. As I saw in the docs, it's mainly for ASP.
Checking the TODO sample, doing an action calls /api/todos/SaveChanges with a payload of:
{
"entities": [{
"Id": 2908,
"Description": "Wine",
"CreatedAt": "2012-08-22T09:06:00.000Z",
"IsDone": true,
"IsArchived": false,
"entityAspect": {
"entityTypeName": "TodoItem:#Todo.Models",
"entityState": "Modified",
"originalValuesMap": {
"IsDone": false
},
"autoGeneratedKey": {
"propertyName": "Id",
"autoGeneratedKeyType": "Identity"
}
}
}],
"saveOptions": {
"allowConcurrentSaves": false
}
}
The only sensitive thing there is the Id. Even if you don't use JavaScript you still have to expose some data in one way or another. I'm not saying this is best way of doing it, but this does not have any immediate drawbacks that I can think of. At least not in the JS component.
It falls on behalf of the application (just like in any situation) to sanitize any input from users. This includes any AJAX calls, be it done with breeze or not.
If you can comment with some of the ASP code used to sanitize/run the queries, we can offer more insight on the matter.
So in summary. No issues. JavaScript by itself does NOT connect to the database so it does not have any inherent security issues.