I'm trying to build a (as close as it gets) generic REST API to simplyfy all the standard CRUD rest calls. For example I'd like to write one read method
models.{Entity}.findById(id)
.exec(function(err, entity) {
res(err, entity)
}
{Entity} should be dynamically filled with a param from the rest call e.g.:
GET /api/v1/entity/:id/:type
GET /api/v1/entity/1234567890/user
Obviously I could do a semi-generic solution like this:
if (type === 'user') var query = models.User;
But that not really a nice solution in my opinion.
Questions
Is there an easy way to implement this and would this be viable on a bigger application? Cause everything I know about reflections from other languages their performance isn't that great.
Has anyone other recommendations on how I could implement such a framework?
Solution:
just like Daniel suggested I created a Map
var map = new Map();
map.set('user', models.User);
Reflection performance is all over the map when it comes to different reflection techniques, different languages, different language implementations, underlying hardware/OS platforms. While Java has been notoriously poor, other languages incur negligible overhead. TLDR: I wouldn't rule it out unless you have evidence it will really slow you down.
In this case, being JavaScript, I think you can just do models[name.toLowerCase()], it's the same as models.User and so on, but generic.
If there really was a cost to reflection, you could still handle it generically by memoizing the result, i.e. compute it once generically and cache the resulting class indefinitely.
You are trying to solve a problem that can be avoided. If your server would provide a REST API instead of an HTTP API (know the difference!), you would not need to construct URLs because the server would tell you what you can do next through hypermedia controls in the response. Take a read on HATEOAS and hypermedia if you are interested.
As hypermedia is all about the MIME-type of the responses, a generic client is usually built around a single MIME-type. (To name a few: HAL, UBER, Siren, Collection+JSON).
Related
I'm trying to make a generic table-top RPG helper web app. It has to be generic because Wizards of the Coast is very protective of their copyrights. So, in order to avoid cease-and-desists, the system has to be capable of loading arbitrary rules. That said, my buddies and I (along with most any other user) will be using it for D&D.
In D&D, modifiers are based on a character's stats. To get a modifier, you take the stat, subtract 10, divide by 2, and round down.
function getModifier(statValue) {
return Math.floor((statValue - 10) / 2);
}
My app will be capable to loading a game's rules from a .json file. I want to be able to make this modifier function user-definable. The easiest way would be to just eval() whatever they provide in the .json file, but obviously that's a terrible idea due to security issues.
Unfortunately, I can't think of a simple way to get around this in a secure manner. Obviously I could write my own parser, but that's more complicated than I'd like/am currently capable of.
Ideas?
I did a quick google search and found two options - http://mathjs.org and http://jsep.from.so
btw. writing your own parser is not that difficult. You may be able to write one that's sufficient for your purpose in less than 100 lines of code
First, bear in mind that this is all on the client-side. So long as you trust the JSON (it came from your server or it is controlled alongside the application itself), then you can use it. That doesn't completely rule out some trojan inserting malicious rules, but I don't think botnets have learned to play D&D yet.
To literally avoid eval and provide some modicum of security, you can use new Function. It takes a list of argument names followed by the function body, allowing you a large amount of control over how the function is defined and where it can be referenced from. You only need to store the body in the JSON file, allowing you to largely control the parameters and make it difficult to assign to the global scope, minimizing your attack surface (such as it is) quite a lot.
If you want to have fun with this and now your rules will largely be mathematical, you may consider using a tool like pegjs to generate a parser for your rules. Calculators are one of the classic introductions to parsers and compilers, so this could be an opportunity to play with some fun tech. The rules and parsers built by pegjs are very easy to use in a JS project (I have an example using PegJS and ES6 via Babel).
There are two options as I see it. One, if you 100% want to define the function in .json, you will need to parse. JSON does not allow functions, so you need to store the function as a string and eval it.
The better way would be to also provide a config.js ability. Perhaps the .json would have a "configUrl": property, and then expose well known apis, eg RPGHelper.setModifer(someFunction) that you can call from inside the config.js file. The user is then free to define whatever arbitrary logic there.
One solution to your problem is to use sandboxed iframes (current supported by all major browsers
You run user code on a separate page, with limited priviliges, in such a way that the code does not interfere with the rest of the application. You can communicate with the sandboxed page using postMessage() and window.onmessage.
When saving changes in Breezejs, since I have entities with lots of fields, I would like to send to the server (to the Breeze SaveChanges() method) not the entire entities, but instead only a subset of the entity, containing just the properties that have changed. I know that this is what the OriginalValuesMap property inside the entities is there for, it's just in order to reduce the network traffic to improve performance (though it may be a very small improvement). On the official Breezejs' website I couldn't find anything about that, nor on the internet. Thanx
We ran into a similar requirement for a different reason. One section of our app interfaces with a 3rd party API that insists on deltas for puts (putting full entities causes server errors or serious performance issues on their end for whatever reason).
We ended up rolling a new data service adapter to address this, and it was a relatively painless process. We extended directly off of the base AbstractDataServiceAdapter, but you may be able to get away with a custom _prepareSaveBundle on top of whichever concrete data service adapter you happen to be using.
You'd just have to register the custom adapter:
ctor = ->
#name = 'custom_ds'
ctor.prototype = new breeze.AbstractDataServiceAdapter() # or whatever your base is
ctor.prototype._prepareSaveBundle = (saveContext, saveBundle) ->
# Do whatever your base implementation does, but use helper.unwrapChangedValues
# instead of helper.unwrapInstance to get at the delta
breeze.config.registerAdapter 'dataService', ctor
And then bind your entity manager to a data service that uses it:
breeze.config.initializeAdapterInstance 'dataService', 'custom_ds'
ds = new breeze.DataService
adapterName: 'custom_ds'
# plus whatever other properties you need to init
manager = new breeze.EntityManager
dataService: ds
# plus whatever other properties you need to init
But if you're just doing this to shrink your payloads, it's probably not worth the hassle and added brittleness for all of the reasons that Jay Traband called out.
We deliberately decided not to do this, because we felt that the performance improvement was not worth the additional complexity. We made this decision based on several considerations.
It would only be useful for modifications and deletions, additions would still need to carry every field.
In most applications, save payloads tend to be much smaller than query payloads.
Standard HTTP compression makes even the largest of these payloads much smaller.
We have been building RIA applications across a range of technologies for a number of years and in our experience optimizing the save payload rarely gives much in the way of overall application performance gains.
But... please feel free to add this request to the Breeze User Voice. As with other, requests if enough of our users think that this is important, then we will do it.
Just starting to play with breeze.js because of the obvious gains in coding time, i.e. managing to access model data from the server direct within Javascript (I am a newbie here, so obviously bare with!).
In the past I have used the stock ajax calls to get/post data to the server, and I have used a few different client tools in the past to provide some help in querying local data, such as jLinq.
My question is this. Isn't it dangerous to have essentially full model query access in Javascript? I must be missing something because it looks like a really well thought through tool. In the past I have at least controlled what can be sent to the client via the backend query process, and again using something like jLinq etc I could filter the data etc. I can also understand the trade-off perhaps with gaining the direct query/none-duplicating local model problem, so just if anyone could provide some insight to this?
Thanks!
EDIT
Obviously I am not the only one, however I am guessing there is a reasonable response - maybe limiting the data being requested using DTO methods or something? The other question posted is here
It can be dangerous to expose the full business model. It can be dangerous to allow unrestrained querying of even that part of the model that you want to expose to the client. This is true whether you offer an easy-to-query API or one that is difficult to query.
That's why our teams are careful about how we construct our services.
You should only expose types that your client app needs. If you want to limit access to authorized instances of a type, you can write carefully prescribed non-queryable service methods. Breeze can call them just fine. You don't have to use the Breeze query facilities for every request. You'll still benefit from the caching, related-entity-navigation, change-tracking, validation, save-bundling, cache-querying, offline support.
Repeat: your service methods don't have to return IQueryable. Even when they do return IQueryable, you can easily write the service method to constrain the query results to just those entities the user is authorized to see.
Fortunately, you can blend the two approaches in the same service or in collaborating services.
Breeze gives you choices. It's up to you to exercise those choices wisely. Go out there and design your services to fit your requirements.
Breeze isn't meant to be your business logic in that sense. Keeping in mind the rule of thumb that if you do something in Javascript, anyone can do it, you ought to be restricting the visibility of your own service data as needed.
In other words, it's useful for you if you meant to make the data publicly visible anyway. But only expose the entities that you're happy exposing and allowing anyone to query; another way to look at it is that your API becomes a public API for your website (but not one you advertise and tell everyone to use).
I am personally not a fan of doing things this way as there is a dependency created on the schema of the backend implementation. If I want to make changes to my database tables, I now have to take my Javascript into consideration. I also lack in terms of integration and unit testing.
However, it can have its uses if you want to quickly build a website feature on non-sensitive data without having to build the service methods and various layers of implementation of it.
What about when you expose the Metadata? Isn't that considered dangerous. IMHO is not safe to expose metadata from the DbContext. I know you can construct metadata on the client, but the point is to do things as quickly as possible(if safe).
I'm looking for (arguably) the correct way to return data from a XmlHttpRequest. Options I see are:
Plain HTML. Let the request format the data and return it in a usable format.
Advantage: easy to consume by the calling page.Disadvantage: Very rigid, stuck with a fixed layout.
XML. Let the request return XML, format it using XSLT on the calling page.Advantage: the requested service is easily consumed by other sources.Disadvantage: Is browser support for XSLT good enough?
JSON. Let the request return JSON, consume it using javascript, render HTML accordingly.Advantage: easier to 'OO-ify' the javascript making the request. Disadvantage: Probably not as easy to use as the previous two options.
I've also thought about going for option one while abstracting the view logic in the called service in such a way that switching in and out different layouts would be trivial. Personally I think this option is the best out of three, for compatibility reasons.
While typing this, I got another insight. Would it be a good idea to allow all three response formats, based on a parameter added to the request?
If you're looking for a quick solution that should work with most available frameworks, I'd go for JSON. It's easy to start with and works.
If you're trying to build a larger application that you're going to extend (in terms of size or maybe your own API for 3rd party extensions) I'd go for XML. You could write a proxy to provide the information in JSON or HTML too, but having XML as the main source is definitly worth the time and effort while building the app.
Like #John Topley said: it depends.
I'd agree with John Topley - it depends on the application. There's a good article on quirksmode that discusses the advantages and disadvantages of each format that you might want to read: http://www.quirksmode.org/blog/archives/2005/12/the_ajax_respon.html
I think this sort of depends on the level of "ajaxyness" your app is going to have. If your front end is a "rich client", al'a gmail, I'd go with the JSON solution, as you'd have to solve the problem of having client side view generation anyway. If you're using ajax sparingly, to provide simple messages to the user, update a few fields now and then, etc, then I'd go with option 1, since most of your view logic is already on the server.
I think trying to use XmlHttpRequest will be a huge headache, unless its the type of headache you don't mind - to do it properly you're almost reinventing the wheel. Then again, people like to reinvent wheels in their spare time, just to say, "Hey, I did it". Not me...
I would get a framework like prototype or Extjs, that has alot of data loading functions built in for XML and JSON, plus you'll get more predictable results, as the frameworks have event handlers to make sure your XmlHttpRequest succeeded or failed. Plus you get support for all the various browsers.
Different response format shouldn't be difficult to produce.
JSON works best for me, i like to keep OO in js, and don't know how to parse XML well :)
How is your javaScript code organized? Does it follow patterns like MVC, or something else?
I've been working on a side project for some time now, and the further I get, the more my webpage has turned into a full-featured application. Right now, I'm sticking with jQuery, however, the logic on the page is growing to a point where some organization, or dare I say it, "architecture" is needed. My first approach is "MVC-ish":
The 'model' is a JSON tree that gets extended with helpers
The view is the DOM plus classes that tweak it
The controller is the object where I connect events handling and kick off view or model manipulation
I'm very interested, however, in how other people have built more substantial javaScript apps. I'm not interested in GWT, or other server-oriented approaches... just in the approach of "javaScript + <generic web service-y thingy here>"
Note: earlier I said javaScript "is not really OO, not really functional". This, I think, distracted everyone. Let's put it this way, because javaScript is unique in many ways, and I'm coming from a strongly-typed background, I don't want to force paradigms I know but were developed in very different languages.
..but Javascript has many facets that are OO.
Consider this:
var Vehicle = jQuery.Class.create({
init: function(name) { this.name = name; }
});
var Car = Vehicle.extend({
fillGas: function(){
this.gas = 100;
}
});
I've used this technique to create page-level javascript classes that have their own state, this helps keep it contained (and I often identify areas that I can reuse and put into other classes).
This is also especially useful when you have components/server controls that have their own script to execute, but when you might have multiple instances on the same page. This keeps the state separate.
JavaScriptMVC is a great choice for organizing and developing a large scale JS application.
The architecture design very well thought out. There are 4 things you will ever do with JavaScript:
Respond to an event
Request Data / Manipulate Services (Ajax)
Add domain specific information to the ajax response.
Update the DOM
JMVC splits these into the Model, View, Controller pattern.
First, and probably the most important advantage, is the Controller. Controllers use event delegation, so instead of attaching events, you simply create rules for your page. They also use the name of the Controller to limit the scope of what the controller works on. This makes your code deterministic, meaning if you see an event happen in a '#todos' element you know there has to be a todos controller.
$.Controller.extend('TodosController',{
'click' : function(el, ev){ ... },
'.delete mouseover': function(el, ev){ ...}
'.drag draginit' : function(el, ev, drag){ ...}
})
Next comes the model. JMVC provides a powerful Class and basic model that lets you quickly organize Ajax functionality (#2) and wrap the data with domain specific functionality (#3). When complete, you can use models from your controller like:
Todo.findAll({after: new Date()}, myCallbackFunction);
Finally, once your todos come back, you have to display them (#4). This is where you use JMVC's view.
'.show click' : function(el, ev){
Todo.findAll({after: new Date()}, this.callback('list'));
},
list : function(todos){
$('#todos').html( this.view(todos));
}
In 'views/todos/list.ejs'
<% for(var i =0; i < this.length; i++){ %>
<label><%= this[i].description %></label>
<%}%>
JMVC provides a lot more than architecture. It helps you in ever part of the development cycle with:
Code generators
Integrated Browser, Selenium, and Rhino Testing
Documentation
Script compression
Error reporting
MochiKit is great -- and was my first love, so-to-speak, as far as js libraries go. But I found that while MochiKit has very expressive syntax, it didn't feel nearly as comfortable to me as Prototype/Scriptaculous or jQuery did for me.
I think if you know or like python, then MochiKit is a good tool for you.
Thank you all kindly for your answers. After some time, I'd like to post what I've learned so far.
So far, I see a very large difference the approach using something like Ext, and others like JQuery UI, Scriptaculous, MochiKit, etc.
With Ext, the HTML is just a single placeholder - UI goes here. From then on, everything is described in JavaScript. DOM interaction is minimized under another (perhaps stronger) API layer.
With the other kits, I find myself starting by doing a bit of HTML design, and then extending the DOM directly with snazzy effects, or just replacing the form input here, an addition there.
The major differences start to happen as I need to deal with event handling, etc. As modules need to "talk" to each other, I find myself needing to step away from the DOM, abstracting it away in pieces.
I note that many of these libraries also include some interesting modularization techniques as well. A very clear description is contributed on the Ext website, which includes a fancy way to "protect" your code with modules.
A new player I haven completely evaluated is Sproutcore. It seems like Ext in approach, where the DOM is hidden, and you mostly want to deal with the project's API.
Tristan, you will find that when you try to architecture JavaScript as an MVC application it tends to come up short in one area -- the model. The most difficult area to deal with is the model because the data does not persist throughout the application, and by nature the models seem to change on the client-side pretty consistently. You could standardize how you pass and receive data from the server, but then at that point the model does not really belong to JavaScript -- it belongs to your server-side application.
I did see one attempt awhile back where someone created a framework for modeling data in JavaScript, much like the way SQLite belongs to the application. It was like Model.select( "Product" ) and Model.update( "Product", "Some data..." ). It was basically an object notation that held a bunch of data to manage the state of the current page. However, the minute you refresh, all that data is lost. I'm probably off on the syntax, but you get the point.
If you are using jQuery, then Ben's approach is really the best. Extend the jQuery object with your functions and properties, and then compartmentalize your "controllers". I usually do this by putting them into separate source files, and loading them on a section-by-section basis. For instance, if it were an e-commerce site, I might have a JS file full of controllers that handle functionality for the checkout process. This tends to keep things lightweight and easy to manage.
Just a quick clarification.
It is perfectly feasible to write GWT apps that are not server-oriented. I am assuming that from Server-Oriented you mean GWT RPC that needs java based back-end.
I have written GWT apps that are very "MVC-ish" on the client side alone.
The model was an object graph. Although you code in Java, at runtime the objects are in javascript with no need of any JVM in either client or server-side. GWT also supports JSON with complete parsing and manipulation support. You can connect to JSON webservices easily, see 2 for a JSON mashup example.
View was composed of standard GWT widgets (plus some of our own composite widgets)
Controller layer was neatly separated from View via Observer pattern.
If your "strongly-typed" background is with Java or similar language, I think you should seriously consider GWT for large projects. For small projects I usually prefer jQuery. Upcoming GWTQuery that works with GWT 1.5 may change that though not in near future because of abundance of plugins for jQuery.
Not 100% sure what you mean here, but I will say that after doing ASP.NET for the last 6 years, my web pages are now mostly driven by JavaScript once the basic page rendering is done by the server. I use JSON for everything (have been for about 3 years now) and use MochiKit for my client-side needs.
By the way, JavaScript is OO, but since it uses prototypical inheritance, people don't give it credit in that way. I would also argue that it is functional as well, it all depends on how you write it. If you are really interested in functional programming styles, check out MochiKit - you may like it; it leans quite a bit towards the functional programming side of JavaScript.