I have struggled to find many resources on this online. I am developing an application that multiple users will be using at the same time. This means that one user may edit the database after another user has loaded the data from database. This means that this second user will not have an up to date view of the current state of the database. What is the best way to subscribe to database changes and deal with them. I am using a MEAN stack.
If you are trying to develop a real time system where changes are reflected instantly upon changes in database, you need to make use of web sockets. Since you are using Node.js as backend, see Socket.io
A good resource for implementation can be found here
However, if you plan on implementing web sockets, you will have to make significant changes to both your Node.js and Angular code.
Another method (which I would not recommend) is to make periodic api calls for those views which you want to reflect real time changes. You can make use of setInterval for this
I have two Bokeh apps (on Ubuntu \ Supervisor \ Nginx), one that's a dashboard containing a Google map and another that's an account search tool. I'd like to be able to click a point in the Google map (representing a customer) and have the account search tool open with info from the the point.
My problem is that I don't know how to get the data from A to B in the current framework. My ideas at the moment:
Have an event handler for the click and have it both save a cookie and open the account web page. Then, have some sort of js that can read the cookie and load the account.
Throw my hands up, try to put both apps together and just find a way to pass it in the back end.
The cookies idea might work fine. There are a few other possibilities for sharing data:
a database (e.g. redis or something else, that can trigger async events that the app can respond to)
direct communication between the apps (e.g. with zeromq or similiar) The Dask dashboard uses this kind of communication between remote workers and a bokeh server.
files and timestamp monitoring if there is a shared filesystem (not great, but sometimes workable in very simple cases)
Alternatively if you can run both apps on the same single server (even though they are separate apps) then you could probably communicate by updating some mutable object in a module that both apps import. But this would not work in a scale-out scenario with more than one Bokeh server running.
Any/all of these somewhat advanced usages, an working example would make a great contribution for the docs so that others can use them to learn from.
I work on a web app which store projects data. Data are saved in a couchDb database A. The app pull and push data with a local pouchDb database B, which is sync with A.
So the app can also work offline. When user has connection back, changes made on localDb B during offline time are sent to A using a classic replication.
I store 1 document per project in couchDb, it is a big JSON object with lot of data (project todos, collaborators, advancements, risks, problems, etc...).
It is working like a charm, but I have some problems, and it seems I use pouchDb in wrong way. Situation example:
User A is offline and he adds a todo on project 1.
User B is online and he adds a new collaborator on project 1.
User B changes are pushed to couchDb by the automatic sync.
The project 1 _rev has been incremented.
User B pulls its own changes from couchDb, because the app downloads all documents on any couchDb changes detected. Weird... Idk how to prevent that. But the app still work fine so it's not a big problem.
User A gets its connection back.
User A changes are ignored because of older _rev. But the user did a modification on a different project property, can couchDb detect that himself and merge with newer _rev ?
I clearly see my problem is I'm using 1 document per project. I could use thousands documents to store each properties of each project and my problem woudn't happens, but it seems quite weird: To retrieve all data of a project I would fully scan my database, check document type (collaborator, todos, ...?), and check if the document is linked to the project by adding a new _projectId property to any document.
Currently I just have to request one document, which contains all project data, then I manipulate my JSON easily. It's quite convenient to handle.
How to manage this ? A project may contains averagely 10 to 10 000 properties that multiple users can edit being online or offline.
But the user did a modification on a different project property, can couchDb detect that himself and merge with newer _rev ?
PouchDB/CouchDB conflict handling is described in the PouchDB guide: http://pouchdb.com/guides/conflicts.html
the app downloads all documents on any couchDb changes detected. Weird... Idk how to prevent that.
This is standard PouchDB/CouchDB behavior - you asked it to sync the whole database, so it synced the whole database. :) You can prevent it by using filtered-replication: http://pouchdb.com/api.html#filtered-replication.
How to manage this ? A project may contains averagely 10 to 10 000 properties that multiple users can edit being online or offline.
It really really depends on your data, how frequently it may change, what the unique identifier of a single "property" is... Storing 10,000 separate documents in PouchDB/CouchDB is not a crazy idea, though, and may help you out when it comes to conflicts, since only those individual documents can ever be in conflict.
In general, I'd recommend you read the guide to conflict resolution as described above and review your options. There's also a plugin that may help you with conflict resolution: https://github.com/jo/pouch-resolve-conflicts
I am in an on-going 'debate' with a co-worker as to "What" the view entails in the MVC "pattern". We both come from the web world - so we've both been trying to 'fit' the 'strict' MVC concept into the "model 2" (which I just became aware of) world.
He contends that WHATEVER is coming from the model IS the view... be it JSON, EDI, HTML etc. it's what ever the "end consumer" consumes, be that a browser, or another system...
I contend that the UI is the view, HTML in our case...or it could be FLASH or some other UI. But that JSON or some other formatted data being transferred to another system or browser isn't the VIEW in MVC, but rather data packets just being transferred and consumed, most likely by another controller...to then be passed to a VIEW, or different MODEL.
**SERVER** (Model - business logic)
**CONTROLLER** (server side script, or javascript on client - seems the roles are split these days)
**VIEW** (browser, either web, vector or some other UI)
And am I correct in thinking that the controller(s) are divided between server and client - especially in the web app world??? or is this just a case of MVC architecture on the server being separate from MVC on the client... as there are JS MVC frameworks...
And lastly - does writing an WEB API imply a use for MVC - if it is going to be consumed by a JS app on the client???
Thank you all for you expertise and input.
ADDENDUM - Thu some of these discussions it becomes clear that what used to be MVC on one system, expaned to MVC in the web, which pushed the V out of direct connection with the MVC, not only that but the client side has matured enough that client MVC systems are becoming the norm... And as the 'device' becomes ist own MVC, is it the VIEW that communicates with outside systems or is it the MODEL that communicates with a different systems controller... I think it's starting to make sense if I start encapsulating systems...
Neither you nor your co-worker is explicitly correct, although your issues seem to stem from conceptual misunderstandings of the pattern, as I will attempt to illustrate through this answer. The bottom line is, the view is the view, the model is the model, and the controller is the controller. If you are going to take an extremely strict position on MVC, then you need to treat each component of the pattern as a valid participant in the pattern and not sully the view component by representing it as a simplistic but ultimately disregardable member. (I'm responding here specifically to the notion that the view is simply the return value of the model.)
More confusion presents itself taking this question into consideration:
does writing an WEB API imply a use for MVC - if it is going to be consumed by a JS app on the client?
The model-view-controller pattern focuses on a separation of concerns; specifically, it concerns itself with separating the way in which domain data is processed from the way in which it is presented to the end consumer. The controller is responsible for mediating between the two: that is, it notifies the model of changes made in the view and notifies the view of changes made in the model. (See further: Martin Fowler's formal description of the pattern in Patterns of Enterprise Application Architecture, p. 330)
Crafting a Web API does not remove the applicability of the MVC pattern. Usage of the MVC pattern is specific to the architecture of the application and not to the architecture of the system as a whole. All of the applications that form your system may use the MVC pattern, but it is just as applicable that one application uses the MVC pattern while another application accesses a datastore from the same layer in which it has presentation logic.
To be clear, I am defining an application in this context as something which can be loaded into a runtime environment as opposed to a system which is a set of inter-related applications which may or may not be executed in the same runtime environment or even on the same hardware. I should also point out that I will be using the terms 'paradigm' and 'pattern' interchangeably, although the word 'architecture' is specific to its context.
An Example
As an example let's assume that I am crafting a system that allows a user to create and manage blog posts. To do this, I am going to write two applications: one that is executed on the server that exposes a RESTful Web API and one that is hosted on the server but executes on the user's computer in the user's browser as a single page application (SPA). Both the RESTful Web API and the SPA will be composed using a model-view-controller architecture, so I will present them in those formal terms.
In the Web API application, I have my model layer which represents the domain model (in this case, blog posts and users). I also make my model layer aware of how the domain model maps to my relational database through the use of an object-relational mapping library. My model layer is responsible for persisting and retrieving objects from the datastore (that is, my database) and handling the necessary domain logic related to those objects (e.g., throw an error if the post title is too long).
In the Web API application, I have my controller layer which accepts incoming HTTP requests and connects them to individual controllers in the controller layer implemented as methods. One controller might accept incoming HTTP POST requests for a specific API endpoint with a JSON payload representing a new blog post. The controller deserializes that payload and passes the raw data to the model layer to be validated and persisted (that is, written to the database). The model layer has to inform the controller layer about whether it could successfully validate and persist the data it was given. This outcome is then sent to the view layer for final presentation to the consumer.
In the Web API application, I have my view layer which is responsible for taking the result of the controller's operations and returning that in some meaningful way to the API consumer. In this case, my view layer is responsible for returning some type of HTTP response to the consumer. If the controller described in the paragraph above passes on to the view layer that the model layer could not successfully validate the data, along with the reason it could not validate, the view layer is responsible for crafting a payload that describes the error (and possibly for setting the appropriate HTTP status code on the HTTP response, although this could be the responsibility of a higher service layer that handles returning the view to the consumer). On the other hand, the controller being called might have been asked to retrieve a certain blog post. The controller passes on the data it received from the model layer to the view layer. The view layer transforms the data it received from the controller into a JSON payload which forms the body of the response.
In the SPA, I have a model layer which represents my domain model (again, blog posts and users). I also make my model layer aware of how the domain model maps to my Web API through the use of a RESTful AJAX request library. My model layer is responsible for persisting and retrieving objects from the datastore (that is, my Web API) and handling the necessary domain logic related to those objects (e.g., throw an error if the post title is too long).
In the SPA, I have my controller layer which is responsible for responding to user input by informing the model layer and view layer of changes through individual controllers implemented as methods. One controller might handle the user's navigation to the blog post creation screen by informing the view layer to update itself with the blog post creation form. Another controller might handle the user's request to save that post, in which case the controller passes the blog post data to the model layer to be validated and persisted (that is, sent to the Web API). The model layer has to inform the controller layer about whether it could successfully validate and persist the data it was given. This outcome is then sent to the view layer for final presentation.
In the SPA, I have my view layer which is responsible for taking the result of the controller's operations and performing a transformation on an HTML template file which is then presented in the browser window. Certain controller operations might not necessitate any transclusion of data into the template; that is, the HTML template file is presented directly to the user without changes. Just the same, the controller operations might pass along data which need to be transcluded into the template by replacing certain attributes in the file. The template files might also include certain UI elements that call on a controller in the controller layer to activate.
This example was designed to illustrate three things:
The model layer is responsible for the domain model and the domain model's business logic. It is not necessarily even responsible for persisting and retrieving such models, as this may be outside the specifications of the software. In the event that the software is responsible for persisting and retrieving such models, it is the responsibility of the model layer to communicate with a persistence layer. The persistence layer handles the saving of the data to a place external to the application (e.g., a relational database, an Exchange server, a NoSQL database, the file system, a Web API, a SOAP service, a Core Data store, the master boot record, my fridge, the International Space Station, etc, etc, etc). The model layer doesn't actually care about what is storing the data or where. It just knows what the data looks like and what it can do with it.
The controller layer is responsible for accepting consumer input and mediating between the model layer and the view layer as necessitated by that input. The controller layer is not responsible for performing business logic; it is not responsible for retrieving or persisting data with an external store; and it is not responsible for rendering the view. A separate service layer may handle sending input to the controller layer, depending on the architecture of the system.
The view layer is responsible for presenting any data resulting from the controller layer in some way that is expected by the consumer. This could mean redrawing the UI or sending an HTTP response to a consumer. The controller may pass along certain data that informs the view layer about what the consumer wants; for example, a Web API consumer might set an HTTP header of Accept: text/plain, and it is the view layer's job to transform the raw data passed in by the controller appropriately so that it meets the demands of the consumer's request. It is not responsible for interacting with the model layer (any data the view layer needs has to be passed in by the controller). It is only responsible for transforming the data passed in by the controller into the format expected by the consumer and presenting that formatted data to the consumer. A separate service layer may handle the actual presentation of the view, depending on the architecture of the system.
You, Your Co-Worker, and the Model-View-Controller Pattern
Now that I've built this foundation, I'll step back to your original question and examine why neither you nor your co-worker is correct. After that, I will address the relation of MVC to Model 2.
Let's begin with your co-workers postulation about the view and its relation to the model:
He contends that WHATEVER is coming from the model IS the view... be it JSON, EDI, HTML etc. it's what ever the "end consumer" consumes, be that a browser, or another system...
As I showed above, the model and the view are two separate layers of the application. The model is not returning JSON, EDI, or HTML or any other type of formatted data. The model is returning encapsulated data that only makes sense to the application in its run-time environment because it is only bytes stored in ephemeral memory that represent a model of the actual data that can be consumed by the controller. In plain English, the data returned by the model is not meant for final consumption by the end-consumer or end-user. Typically, for an application constructed in an object-oriented language for which a MVC architecture is used, the model layer will be returning an instance of a class, i.e., an object, not EDI or HTML. (A JavaScript-based model layer might return valid JSON, but that's by virtue of JSON being a sub-set of JavaScript and not the model formatting the data as JSON).
Your co-worker is correct however, in that the view is the JSON, EDI, HTML or whatever the view layer is returning, including a UI redraw. His problem is in thinking that these representations are being generated by the model layer when in reality they are being generated by the view layer based on data passed from the model to the controller to the view. (The view should not even be aware of the model layer.)
I contend that the UI is the view, HTML in our case...or it could be FLASH or some other UI. But that JSON or some other formatted data being transferred to another system or browser isn't the VIEW in MVC, but rather data packets just being transferred and consumed, most likely by another controller...to then be passed to a VIEW, or different MODEL.
The UI is the view only when the UI is being generated as a component of the view layer and not as a component of another application. For example, if you have a Ruby on Rails application that renders an HTML page from the view layer, then yes, the HTML being generated is the view. But an Adobe Flash application is not a view (unless your view layer is constructing the Flash application dynamically and then returning it to the consumer). The Adobe Flash application falls outside the scope of the original application and is an application in its own right. Therefore it is not part of the original application's MVC architecture. However, an Adobe Flash application can have an MVC architecture internally.
JSON content can be a view as long as it is output by the view layer. It does not matter if this content is being transported to another application or system or being consumed by another application's controller.
SERVER (Model - business logic)
CONTROLLER (server side script, or javascript on client - seems the roles are split these days)
VIEW (browser, either web, vector or some other UI)
And am I correct in thinking that the controller(s) are divided between server and client - especially in the web app world??? or is this just a case of MVC architecture on the server being separate from MVC on the client... as there are JS MVC frameworks...
You are not correct in thinking that. It is the case that the MVC architecture in an application executed on the server is separate from the MVC architecture in an application executed on the client. You are trying to describe the model-view-controller paradigm as if it applied to a system. This is a misunderstanding of the model-view-controller paradigm.
The MVC paradigm is implemented within the context of an application. What you are describing in your question is a multi-tier architecture. This can be scaled down to a client-server architecture in which only the server (handling domain modelling, business logic, and request processing) and the client (e.g., the web browser displaying HTML results from the server) exist. It depends on the scope in which you are viewing the architecture of your system. It is in no way a model-view-controller architecture, though.
As I described above, applications implement the model-view-controller pattern. These applications may be combined in such a way that they happen to form a multi-tier system architecture, such as the example I gave with the Web API and the SPA. Both applications (the Web API and the SPA) implement the MVC pattern. Combined, they represent part of a multi-tier system. However, the system that they form cannot have an MVC architecture.
I think one thing to point out is that the MVC pattern is not something that exists solely in web technology and by virtue of that happened to be included in JavaScript frameworks. The MVC pattern is a widely applicable pattern that simplifies codebase maintenance by segregating the concerns of codebase regions. (That is, I can say that this code should not have any view-specific logic because it is only concerned with the domain model and that code should not have any knowledge of how the domain model works because it is only concerned with how the view is rendered.) In theory, the M part could be extracted out of my application and written as a library which is called as a dependency by multiple applications. One might be a web service and another might be a GUI application but both utilize the same domain model code because the level of abstraction provided by the MVC pattern allows for such simple reuse.
Model 2 and MVC
As regards your debate about Model 2 and the Model-View-Controller paradigm, Model 2 is not congruent to the MVC pattern. The first thing to realize is that the word "Model" in "Model 2" has nothing to do with the domain model referred to in "Model-View-Controller". The word, in this case, is actually a synonym of "Example" and should be thought of as such. It was originally presented as an example of how to use JavaServer Pages. This model eventually became a usage paradigm.
Model 2 only specifies that the dynamic content generation should happen outside the actual construction of that content in a final view-generator and that the final view-generator should not contain business logic. Specifically, a Java Servlet accepts the incoming requests, parses it, performs any necessary datastore communication, and then generates a Java Bean which contains any dynamic data which will be formatted for presentation to the consumer. The JavaServer Page then takes over and accesses the generated Java Bean to compile a presentation sent back to the consumer.
The Model 2 paradigm can therefore be considered to implement the "controller" component of the MVC pattern in that the Java Servlet is responsible for receiving and parsing requests. It can also be considered to implement the "view" component of the MVC pattern in that the JavaServer Page is responsible for rendering the view. Model 2 does not have an explicit "model" component, however. In the original example, the Java Servlet accesses the datastore directly, violating the separation of concerns control enforced by MVC. Model 2 only separates out the concern of generating the view and leaves everything else to the Java Servlet. So Model 2, as a pattern, does not exist in direct parallel with MVC.
This does not mean that a Model 2 application cannot also be an MVC application. If a software architect adds a model layer to an application with a Model 2 architecture, and that model layer is only called from Java Servlets, then implicitly the architecture also meets the criteria of the MVC pattern. The two can co-exist quite easily. Model 2 just happens to have a limited scope of applicability: namely, to applications using Java Servlets and JavaServer Pages.
The controller's job in mvc is to control the interaction between the model (data) and the view (display). Whether that happens on the server or on the client is irrelevant. The view knows nothing about the data other than how to display it. The data should be independent of the view. If you are not using a view then you wouldn't need a model (and really I mean a view model). You would be making a data request then.
I am constructing a standalone application that is comprised of HTML, CSS and JS files. The data that is being used by the application is being loaded from an XML file.
I, however, require the application to use a local database - something that would allow me to load, create and edit the data in this database using Javascript. Then package up the application and send it on (I am using webapp-xul-wrapper for this).
Could anybody give me some advice on how I could achieve this? The majority of solutions I have looked at use local storage or only keep the db table data for that particular session or require server side code.
To clarify, my application has a settings page that I would like to allow to edit my data and then keep that data persistent so that when the application is opened again the data is intact. Furthermore, if I was to send the application to someone else - that data would also be intact. Ideally my app would take its data from a physical file that could be passed around.
I hope this question makes sense!
Many thanks,
G.
I actually ended up using a packager called TideSDK (http://www.tidesdk.org/) which supports SQLLite out of the box and also seems to render my applications layout much clearer.
Many thanks!