I have a page which store data in CouchDB. The page accesses the database directly via javascript, so not much of the logic is hidden from the browser. When creating a new document there is some logic which extracts elements of the data into separate fields so that they can be searched on.
Is it possible to do this logic on the server when creating or updating the documents, or am I stuck doing it before hitting the database?
You have a couple of options.
First, see this question about CouchDB update functions. Update functions receive a request from the browser and can modify them in any way before finally storing them in CouchDB. For example, some people use them to automatically add a timestamp. Also see the wiki page on CouchDB document update handlers.
Another option is to receive CouchDB change notifications. In this case, a separate program (either your own browser, or even better, a standalone program that you run) can query CouchDB for _changes. CouchDB will notify this program after the document is saved. Next, the program can fetch the document and then store any new revisions that are necessary.
To me, it sounds like you should try the _update function first.
Related
I am working on a web app where I have made a call to an API and stored the data using MongoDB. This data gets updated daily so I will need to be able to update the data daily by clicking a button in admin site. What is the best way to approach this?
I am new to using databases so I do not know the best approach. The reason I am wanting to store data in database is so I can store it using Redux or Context API so when someone goes to a page the data will be available faster instead of having to make a new API call (and wasting an API call) every time someone visits a page.
My database contains about 630 documents at a time.
Issue:
I need to update the 630 documents in my database to match the 630 documents coming from API that changes daily so I need to figure out what to query MongoDB to accomplish this.
You can use node-schedule.
It's very much like cron-job. But runs on the node application. Make sure the scheduler runs every 24 hours interval and put this database oprtation there.
Note that Node Schedule is designed for in-process scheduling, i.e.
scheduled jobs will only fire as long as your script is running, and
the schedule will disappear when execution completes. If you need to
schedule jobs that will persist even when your script isn't running,
consider using actual cron.
I ended up finding a way to solve my issue.
Previously I have updated a document one at a time so I used something along the lines of db.collection.update(<query>,<update>) to update one document. Issue I was facing was I needed to update all the documents in collection at one time. So by using db.collection.remove({}) I was able to remove all the documents in collection then used db.collection.create(myData) to add new updated data.
EDIT: I am rephrasing my question again :)
I have an Web Application that uses AngularJS as a client framework. I am currently loading a set of reference data (dependant data sets like Country, State, Region, Codes etc) on every page load. Is there a better way to make the server calls and store the data in a cache. At what point in the Angular life cycle do I make these calls to retrieve and store data?
There are many possible ways to do that. You can make a init call to your api as your page gets loads, and then save the data wherever you want.You can save them using localstorage,sessionstorage, factory, service,constants etc.
I've coded some php which retrieves facebook user ids, stored in my database, and then goes on to request their user names. It all works fine, however this process adds an additonal 1.8 seconds to my pageload, which I would like to avoid if possible.
So I rebuilt the code, and added some javascript which looks for the user ids and then requests the user names. This method runs a lot faster and my page load is 1.8 sec shorter. However in order to make it work I'm adding my app access token to the javascript, which ofcourse is a security black hole
The javascript fb.api call looks like that at the moment
FB.api('/'+userid, {access_token : 'appAccessToken'}, function(response) {
userNameSpan.text(response.name);
});
Is there any other way I can make this work, without falling back to the pure server-side solution which is awfully slow?
I'm getting no answers at all to my questions lately. Anyway, the only other solutions I can think of are ajax calling the php file, which will be requesting the user names itself, or storing the user names in the database along with the user ids.
The ajax solution is problematic for a lot of reasons, mainly because the platform is joomla and the php is a custom module, which means that I will have to either call the file outside of the joomla framework (security issues arise) or build my own helper file for the custom module in order for it to work with the com_ajax component of joomla, which ofcourse is a timesink.
The database solution has the downside that the usernames will not be dynamic, meaning that if a user chnages his/her user name after he/she has been added in the database, the change will not be available to me. I guess I could also store the time the user was inserted in the database and check how much time has passed, and perform a facebook request for his/her username only after a predefined time has passed, and then updating the database if the username has changed. Not quite dynamic, but close enough
If any1 has any other ideas, please don't be shy
Although I'm gonna go for the database solution, I found another solution that can decrease the pageload significantly.
Instead of making an api call for each user, you can do a batch call, requesting info about many users in one call. It is explained here https://developers.facebook.com/docs/graph-api/making-multiple-requests
That would cut down my calls from 6 to 1. However since reading up a bit about facebook names, I realized they do not change that often any more, so the database solution is better for my case
I have a CQRS application with eventual consistency between the event store and the read model. In it I have a list of items and under the list a "Create new" button. When a user successfully creates a new item he is directed back to the list but since the read model has not been updated yet (eventual consistency) the item is missing in the list.
I want to fake the entry in the list until the read model has been updated.
How do I best do that and how do I remove it when the new item is present in the actual list? I expect delays of about 60 seconds for the read model to catch up.
I do realize that there are simpler ways to achieve this behavior without CQRS but the rest of the application really benefits from CQRS.
If it matters the application is a c# mvc4 application. I've been thinking of solutions involving HTML5 Web Storage but want to know what the best practice is for solving this kind of problem.
In this situation, you can present the result in the UI with total confidence. There is no difference in presenting this information directly and reading it from the read model.
Your domain objects are up to date with the UI and that's what really matters here. Moreover, if you valid correctly your AR state in every operation and you keep track of the concurrency with the AR's version then you're safe and your model will be protected against invalid operations.
At the end, what are the probability of your UI going out of sync? This can happen if you there are many users modifying the information you're displaying at the same time. This can be avoided by creating task based UI and following the rule 'one command/operation in the AR per request'.
The read model can be unsynced until the denormalizers do their job.
In the other hand, if the command will generate a conversation (long running operation) between a saga and AR's then you cannot do this and must warn the user about it.
It doesn't matter that's a asp.net mvc app. The only solution I see, besides just telling the user to wait a bit, is to have another but this time synchronous event handler that generate the same model (of course the actual model generation should be encapsulated in a service) and sends it to a memory cache.
Being everything in memory makes it very fast and being synchronous means it's automatically executed before the request ends. I'm assuming the command is executed syncronously too.
Then in your query repository you also consider results from cache, removing it if that result is already returned by the db.
Personally, for things that I know I want to be available to the user and where the read model generation is trivial, I would use only synchronous event handlers. The user doesn't mind waiting a few seconds when submitting something and if updating a read model takes a few seconds, you know you have a backend problem.
I see that eventual consistency is applicable to application only if application environment has multiple front-end servers hosting application and all these servers has own copy of read model. All servers uses same copy of event store.
When something is changed to event store, read model that is used to read result to user must be updated in sync with event store. Rest of servers and read models managed by them can be updated using eventual consistency.
This way result to user (list of items) can be read from local read model copy because it is already updated in sync. No need for special complex fake updates/rollbacks.
Only case when user can see incomplete list is that user hits F5 to refresh list after update change and load balancing directs user request to front-end server which read model is not yet updated (60 second delay), but this can be avoided so that load balancing does not change users server in middle of session.
So, if application has only one front-end server, eventual consistency is not very usable or it does not give any benefits without some special fake updates/rollbacks with read model...
I'm working on a web based form builder that uses a mix of Jquery and PHP server side interaction. While the user is building the form I'm trying to determine the best method to store each of one of the form items before all the data is sent to the server. I've looked at the following methods
Javascript arrays
XML document
Send each form item to the server side to be stored in a session
The good, the bad and the ugly
Depends on your application functionality and requirements, but Javascript would probably be the best way. You can use either arrays or objects or whatever in javascript. It's server independent and it will preserve data over a long period of time as long as client session stays present (browser window doesn't close for whatever reason) but this can be quite easily avoided (check my last paragraph).
Using XML documents would be the worst solution because XML is not as well supported on the client side as you might think.
Server side sessions are good and bad. They are fine if you store intermediate results from time to time, so if client session ends because of whatever reason, user doesn't loose all data. But the problem is that it may as well expire on the server.
If I was you, I'd use Javascript storage and if needed occasionally send JSON serialized results to server and persist them there as well (based on business process storig this data somewhere else than session could be a better solution). I'd do the second part (with sever side combination) only if I would know that user will most probably build forms in multiple stages over a longer period of time and multiple client sessions. but can be used for failure preventions as well. Anyway. Javascript is your best bet with possible server-side interaction.
Preserving data between pages on the client
Be aware that it's also possible to preseve data between pages on the client side. Check sessvars library for this. So even if the page gets refreshed or redirected and then returned all this can be stored on the client side between these events like magic. Marvelous any rather tiny library that made my life several times. And lessened application complexity considerably that would otherwise have to be implemented with something more complex.
I used TaffyDB to store data, and it's just wonderfully easy to implement.
Hope this helps you
You may want to check out PersistJS, which exposes a cross-browser persistent storage object. Of course, being persistent, data stored with this library survives sessions, not just page changes.
The latest version (0.2.0) is here – note the version in the above linked post is 0.1.0.
A combination of #1 (although I'd use objects, not arrays necessarily) and #3 would seem like a good approach. Storing the data locally in the browser (#1) makes it immediately accessible. Backing that up with session-based server-side storage defends you from the page being refreshed; you can magically restore the page just as it was.