I'm new to CouchDB and I wonder how experienced CouchDB handles the large amount of design documents.
My questions:
Where do you store design documents: In CouchDB with Futon's "save" or on harddrive then uploading it through the HTTP API?
If you are uploading them, how do you do it? You create custom queries or are you using any framework?
I'm using node.js, are there good libraries/frameworks that I could use to ease the CouchDB development?
Please share your experiences!
1) Since the whole of CouchDB works through a REST API, all Futon is doing really is working through the REST interface. Basically, you will always load it through the HTTP API. It's a matter of choice of which tool you will do it with, in which the tool abstracts the process for you. At the end of the day, all a design document is, is another type of document.
2) Personally, since I am coding for it in python, I use the couchdbkit framework, which has auto document loading on it. I am sure there is something similar for node.js. I am unable to answer though.
3) Maybe node-couchdb, although it seems to be no longer maintained. There is alot of material on the internet, just google it!
CouchApp is the best tool for managing/uploading your design docs. Its main purpose is for pure Couch apps but you dont have to use that aspect, it makes design doc managing a breeze (couchapp push).
CouchDB is JSON over HTTP, both things Node does more or less natively. I've found the npm module Nano to work fairly well.
As far as CouchApp goes, there's a couchapp for node.
Related
I'm currently working on a new google polymer web application and wondered if I should use firebase as the backend/db. I took a look at the project, made some test applications and really liked it! But to fully convince me, that firebase is the way to go I need the following questions answered:
I'm a little bit concerned about security: So, I know, that firebase uses read, write and validate to implement server side security. From the samples, I noticed that the validation basically is a one-line JS script, that represents a 'if'. As I'm planning to build a web e-commerce application I need to validate quite some inputs. Is there a possibility, to outsource the validation in a separate file, to make it more readable? Also I wondered, if there is a possibility, to test these server side validations, with for example unit tests?
I'm not 100% sure at the moment, that firebase can cover all of our use cases. Would it be possible/a good solution to use a "normal" backend for some critical functions and then persist the data from the backend in firebase?
I saw some nice polymer elements for firebase. Is firebase 100% supported in polymer/web components?
Is there an other way (like Java approach) to implement server business logic?
Is there a way, to define update scripts, so that new releases can easily be pushed to production?
Thanks & kind regards
Marc
So, I asked the firebase supprt and got the following answer:
Great to meet you.
I'm a little bit concerned about security: So, I know, that firebase uses read, write and validate to implement server side security. From the samples, I noticed that the validation basically is a one-line JS script, that represents a 'if'. As I'm planning to build a web e-commerce application I need to validate quite some inputs. Is there a possibility, to outsource the validation in a separate file, to make it more readable? Also I wondered, if there is a possibility, to test these server side validations, with for example unit tests?
You can implement extremely complex and effective rules using our security rules language. You can deploy security rules as part of your hosting deploy process, or via the REST API. It's not possible to break the contents into multiple files on the server, but you could certainly build your own process for merging multiple files into a single JSON result.
I'm not 100% sure at the moment, that firebase can cover all of our use cases. Would it be possible/a good solution to use a "normal" backend for some critical functions and then persist the data from the backend in firebase?
Generally speaking, synchronizing Firebase and a SQL back end is not very practical and they don't translate well. It's probably entirely redundant as well.
I saw some nice polymer elements for firebase. Is firebase 100% supported in polymer/web components?
I don't know what 100% supported means in this context. We offer a JavaScript SDK so they should play fine together.
Is there an other way (like Java approach) to implement server business logic?
We offer official SDKs in Java, Objective-C/Swift, Android, Node.js, JavaScript, and a REST API for use with other languages.
Is there a way, to define update scripts, so that new releases can easily be pushed to production?
I'm not sure what this means. Most likely the answer is no, since we don't provide a build process or any tools to release your software.
I hope that helps!
I responded:
Thank you for the information, it helped me very much! After reading your response on question number 5 one further question popped into my mind:
…
5. Is there a way, to define update scripts, so that new releases can easily be pushed to production?
I'm not sure what this means. Most likely the answer is no, since we don't provide a build process or any tools to release your software.
Is there like a best practice on how to handle the database schema? I only have one web application (without apps, etc.) in my case... I expect, that the database will change drastically over time and releases. Should I write JS logic, that checks the current database version and update it, if it's necessary? Maybe this would make a nice feature...
For example: I deployed Version 1.0 of my application and everything works fine. After 3 months of programming I notice, that the user data needs a further attribute: address, which is a 'not null' attribute. I now deploy Version 2.0 of my application and every new registered user has a address, but the old users (from Version 1.0) do not have this field or a value.
How should I handle this?
Support responded:
Hi Marc,
There’s no best practice here, but your ideas seem fairly sound. You probably don’t need to check in your JavaScript. You can probably store a version number in the user’s profiles, and when they upgrade to the latest software, you can upgrade that in their profile data.
Then your validation rules could use something like the following:
{
"user": {
".write": "newData.hasChild('address') || newData.child('appVersion') < 4",
"address": {
".validate": "newData.isString() && newData.val().length < 1000"
}
}
}
So if you are concerned about versioning, this could be used to deal with legacy releases.
Another popular approach I’ve seen from devs is to do intermediate upgrades by duplicating data. Thus, you release an intermediate version that writes to the old path and to the new path with the updated data structure (which keeps the app working for old users till they upgrade). Once a reasonable percent of clients are upgraded, then release a final version that no longer does a dual write to the old structure and newer structure.
Of course, flattening data, while it makes joining and fetching data bit more of a pain, will make upgrades much easier as the modular data structure adapts more easily to changes. And, naturally, a pragmatic design where you wrap the various records in a class (e.g. the UserProfile class with getter/setter methods) makes transitions simpler as you can easily hack in versioning at one place.
Hope this helps someone :)
I'm about to start writing an application and I want to include newer technologies to improve my knowledge and gain a skill set.
I'm using mysql to store user data (no choice), Angular.js to display/template content on the dom, and I wanted to use node.js to return results from the database in json format to be used by Angular.js.
My assumption was I could use node.js (along with the node-mysql module and maybe express)to accomplish my database queries that feed into my Angular.js templates.
Additionally, the application will live on a different server than the database itself and may reside in a mobile framework.
Am I way off base when it come to how to use node.js? Am I just using the wrong tool? Am I on the right track?
Any help, explanation, and points in the right direction would be great. Most of the info I've seen are copy/paste from the node.js site/wiki and don't really answer my question.
This is largely an opinion-based question and likely to be closed, but you're not way off base. What you want to do is absolutely (and commonly) done using Node.js.
As a complete aside, you might like to check out Platform-as-a-Service providers such as Heroku.com and Nodejitsu.com; both offer great and easy Node.js hosting and addons (such as NoSQL and RDBMS data stores) which have free tiers.
For that matter, MongoLab and MongoHQ both offer free tiers independent of Heroku, which you could use from your MySQL-only hosting provider if you like.
After past experience with Laravel and Backbone, I've recently gotten into contact with Ember.js and Meteor (for different projects). I really like Ember.js for its structured, route-based approach and for its open and forward-thinking community. On the other hand, I really like Meteor for its simplicity; how fast you can get something working.
There is one thing that I really love about Meteor, which is the DDP Protocol. The reason I ended up using Meteor for a project was because I wanted the best Websocket solution, and DDP's simple combination of RPC for client->server and PubSub for server->client is absolutely brilliant.
Would it be possible to connect Ember.js with a Meteor backend, and how would I go about doing this?
There's some work done for AngularJs: https://medium.com/#zfxuan/the-wonderful-duo-using-meteor-and-angularjs-together-4d603a4651bf
The best option that you have currently for Emberjs is to create your own solution.
I've a background in AngularJs but I don't like the idea of using AngularJs + Meteor. The same goes for Emberjs or any other frameworks that are redundant. By adding AngularJS/Ember/others you're increasing your project complexity. Meteor is great because every plays nice together but there is still a lot of work to be done by the MDG or by community packages.
You can simply use DDP in your application but then you're going to reinvent the wheel. Meteor already handles DDP, in-browser database (minimongo), the UI is reactive (Blaze), the auth system is integrated at the client and server, etc, etc, etc.
You can use 3rd party plugins with your Meteor app but trying to replace parts that Meteor already provides is the quickest way to have a really hard project to maintain. But, in the end, it's just my opinion on this topic and you should use whatever makes you happy.
Maybe you should consider sails.js: http://sailsjs.org/ it has support for SQL systems and probably serves best your purpose.
You can indeed connect other apps to a meteor backend without using meteor on the client.
Meteor has a full wiki page on different libraries. I think that is the most up to date source for information on libraries to connect to a DDP meteor backend.
A link to the JavaScript section: http://meteorpedia.com/read/DDP_Clients#JavaScript
This question's answers are a community effort. Edit existing answers to improve this post. It is not currently accepting new answers or interactions.
I was wondering how one would go about developing a website from scratch with Node.js.
I understand how I could possibly do it, but I am interested in the best design practice.
I need this theoretical website to:
Do a lot of AJAX
Be very straightforward
Be relatively small
Connect to... let's say a MySQL server
In PHP, building a pretty small website was very straightforward - I set up PHP on Apache and a MySQL server and then do something like:
includes/db/ which has connect.php for connecting to the db, a file with common db related functions and so on
includes/layout/ which had stuff like footer.php, header.php, and other layout related stuff
includes/users/ to handle user related actions
Then PHP just let you build pages and include these files together to form a website - I could go something like:
<?php
require_once('inclues/users/user_session.php');
require_once('inclues/db/connect.php');
require_once('inclues/design/header.php')
?>
// Other php or html or related content relating to the page
<?php
require_once('inclues/.../footer.php');
?>
I was wondering what might be similar in Node.js - I am looking for a way to accomplish this which is as simple, fast and straightforward as possible.
If the answer is not simple, I would love a book recommendation, I don't mind reading.
I love event based programming, I really love JavaScript's abilities and I'm really excited about Node.js. I want to learn how to develop this sort of stuff with it the right way from the start.
To start with the bad news: As Node.js is a pretty young technique, I think you'll find that the proces of creating a full fledged website and maintaining/operating it will be very different than what you're currently used to.
Josh3736 adds: Once you figure out how Node.js and its various packages (Connect, Express) work, I found that you can develop new sites very quickly.
The rough edges that currently exist in Node.js, combined with the fast pace of its development and all modules involved can complicate things though, and make things less simple, fast and straightforward than you'd like.
Having that out of the way, here's the good news:
The Node Package Manager, NPM has a lot of good tools and frameworks to expand Node.js's bare bones functionality, making it suitable to create a webserver.
Most notably would be the Express Framework which contains almost everything you need to run a webserver (including cookies, sessions and path routing). Additionally Express supports partials, which take care of your header and footer includes.
Express is built on top of Sencha's Connect. Cookies and sessions are actually powered by Connect. Express is what simplifies your routing and handles views/partials. So if you don't need all bells and whistles that come with Express you could just go for Connect instead.
If you like to use templates for these partials, the Jade Template Engine can speed things up for you. Though Josh3736 points out that Jade is slow and whitespace-significant. A more complete overview can be found here, which includes his favourite, doT. (I personally use Node.js for socket.io based applications only, so he's a better source than me when it comes to templating).
You can connect to MySQL from Node.js using the db-mysql module, but if you don't need that because you're accessing data connected to an already present system, I'd advise to use a more... 'modern' approach, which is to use a NoSQL database as most Node.js projects seem to do. MongoDB via Mongoose is the popular way to go.
Or if it's just storing objects you're interested in, just go for Redis instead (which you're probably going to need at some point anyway).
Once your website is complete, you'll have to deploy it and make sure it keeps running. There are many ways to do so, like using built-in cluster support or use the more feature-friendly forever npm module. See this SO question of mine for more information.
Conclusion:
What I'm trying to get at is this:
Asking what the best practice for building a website in Node.js is, is about the same as asking what the best way to build a website in PHP is: 100 developers will give you 100 different answers.
NPM is blessed with a variety of excellent frameworks that greatly simplify a lot of tasks involved, but it's all based on preference which one is the way to go really.
As I've said, Node.js is still a pretty young technique, so none of the frameworks or additional tools have emerged as 'defacto standard' yet; for most things you're trying to do there are probably various alternatives, and expect your code to break when using most of them during updates, because development of Node.js itself and most modules is fast paced. You'll have to keep up.
Putting it all together:
As I've said, my main production use for Node.js is to be able to use socket.io, so I don't have any good production examples present (And as I'm about to leave on a well-deserved vacation I don't have the time to put one together either). There are some good examples though:
Setup and deployment using Express and Jade
A very complete blog example using Express, Jade and MongoDB
Combining Restify (an extension of Express), Backbone.js and Mongoose
Again, the way to go (and subsequently the example to follow) depends greatly on your ultimate goals and the techniques chosen, but luckily there are plenty of resources available for all of the choices available. Most modules use well documented GitHub repositories and include examples in combination with the most popular modules (See the /examples/ dir that seems to be present in most repositories).
(thanks to Josh3736 for rectifying my errors.)
On the frontend Im using Sproutcore.
The question is how the backend is going to look like.
Solutions:
CouchApp
Node.js between Sproutcore and CouchDB
Could CouchApp really replace Node.js on the backend regarding?
Websockets/streaming
Asynchronous processing
Collection of third party libraries
Security
Speed
CouchApp is nothing more than an easy way to deploy code to your CouchDB. It doesn't expand it's functionality but let's you use what's given easy way.
So it's not CouchApp vs Node.js but rather what doesn't CouchApp have from the list of features you need to build your app.
If you need websockets/streaming go with Node.js middleware. CouchDB doesn't have 'em.
And if you give some details about your project I can try to give more complete answer.
I would recommend you watch Mikeal Rogers talk on how CouchApp and Node.js work together. http://jsconf.eu/2010/speaker/nodejs_couchdb_crazy_delicious.html
Keep in mind that a couchapp is, at it's very basic level, simply a file structure within Couchdb (_design) where your js (or other) code resides and is served from. The key benefit is that any code changes in one place will be replicated to all couchdb instances. It makes it a lot easier to maintain widely distributed code.